Context: Solo research project at Spielworks GmbH (2024) exploring ML-driven creature AI for potential integration into the Spielworks AI platform.

The Concept

What if game creatures didn't follow scripted behavior — but actually learned to fight? Not randomized, not behavior trees, but genuine emergent strategies developed through generational training.

I prototyped this using Chainmonsters-style creatures: autonomous agents that learned movement, positioning, and combat through reinforcement learning, developing their own fighting styles over hundreds of thousands of generations.

Technical Approach

Stack: Unity ML-Agents with external Python neural network

Agent context (intentionally limited):

  • Position and rotation
  • Health points
  • 3 abilities with independent cooldowns
  • Opponent state (same parameters)

Goal: Survive. Defeat the opponent. 1v1 battles.

Curriculum learning: Training happened in phases:

  1. Movement — Agents learned navigation through obstacle courses, with goals and avoidance rules (maintain safe distance from opponents)
  2. Combat — Battle mechanics layered on top of trained movement

This sequencing was critical. Movement competence had to exist before combat made sense.

What Emerged

Early generations tried to close distance immediately — a holdover from movement training where reaching a goal was rewarded. But since direct contact dealt minor damage, this behavior was penalized. Over subsequent generations, agents learned spacing and approach timing.

Training scale:

  • Movement competence: ~100K generations
  • Combat competence: ~500K generations (against a single opponent type)

The limitation: agents were only trained against one creature type, so they developed specialized strategies rather than generalized combat intelligence. Multi-opponent training was the logical next step.

Where This Goes

The prototype proved the concept: creatures can develop genuine fighting styles through generational learning rather than scripted behavior. Applied to a live game, this could mean:

  • Player-owned creatures that evolve unique strategies
  • Background training that continues between play sessions
  • Emergent meta-games as creature populations develop counter-strategies

This remains a research prototype, but the foundation is there.