Open-endedness

Chapter 12 introduced open-endedness as the aspiration of building learning processes that keep generating new challenges and new capabilities indefinitely. This chapter is the closer look at the field that takes that aspiration as its central research question.

The motivating observation is simple and stunning. Biological evolution has been running for billions of years and is still inventing new niches, new body plans, new ecosystems. It has not converged. It has not stagnated. It produces novelty at every scale, on every timeline. Nothing we have built in ML comes close to this. The question of how to build a process that keeps surprising itself is one of the most consequential research questions in the field — and one of the least solved.

What “open-endedness” means, operationally

The casual sense of the term is “keeps producing new stuff forever.” The operational sense is more particular. An open-ended process is one that:

• Generates novel artifacts (solutions, behaviors, agents) over time, where “novel” is measured against the history of what it has already produced.
• Maintains the capacity to keep generating novelty — i.e., does not converge to a stable distribution.
• Does so without external curation or hand-designed curricula.

Each of these conditions is harder than it looks. Most ML systems converge. Most that do not converge oscillate or collapse. Designing a system that productively keeps moving is the open problem.

The major lines

A non-exhaustive map:

Quality-diversity (QD). The cluster of methods that explicitly optimize for diverse high-quality solutions rather than the single best one. Novelty search rewards behaviors that have not been seen before, decoupling exploration from the task reward entirely. MAP-Elites maintains a grid of behavioral descriptors and keeps the best solution per cell, producing an archive of distinct strategies. QD has been useful for robotics (where many distinct gaits and strategies each have their use), for design exploration, and as a substrate for more elaborate open-ended systems.

Co-evolutionary environment generation. Train an agent and an environment-generator at the same time. As the agent gets better, the environment-generator proposes harder challenges. POET (Paired Open-Ended Trailblazer, Wang et al.) is one early instantiation; more recent variants have refined the basic idea. The hard problem is keeping the environment-generator from collapsing into either trivially-easy or impossibly-hard challenges.

OMNI / OMNI-EPIC and successors. Recent work that uses learned models to predict whether a candidate task is interestingly hard for a given agent — a learned curriculum signal that can substitute for hand-designed difficulty progression. This is an active and fast-moving subfield.

Stanley and Lehman’s broader program. The line of thought that gave us novelty search has also produced a broader philosophical stance, articulated most prominently in Why Greatness Cannot Be Planned — the claim that objective-driven search is a surprisingly limited paradigm, and that interesting discoveries (in evolution, in human creativity, in research) emerge from non-objective-driven processes. Open-endedness is the technical instantiation of this stance.

Why open-endedness is hard

A few structural reasons:

• It is hard to measure. Most ML benchmarks are bounded — a finite test set, a fixed task. An open-ended system, by definition, is producing things outside any pre-specified evaluation. Evaluating an open-ended system requires either an open-ended evaluator or accepting that the evaluation is itself an exercise in taste.
• It is hard to engineer. Most optimization processes converge or collapse. The interventions that keep a system “open” — explicit novelty rewards, environment co-evolution, intrinsic motivation — are themselves prone to gaming, collapse, or stagnation.
• It interfaces awkwardly with goals. A truly open-ended system may produce useful things, but it is not for producing useful things. The tension between “open-ended exploration” and “task-directed performance” is genuine and not yet resolved.

Where the field is in 2026

Open-endedness is a small, productive, mostly research-flavored field. It has not yet produced a system that runs indefinitely without engineering intervention, but it has produced a growing collection of partial successes — systems that stay open longer than naive baselines, that produce qualitatively novel artifacts at intermediate scales, that point at the right structural ingredients.

The intersection with LLMs as creative substrates is one of the most interesting recent currents. Foundation models can be used as generators of candidate tasks, candidate environments, candidate solutions — and several recent open-endedness systems use LLMs in this role, with promising early results.

Where to go next

• The novelty-search and quality-diversity literature (look for Stanley, Lehman, Mouret, Clune and collaborators for the canonical references).
• POET and its descendants for the co-evolutionary environment story.
• Recent OMNI-EPIC-style work for the learned-curriculum side.
• Why Greatness Cannot Be Planned (Stanley and Lehman) for the philosophical positioning.

If you finished Chapter 12 wanting to actually engage with this work — rather than just gesture at it — this chapter is the launching point.