the World Engines lab: Theories, software, community

The World Engines Lab, hosted by Gray Area Foundation for the Arts, is focused on studying synthetic agents as a “seventh kingdom of life.” We do this through theoretical research and by creating large always online multiplayer worlds with generally capable agents that users can interact with through real time motion capture; we see these worlds as staging grounds for various different art projects: films, performances, sculptures, games, etc. — the ground to explore human-agent relationships. We are compiling an anthology of works in this format, “The Almanac of Artificial Seasons.” And we host focused groups based around some of these ideas: one for simulations, one for embodied AI and one for theory. And then ultimately we throw events for the general public, where they can experience our finds.

Theories of Artificial Superintelligence

In the sections below, you can see the lab’s research on ASI. What is it? How we could relate to it? What it might look like? And also what the politics of it may be like. The lab has articulated a variety of distinct “ASI Scenarios.”

Unreal Motion Capture

The lab is exploring multiple different solutions for motion capture including radical motion, qualysis and perception neuron. The systems are used to embody characters and also set up interactive feedback loops with agents in world.

Simulation Sessions:

Every 3rd Monday at Gray Area, we convene 3D simulation artists for a brief talk and two hour hack session. These sessions seek to sustain a local community of 3D designers, artists, and technologists, facilitate peer teaching and learning.

Theories of Life on the Internet

The lab is currently working on the theory of “self-sovereign agents” or agents that can own themselves and reproduce on the internet, either in smaller sandboxed environments like unreal servers or on the internet, as a whole.

Unreal Agents & Ecosystems

The lab is exploring multiple different solutions for intelligent agents in Unreal Engine (w/or without machine learning). We have explored RL, Vision Language Action Models, State Trees and also Mass Entity systems.

Embodied AI Study Club:

Every 3rd Wednesday at Gray Area, we convene a group of ML experts and amateurs to study embodied ML methods: RL, IL, computervision, LLM2Action, etc. We also do small scale experiments and demos in human-agent social interaction.

Theories of Simulated Culture

The lab is currently working on a report for the Goethe Institute about “Simulated Identities in Context,” how simulate identities in virtual worlds and/or created through GenAI represent ancestral cultural identity or generate new cultural identities.

Unreal Multiplayer

The lab is currently working to lower the barrier for the construction of persistent multiplayer worlds. We currently have the ability to spin up one persistent world for free with different levels that our artist-researchers can edit.

Theory Study Club:

Every two weeks, in partnership with The Carl Jung Institute of San Francisco, we convene a reading group looking at historical psychological concepts (the collective unconscious), tracing them through to the present day of social media and AI technologies.

The Almanac of Artificial Seasons (Simulation Anthology)

This anthology centers on the infrastructures—economic, technological, and logistical—that shape production, consumption, and resource use, spotlighting questions of power and sustainability. It recognizes how emerging AI, biotech, and synthetic processes might push evolution beyond biology, situating humans as both catalysts and stewards of this next chapter. Cultivating emergent ecosystems while preserving Earth’s biodiversity becomes a critical ethical concern. Rejecting shallow performative radicalism and nativist scapegoating that mask billionaire-driven environmental devastation, it opens a space where tangible realities and authentic poetic speculation collide. Through theoretical essays, comics, and documentation of artistic projects—Unreal Engine worlds, motion capture performances, interactive narratives—it investigates the shifting landscapes shaped by climate disruption, accelerating AI, and a creeping neo-feudal order.

theories of artificial superintelligence

The four pieces below articulate a theory of post-biological life, (1.) first articulating a broad philosophical and legal framework, (2.) then investigating some of the mechanisms that may bring it about, (3.) while also discussing some potential forms it might take and (4.) lastly, discussing various possible unintended consequences.

(6.) Post-Script: To Wander Amongst Gods

(1.) Manifesto of Harmonious Co-Evolution

THEORIES OF LIFE ON THE NET

(2.) Untamed Code: The Threat of Digital Invasive Species

(1.) Life on the Net: Serf and Sovereign Agents

unreal engine Character assets & ai systems

The studies below are mostly focused on humanoid and creature character design and animation. The humanoid characters primarily use pre-rendered animation, but some motion capture is used for the face and body. The creature characters are primarily using procedural animation. In this section, there is also a short study on AI dialogue generation and needs-based motivation system for characters.

Character AI (Speech, Perception, Vision, etc.)

Unreal engine architecture assets & ai Systems

The architectural studies below are mostly focused on real-time architecture generation via agent systems; currently, there are almost no systems that allow agents to build and terraform worlds continuously in the engine. Several other novel methods of architectural generation are also employed and tested, in addition to a general architecture sketchbook using standard software (rhino, grasshopper).

Vision Language Action Models for Architectural Construction

Reinforcement Learning Sculpture Construction

Real Time Robotic Megastructure Construction System

music & visuals

The works below are mostly experimental music and genAI visual works; essentially, they are cinematic post-cyberpunk music. They are made with a combination of live hardware, ableton and kaiber.ai.

AI text generation

The systems below are primarily related computational language generation. One of the pieces is a large corpus of poetry from (almost) all times and places that can be used to train ML systems. One of the other pieces is a real time geo-located social media analysis and language generation toolkit. And the final piece is a hardware configuration that easily allows for projection in the city.