Technical Research

Main Research Areas

Motion Capture

The lab is exploring multiple different solutions for motion capture including radical motion, qualysis and perception neuron. The systems are used to embody characters and also set up interactive feedback loops with agents in world.

Multiplayer

The lab is currently working to lower the barrier for the construction of persistent multiplayer worlds. We currently have the ability to spin up one persistent world for free with different levels that our artist-researchers can edit.

Agents & Ecosystems

The lab is exploring multiple different solutions for intelligent agents in Unreal Engine (w/or without machine learning). We have explored RL, Vision Language Action Models, State Trees and also Mass Entity systems.

unreal engine Character assets & ai systems

The studies below are mostly focused on humanoid and creature character design and animation. The humanoid characters primarily use pre-rendered animation, but some motion capture is used for the face and body. The creature characters are primarily using procedural animation. In this section, there is also a short study on AI dialogue generation and needs-based motivation system for characters.

Character AI (Speech, Perception, Vision, etc.)

Unreal engine architecture assets & ai Systems

The architectural studies below are mostly focused on real-time architecture generation via agent systems; currently, there are almost no systems that allow agents to build and terraform worlds continuously in the engine. Several other novel methods of architectural generation are also employed and tested, in addition to a general architecture sketchbook using standard software (rhino, grasshopper).

Vision Language Action Models for Architectural Construction

Reinforcement Learning Sculpture Construction

Real Time Robotic Megastructure Construction System

unreal engine Texture, post process, ui

The studies below are mostly focused on textures, materials, post process and UI

music & visuals

The works below are mostly experimental music and genAI visual works; essentially, they are cinematic post-cyberpunk music. They are made with a combination of live hardware, ableton and kaiber.ai.

AI text generation

The systems below are primarily related computational language generation. One of the pieces is a large corpus of poetry from (almost) all times and places that can be used to train ML systems. One of the other pieces is a real time geo-located social media analysis and language generation toolkit. And the final piece is a hardware configuration that easily allows for projection in the city.