Two projects are in progress: a stealth game on Unreal Engine 5, and a language model architecture trained on one consumer GPU.
A stealth game on Unreal Engine 5. You play a golf ball, and a robot golfer hunts you across an island course, an underground and a metro. What the robot can see and hear decides every level.
Open the devlog →
Heixing is a model architecture that mixes Gated DeltaNet layers with a few attention layers and sparse experts. At one million tokens of context XimO-1 Small keeps a cache of 1.5 GiB. GPT-2 small would need 36.
Read the write-up →