Engineering case study / 2025
TERARiUM
Making generated characters work in a live exhibition.
PM · Technical director · Motion capture & retargeting

The problem
TERARiUM has two connected experiences: a visitor's sketch becomes an Echo character, while a photograph becomes a personalized 3D avatar. The avatar follows the visitor's body, and the Echo character responds to its actions. The challenge was bringing generated assets into a real-time environment where several people could participate.
My contribution
I was PM and technical director in a five-person capstone team. I led real-time motion capture and retargeting, built the Azure Kinect-to-Unity pipeline, modified UniRig post-processing, and implemented visitor re-identification. I also integrated the team's sketch input, character generation, 3D conversion, rigging, and motion-response stages.
Making the rig usable in Unity
UniRig produced joints with names such as joint_1 and joint_2. These names did not map automatically to Unity Humanoid, and the mismatch could produce abnormal deformation. A generated mesh and rig were therefore not yet a usable avatar.
I developed a Blender Python post-processing step that traversed the skeleton tree and assigned Humanoid-compatible joint names. This provided an explicit bridge between the rigging output and Unity's expected skeleton structure. The runtime pipeline then used Azure Kinect's 32-joint skeleton tracking to drive avatar motion.
Keeping the visitor attached to the right avatar
With multiple visitors, tracking a body and identifying its owner became separate problems. The active skeleton could become confused with another participant, breaking the relationship between a visitor and their personalized avatar.
I used RetinaFace and ArcFace for face-embedding-based re-identification. This introduced an identity signal alongside skeleton tracking, helping maintain the association between each visitor and their avatar as the exhibition ran.
Outcome and takeaway
The integrated system was used in a live exhibition. It connected a generated character pipeline with motion-controlled interaction, rather than ending at asset generation.
My main takeaway is that compatibility and identity are part of the interaction system. Skeleton structure determines whether an asset can move correctly; identity determines whose movement it should follow. Both have to survive the handoff from an AI pipeline to a running engine.