Meet the one and only Kung Fu Panda in a unique Mixed Reality encounter where the Dragon Warrior himself, takes time out of his Kung Fu training to teach you some meditative moves. Follow Po's lead using your whole body to copy his moves. Learn all the meditative patterns, with some helpful coaxing by Po and achieve a moment of tranquility atop the misty mountains in the Valley of Peace.
Download Kung Fu Panda: School of Chi on Apple Vision Pro!
Video Showcase
School of Chi was created using Apple Native Tools for the Vision Pro platform, with fully fledged mixed reality interactions which track your movements through a guided meditation experience.
The above showcase video was created by Nathie.
Working from initial concept until completion, I worked in a small team at Nexus Studios to create this experience - namely focusing on Swift development and technical art. The aim of this project was to utilise the capabilities of the Apple Vision Pro and create an engaging and wholesome experience for the upcoming movie: Kung Fu Panda 4.
Technical Art
I had a huge part of developing the visuals utilising Technical Art inside of Reality Composer Pro; namely the particle systems (blossom, sun rays, initial and ending sparkles), visual support (blossom and grass movement) and materials that required lerping from greyscale into full saturation.
Development
Development for this project was conducted using Swift inside Reality Composer Pro. This was quite a bit different from the initial Unity development environment but provided a challenge throughout.
Working with other developers who maintained the core flow and state, I was tasked with tasks such as importing models, creating VFX, animating supporting elements such as the tree etc. Alongside this, I looked at optimisation using RealityKit's metrics in certain heavier scenes such as the fog, as well as creating debug elements to display the FPS, states and statistics for the player.
Video Player
For the second element of the app, we wanted to include a Video Player of the trailer; I was tasked to create this element of the application.
The main issue was the 3D video and Apple's UI. Normal AVPlayer solutions could not run 3D, so I looked into extending this. This video player's control scheme was not accurate to our user experience either, so it was concluded we would look for a custom solution.
I took an Entity and allowed the video to run through it, and added the UI controls onto a 'toolbar' element within SwiftUI. Initially, we used SwiftUI's views but the 360 background overrode the layering on these and hid the controls bar. After experimentation, I discovered the toolbar was retained and this allowed me to create this functionality following the UX.