Motion and Music go hand in hand. Dance has been a major focus of interactive entertainment over the last few decades, transitioning from the arcades, to the living room, and now into the realms of Virtual Reality. One of these titles, Holodance, is a collaborative multiplayer virtual reality game for the HTC Vive (With PSVR and Oculus Touch Ports incoming) that is currently being developed by narayana games.
We had the privilege of spending some time with CEO of Narayana, Jashan Chittesh, to talk about 3D audio and the impact of the technology in Virtual Reality and beyond!
Q) Can you give our readers a brief background into who you are and your work?
A) My name is Jashan Chittesh, I’m founder and CEO of narayana games, a small independent VR game development studio located in Germany.
As a kid, while listening to some of the old albums of Jean-Michel Jarre (Equinoxe and Oxygene), I had fairly vivid visuals that quickly turned into little stories that back then, I shared verbally with my parents. I wasn’t satisfied with just consuming music and trying to describe what I saw. So as a teenager, after creating my first compositions using a tracker on the C64 and later the Amiga, I started building a little home-studio. Also, I learned a little 3D modeling and animation and had scripts ready for full videos for some of my songs, some of which I even started producing but never finished due to lack of skills and technology that just wasn’t where I needed it, yet. So for quite a long time, I shifted my focus very much into software-engineering. Waiting.
Interestingly, in my freelance software-engineer invoice template, that I had created around 1995, I already had “Virtual Reality” next to “Software-Engineering”. But it wasn’t until 2007 that I started actually developing my first game, after I had found out about the Unity game engine (and got myself a Mac because that was the only platform Unity ran on back in those days).
In 2011, I founded narayana games, as a very low priority side project at first. 2012 I backed Oculus and did some small experiments with DK1 and later DK2. Finally, in 2015, after I had heard about the Vive, I knew now was the time to go full-time with VR development, so that’s what I did, and hence Holodance was conceived and is now in the process of being fully born for HTC Vive, Oculus Rift with Touch controllers and PlayStation VR.
Q) What excited you about the OSSIC X and 3d audio technology ?
A) To me, sound, space and visual form really belong together to create any experience we can think of. Haptics are currently mostly missing, smell and taste I find to be secondary senses that are nice to have but really not necessary. So for audio, what I find important is to be able to locate where it’s coming from: Is the source somewhere in the distance ahead of us, or directly behind our head - or do we actually hear something inside of our head? To create a full audio-visual experience that also connects visual stimuli with what we hear, we need technology that convincingly lets us locate sound anywhere in a virtual space.
Unfortunately, for me, personally, probably due to the headphones and ears I’m using, localization wasn’t convincing enough with any of the generic HRTF solutions I have tried so far. So I’m very excited about OSSIX X because I’m confident that a solution that is customized to each individual user, while significantly more complex to set up, will also finally yield satisfying results.
Q) What is a typical day in the life for you while developing a VR release?
A) Working in my own small independent game studio, I spend a surprising amount of time doing things that are not at all specific to developing a VR release: Paperwork, finding people to collaborate with and managing them, distributed around the globe (we work with people from Germany, India, France, England, China, Russia, Netherlands, USA, Canada, Turkey), going through the process of licensing music, handling Visa applications for interns; just to give you an idea.
Equally important is finding ways to gain visibility: While I started Holodance mid 2015 and shared even the earliest prototype videos on Vimeo, YouTube, Reddit, Facebook and Twitter, had a little (failed) IndieGoGo-campaign, and have released the game into Early Access on April 5th (the day when the Vive was released), even today, a lot of people that find out about Holodance think it’s “another game like Audioshield”. A perspective that everyone that has played both games will strongly disagree with.
More specific to VR development is the time spent exploring possibilities and changing direction due to things we find out in the process: The initial idea of Holodance was to create a pure rhythm game without any environments, let alone a story. Just orbs, outer space and light effects. However, after a short time I realized I’d miss a great opportunity if I didn’t let people play with the music in very different environments and let them do something they have never done before. Like … dance with dragons that came to Earth to remind us that we have a duty of care for the planet. But then, in the middle of development I found out through plenty of player feedback that while some people totally love our dragons, there’s a lot more who rather want an intense rhythmic workout experience, which ironically, reduces the scope of the game so much that we decide to give this to our players even before finishing the story mode.
Concerning actual “work on the VR release”, I just recently found out that while playtesting the game in VR, I could bring up the SteamVR dashboard, which automatically pauses the game, go to the desktop (while still in VR), and type some notes into Wunderlist. This solved the rather annoying issue of noticing a bug or something we need to polish while playing the game in VR, but forgetting about it by the time I was back at the flatscreen and keyboard.
So the one thing I can say for sure is that there is no typical day in my life as a VR developer - it’s constantly evolving, shifting and changing. Which is literally a dream that came true.
Q) What are some of the concepts that you want to bring to VR through Audio ?
A) Two keywords would probably be immersion, which is obvious, and synesthesia which may be abusing that term a little but still gets the concept across very well: Virtual Reality has that unique characteristic of making literally anything possible - at least when it comes to hearing and seeing, so one thing we can do and which excites me a lot is making audible things visible, and visible things audible, to create a harmonic audio-visual experience.
Quite a different concept is awareness: In the storymode of Holodance, we take the player through various stages of land, air and water: From paradisiacal to heavily influenced by human civilization. In third version of the water level, we have a submarine that uses a deafening sonar, and in the second air level, we have a few, and in the third many airplanes passing - both distracting the player from the music.
Q) What are some of the challenges VR faces today?
A) The by far greatest challenge is that you cannot explain to anyone what the Matrix is - you need to show them. This is why we spend countless hours giving demos to as many people as we can, hoping to find the one. Well, no, really, to make as many as we can aware of this new incredible possibility. The difference between seeing VR on a flatscreen and actually being in VR is so big that at this year’s Gamescom, after two days of demoing our game to people and seeing it on the Mixed Reality screens that we had set up, going back in myself was so overwhelming that I literally almost cried because I had almost forgotten how beautiful it is when you are in your world, instead of just watching it through a flat window.
Q) What does the future hold for VR and film?
A) Higher resolutions consuming less processing power allowing for higher and higher audio-visual fidelity of the experience. Content that is built with a deeper understanding of what works well, and what works not so well in this totally unique new medium. Haptics so we can finally touch those dragons instead of just hearing and seeing them, or, more seriously, feel the impact of the orbs when we catch them.
Q) Why should VR users try out Holodance ?
A) Because it’s rhythmic, it’s joyful, you can high-five and fistbump a massive dragon, playing this game will keep you physically in shape while making you aware of where our world is currently heading. Furthermore, Holodance is constantly evolving: Even when we have the story complete and we have provided tools to create your own beatmaps to any song you want to play (with support for both simply stereo songs as well as STEMS), we won’t think that this game is really “finished” and abandon it. Eventually, if things go well, you will be able to remix existing music or even create your own - all from within VR, and you will be able to create your own environments to add to the experience. Eventually, I want people to be able to use Holodance to express and share their inner audio-visual world with others - give them the tool that I didn’t have back when I was a child.