Sometimes the hardware built into a gaming system isn't sufficient to tackle a strange new task. Sometimes you won't be able to produce official accessories to make your game design idea possible.
That's where crazy projects like mine come in! I hooked up a standard PC webcam to an AI image processing model able to communicate with my QuickMoon server. This allowed the game to respond instantly to detecting different objects and events in the camera frame.
In this particular example, I made it possible so that closing your eyes results in getting randomly teleported! A difficult and chaotic challenge that would only be possible by combing unexpected development techniques together to make a product that nobody ever has before.
Built using the Whisper speech recognition library, it's possible to make a microphone do just about anything!
Another member of the QuickMoon development family, this project involved creating a list of keywords that if detected in his speech, would send commands over the server to the game! These could do just about anything, making a truly chaotic challenge playthrough.
Behind the scenes, this project took a lot of tinkering to find the perfect balance on parsing the incoming speech transcriptions.
In the 2nd example video which was created for my own YouTube channel, the same premise applies except that the disallowed words were constantly changing and automatically pulled from the stream's live chat.
This features many of the real time animation and interaction techniques detailed in the previous project. Along with the videos you can see a few behind-the-scenes development screenshots below.