Digital characters in video games are being elevated to new levels of realism and interactivity, thanks to NVIDIA's Avatar Cloud Engine (ACE) microservices. ACE gives developers tools to integrate state-of-the-art generative AI models into digital avatars, allowing non-playable characters (NPCs) to dynamically interact and converse with players in-game and in real-time.
The process begins with providing NPCs with a backstory and purpose to guide the narrative and ensure contextually relevant dialogue. ACE then utilizes various AI models and technologies to enable NPCs to hear, process, generate dialogue, and respond to players.
NVIDIA Riva, a technology that builds real-time conversational AI pipelines, is used for automatic speech recognition (ASR) to transcribe the player's voice. The transcription is then fed into a large language model (LLM), such as Google's Gemma or Meta's Llama 2, to generate a natural language text response. Riva's neural machine translation and Text-to-Speech capabilities are employed to generate an audio response.
Finally, NVIDIA Audio2Face (A2F) generates facial expressions synchronized with the dialogue in multiple languages, displaying dynamic, realistic emotions that can be streamed live or baked in during post-processing.
Leading game developers, studios, and startups are already incorporating ACE into their titles, bringing new levels of personality and engagement to NPCs and digital humans. Ubisoft showcased its NEO NPCs project at GDC, demonstrating NPCs with environmental and contextual awareness, real-time reactions and animations, and conversational memory. Inworld AI and NVIDIA also unveiled the Covert Protocol demo, showcasing AI-powered digital characters that acted as bearers of crucial information and catalyzed key narrative developments.
Beyond gaming, the technology is also being used to animate avatars and digital humans in healthcare, customer service, and other industries. NVIDIA collaborated with Hippocratic AI to showcase a generative AI healthcare agent avatar, and UneeQ integrated NVIDIA A2F into its autonomous digital human platform for enhanced customer experiences.
As AI technology continues to advance, ACE and other NVIDIA AI tools are poised to revolutionize the way digital characters are created and integrated into various applications, offering more immersive and engaging experiences for users.
Are you interested in buying NVIDIA products?
