Senior AI / Real-Time Interaction Engineer

Format: Remote | Full-time / Contract

Project: Confidential AI Product Development

🚀 Key Project Goal: Deliver a working prototype within 8–12 weeks.

Within this timeframe, you will develop the AI interaction architecture powering five distinct interactive characters, fully integrated with the 3D simulation environment. The prototype should support real-time voice conversations, context-aware responses, character-specific behaviors, and AI-driven actions that trigger corresponding animations and interactions. The architecture should be reusable and designed to scale to 60+ characters and additional interaction scenarios.

About the Project

We are developing an AI-powered interactive simulation platform featuring intelligent digital characters capable of natural conversations and context-aware behavioral responses.

We are looking for an engineer experienced in real-time AI systems, conversational agents, multimodal interaction, and event-driven architectures.

Responsibilities

Design and implement real-time AI interaction pipelines.

Integrate speech-to-speech, speech-to-text (STT), text-to-speech (TTS), and low-latency conversational AI services.

Build AI agents capable of understanding user intent and triggering structured actions.

Develop character behavior logic using state machines, rules, and LLM orchestration.

Connect conversational AI with external 3D animation and simulation engines.

Implement configurable character profiles, memory, and interaction scenarios.

Build backend services, APIs, and interaction logging.

Evaluate third-party AI services and SDKs to accelerate prototyping.

Collaborate with 3D developers to establish reliable communication between AI systems and interactive characters.

Required Experience

Strong experience with Python or TypeScript / Node.js.

Hands-on experience integrating LLM APIs into production applications.

Familiarity with streaming APIs, WebRTC, WebSockets, and event-driven systems.

Experience with AI agent frameworks, function calling, and structured outputs.

Understanding of state management and conversational context.

Experience with cloud infrastructure, databases, and backend development.

Ability to design scalable architectures and deliver working prototypes within defined timelines.

Nice to Have

Experience with OpenAI Realtime API, LiveKit, ElevenLabs, or Azure Speech.

Familiarity with Convai, Inworld AI, or conversational NPC systems.

Experience integrating real-time avatars or game engines.

Experience building multimodal AI applications.

Familiarity with agent evaluation and automated testing.

Initial Project Goals

Develop the AI interaction architecture for a prototype featuring five distinct digital characters, with reusable logic that can support dozens of additional characters and scenarios.

The initial prototype is expected to be delivered within 8–12 weeks.

We value engineers who can make sound architectural decisions, leverage existing technologies effectively, and balance implementation speed with scalability and reliability.