PhonemeX Pro – Offline TTS, ASR, Lip-Sync, Avatar & Face Rig System

资源介绍

PhonemeX Pro Edition is a modular, fully offline Text-to-Speech, Speech Recognition, and character voice system built for Unity mobile, desktop, and web builds. It provides real-time, on-device speech synthesis, local speech-to-text transcription, and full access to phoneme timing data at runtime, enabling speech generation, lip-sync, facial animation, and voice-driven character systems. What’s New in Version 1.2.0 PhonemeX Pro has evolved into a complete bidirectional voice suite. Version 1.2.0 introduces a brand-new Speech Recognition (ASR) module powered by local Whisper models, and a zero-latency Audio-Reactive Lip-Sync system for fast, FFT-based mouth movements. The entire codebase has also been deeply modularized, separating Expressions from Lip-Sync to allow for perfectly layered facial animation without conflicts. Existing projects remain fully compatible. Core Systems Included On-Device Text-to-Speech System A cross-platform, fully offline TTS engine that runs entirely inside Unity. Generates real-time audio along with detailed phoneme timing data for runtime logic and animation. On-Device Speech Recognition System (New!) A robust ASR pipeline integrating multilingual Whisper models for native speech-to-text inference. Runs completely locally via Unity Sentis (Desktop/Mobile) and WebAssembly (WebGL). Includes a built-in Download Manager UI for easy model fetching. Real-Time Lip-Sync System A dedicated lip-sync pipeline that schedules phoneme data, blends viseme weights, and drives facial motion in real time. Designed for accurate speech-to-animation synchronization. Audio-Reactive Lip-Sync (New!) A high-speed, zero-latency alternative to phoneme lip-sync. Uses real-time FFT band analysis, Automatic Gain Control (AGC), and direct jaw bone rotation for instant audio-driven mouth movement directly from an audio source. Modular Avatar System PhonemeX Pro includes a flexible avatar integration layer compatible with the AvatarX character set. The avatar system applies lip-sync and expression data to fully rigged characters using blendshapes or custom drivers, enabling speech-driven facial animation with minimal setup. The system is designed to work with a wide range of humanoid characters and animation setups, allowing developers to integrate PhonemeX into existing projects without being locked to a specific character format. Face Rig & Expression System An optional facial rigging and expression framework for authoring, blending, and layering facial poses, emotions, and micro-expressions on top of lip-sync animation. Modular Design Each system can be used independently depending on your project needs: Use only the TTS system for speech playback Combine TTS and lip-sync for talking characters Integrate the avatar and face rig systems for full facial animation control Modules are loosely coupled and designed to be extended or replaced as your project grows. Key Features Fully Offline, On-Device TTS – No internet connection or server required by default Real-Time Phoneme Access – Phoneme timing data available at runtime Cross-Platform Support – Android, iOS, WebGL, Windows, macOS, and Linux Lightweight & Fast – Optimized for mobile and browser-based builds Multi-Voice & Multi-Language ASR Support – Easily switch between models Multi-Voice Support – Easily switch between supported voice models Modular Runtime Architecture – Clean separation between voice, lip-sync, and animation systems Optional online phoneme services can be integrated if desired, but are not required for core functionality. Local AI Inference – Uses Unity’s official on-device AI inference packages Demo Scenes Included – Examples showcasing TTS, lip-sync, avatar integration, and face rig workflows Supported Platforms Android (ARM / ARM64) iOS / iPadOS WebGL Windows (x86 / x64) macOS Linux Dependencies / Requirements Unity 6 LTS or newer On-device AI inference package: Unity AI Inference Engine Unity Editor Coroutines package: (com.unity.editorcoroutines) Newtonsoft Json package: (com.unity.nuget.newtonsoft-json, included with Unity) Required packages can be installed automatically via the PhonemeX installer. Perfect For Games and visual novels Digital humans and virtual characters Interactive NPCs and AI agents Educational and accessibility applications Web-based interactive tools and simulations Need Help or Want to Connect Documentation | Support Email Get updates, ask questions, and share your projects with the community. Demo Builds You can download prebuilt demo applications to test the asset before purchasing: Windows Demo: Download MacOS Demo: Download These demos showcase the real runtime behavior of the asset. No login or registration required. PhonemeX Pro is designed as a scalable voice and character animation foundation, from simple on-device speech to fully animated, expressive characters. Created by Carruto – professional tools for AI-driven audio and character systems. Offline, On-Device Text-to-Speech and Speech-to-Text Real-time speech synthesis running entirely inside Unity builds, with no internet connection required for core functionality. Real-Time Phoneme Data Access Provides phoneme timing data alongside generated audio for accurate lip-sync, facial animation, and runtime logic. Modular Runtime Architecture Independent runtime modules for voice generation, lip-sync, avatar driving, and facial rig systems. Cross-Platform Runtime Support Designed for Android, iOS, WebGL, Windows, macOS, and Linux builds. Lightweight & Performance-Oriented Optimized runtime footprint suitable for mobile and browser-based applications. Multi-Voice Model Support Supports switching between multiple voice models (Piper-compatible format). On-Device AI Inference Uses Unity’s official on-device AI inference packages: – Unity AI Inference / Inference Engine for Unity 6 and newer Optional External Phoneme Integration Supports optional external phoneme services for advanced workflows, without affecting offline core functionality. Demo Scenes & Documentation Included Example scenes and detailed documentation covering setup, architecture, and integration workflows. ✅ Works in HDRP inside the Editor, but Web builds with HDRP are not supported by Unity. Use URP or Built-in for Web. In Built-in RP, if UI input issues occur in the sample scene, select the EventSystem and click the “Fix” button in the Inspector to resolve them automatically.