跪拜 Guibai
← All articles
Backend

A 3D Digital Human That Speaks in Your Browser, Driven by a Parameter Stream

By 喜欢吃豆 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Streaming animation parameters instead of video cuts bandwidth and lets a 3D avatar respond to generated text in real time, which is the missing piece for hooking an LLM up to a visual, interactive frontend without perceptible lag.

Summary

A single `speak()` call sends text to the cloud, which returns a synchronized parameter stream for voice, lip-sync, facial expressions, and body movements. The browser renders the avatar locally via WebGL2, keeping the pipeline lightweight and responsive. An open-source performance console wraps the SDK into a zero-dependency demo with streaming broadcast, auto-show sequences, status controls, and real-time event visualization. The underlying infrastructure claims ten-million-level concurrency and can deploy on low-cost chips, with the same SDK targeting screens, humanoid robots, and AR/VR headsets.

Takeaways
Calling `speak(text)` triggers cloud-side synthesis that returns a parameter stream for voice, lip-sync, expressions, and actions, rendered client-side via WebGL2.
End-to-end latency is claimed at under 500ms, making the avatar responsive enough for real-time LLM-driven conversations.
The SDK requires `localhost` or HTTPS; plain HTTP or IP-based access will fail.
An open-source console wraps the SDK with streaming broadcast, auto-demo sequences, status controls, and dual light/dark themes, running with zero dependencies on Node ≥ 18.17.
The same SDK and `speak()` interface are designed to work across web, humanoid robots, and AR/VR glasses.
New accounts using invitation code XDZARL7NEP receive 1,000 free credits on the Mofa Xingyun platform.
Conclusions

The architecture offloads heavy synthesis to the cloud while keeping rendering local, which sidesteps the bandwidth and decoding cost of video-based digital humans.

Claiming ten-million-level concurrency and deployment on hundred-yuan chips signals an intent to serve high-volume, cost-sensitive production environments rather than one-off demos.

The parameter-stream approach decouples content generation from presentation, so the same cloud output could drive radically different visual forms—a browser avatar, a robot face, or an AR overlay—without changing the API.

Concepts & terms
Parameter Stream
Instead of sending a rendered video, the server transmits structured data—speech audio, lip-sync visemes, facial expression weights, and skeletal animation poses—that the client uses to reconstruct the avatar's performance in real time.
AI End-Rendering
A rendering approach where AI-generated parameters are executed on the client device (browser, robot, headset) rather than pre-rendered into video on the server, reducing bandwidth and latency.
LAM (Large Action Model / Large Avatar Model)
Mofa's proprietary multimodal model that generates 3D avatar behavior—speech, expressions, and body movements—from text input, analogous to how an LLM generates text.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗