Infraestructura

SGLang

Open-source serving engine that streams realtime/causal video over WebSocket — incremental frames with reusable KV-cache state.

Visitar el sitio Volver al directorio

docs.sglang.io · listado ≠ recomendación · revisado el 2026-09-03

Operador: Open source · SGLang project
Formato: WebSocket realtime-video endpoint
Modelo: Realtime pipelines: SANA-WM · LingBot World
Acceso: Self-host (open source)
Rol: Serving layer for your own live-gen endpoint

¿Qué es SGLang?

SGLang — best known as an LLM serving framework — has been extended to diffusion with a realtime path: a /v1/realtime_video/generate WebSocket endpoint streams video frames incrementally while keeping a live session whose causal KV-cache is reused across chunks until disconnect. It ships realtime pipelines for world models (LingBot World, NVIDIA SANA-WM) and is the self-hosted serving layer if you want to run your own live-gen endpoint rather than rent one.

Qué hace SGLang

A /v1/realtime_video/generate WebSocket endpoint that streams frames incrementally
Reuses a session’s causal KV-cache state across chunks until disconnect — no regeneration from scratch
Ships realtime pipelines for world models (LingBot World, NVIDIA SANA-WM)
MessagePack framing plus delta-gzip frame compression for low-bandwidth streaming
Open source — self-host the serving layer rather than renting a live-gen endpoint

¿Para quién es SGLang?

Serving your own realtime/causal video models behind a WebSocket API
Building world-model streaming (camera-controlled exploration) pipelines
Self-hosting live generation to control cost and data flow

Ventajas y límites de SGLang

Ventajas

Open source with real engineering — an LLM serving framework extended to diffusion
Reusable KV-cache state is the right primitive for continuous streaming
Avoids per-second vendor markup on always-on live generation

Límites

A serving engine — you still supply and operate the model
Realtime diffusion support is newer and narrower than its LLM serving core
World-model pipelines are research/niche as of Sep 2026

Lo que se dice

Citas de cobertura pública — recopiladas el 2026-09-03.

“Realtime and causal video pipelines generate video incrementally and reuse state across chunks.”
SGLang docs

Preguntas frecuentes

What is SGLang’s realtime video support?

A WebSocket endpoint (/v1/realtime_video/generate) that generates video incrementally and reuses a live session’s KV-cache state across chunks.

Is SGLang open source?

Yes — SGLang is an open-source serving framework, and the diffusion/realtime extensions are part of the same project.

What can I serve with it?

It ships realtime pipelines for world models like LingBot World and NVIDIA SANA-WM, plus your own causal diffusion models.

Alternativas en Infraestructura