Infrastructure

SGLang

Open-source serving engine that streams realtime/causal video over WebSocket — incremental frames with reusable KV-cache state.

Visiter le site Retour à l’annuaire

docs.sglang.io · répertorié ≠ recommandé · revérifié le 2026-09-03

Opérateur: Open source · SGLang project
Format: WebSocket realtime-video endpoint
Modèle: Realtime pipelines: SANA-WM · LingBot World
Accès: Self-host (open source)
Rôle: Serving layer for your own live-gen endpoint

Qu’est-ce que SGLang ?

SGLang — best known as an LLM serving framework — has been extended to diffusion with a realtime path: a /v1/realtime_video/generate WebSocket endpoint streams video frames incrementally while keeping a live session whose causal KV-cache is reused across chunks until disconnect. It ships realtime pipelines for world models (LingBot World, NVIDIA SANA-WM) and is the self-hosted serving layer if you want to run your own live-gen endpoint rather than rent one.

Ce que fait SGLang

A /v1/realtime_video/generate WebSocket endpoint that streams frames incrementally
Reuses a session’s causal KV-cache state across chunks until disconnect — no regeneration from scratch
Ships realtime pipelines for world models (LingBot World, NVIDIA SANA-WM)
MessagePack framing plus delta-gzip frame compression for low-bandwidth streaming
Open source — self-host the serving layer rather than renting a live-gen endpoint

À qui s’adresse SGLang ?

Serving your own realtime/causal video models behind a WebSocket API
Building world-model streaming (camera-controlled exploration) pipelines
Self-hosting live generation to control cost and data flow

Points forts et limites de SGLang

Points forts

Open source with real engineering — an LLM serving framework extended to diffusion
Reusable KV-cache state is the right primitive for continuous streaming
Avoids per-second vendor markup on always-on live generation

Limites

A serving engine — you still supply and operate the model
Realtime diffusion support is newer and narrower than its LLM serving core
World-model pipelines are research/niche as of Sep 2026

Ce qu’on en dit

Citations issues de couverture publique — collectées le 2026-09-03.

“Realtime and causal video pipelines generate video incrementally and reuse state across chunks.”
SGLang docs

Questions fréquentes

What is SGLang’s realtime video support?

A WebSocket endpoint (/v1/realtime_video/generate) that generates video incrementally and reuses a live session’s KV-cache state across chunks.

Is SGLang open source?

Yes — SGLang is an open-source serving framework, and the diffusion/realtime extensions are part of the same project.

What can I serve with it?

It ships realtime pipelines for world models like LingBot World and NVIDIA SANA-WM, plus your own causal diffusion models.

Alternatives dans Infrastructure