Runway Labs Introduces Diffusion-UI, a Research App That Generates Interactive Interfaces in Real Time and Can Use Structured Data
VP Land
Nov 19, 2026 · 2 min read
Runway Labs introduced Diffusion-UI, a research application in which a real-time model generates every pixel of an interactive interface, and shared an early look at generated interfaces that can use structured data, maintain states and generate controls. Precise map routes, live weather or any API can be passed in as JSON objects, which gives the model specific information to render a dynamic interface.
Four layers drive each generation. Runway Labs says Diffusion-UI generates using data sources, reference layouts, styles and text prompts.
Scenes are laid out by moving a box. Generated interfaces render each frame as you interact, so you can design a scene by moving and resizing a bounding box.
Text prompts edit the scene live. You can type a prompt to change the scene, add objects or edit anything in real time.
The model highlights its own controls. On hover, white outlines highlight nearby buttons, sliders and other controls to show which elements are interactive.
It is an early research application. Runway Labs calls Diffusion-UI a research application and an early look, and the thread shares no pricing, access or release details.
Diffusion-UI's demo clips cover live data, generated controls and 3D game scenes
In the thread, Runway Labs shows each of these abilities in short demo clips, including a generated word-tile board.
A generated, interactive "Word Tiles" board from the Diffusion-UI demo video, spelling PLAY, YES and WOW. Image: X (formerly Twitter)
Runway Labs also says Diffusion-UI is versatile enough to render complex 3D interfaces, pointing to immersive game scenes as an example.
The thread presents all of this as demo clips. It gives no resolution, frame rate or latency figures, and it doesn't name the model behind Diffusion-UI.
Runway Labs turns Runway's video and world models into prototypes
Diffusion-UI comes out of Runway Labs, an incubator inside Runway that turns video and world models into prototypes people can use.
Runway launched the Labs incubator to explore new applications for generative video and General World Models. It named Co-founder and Chief Innovation Officer Alejandro Matamala Ortiz to lead it with a team dedicated to incubating experimental products.
The thread closes with Runway Labs saying it is exploring tools that make composing and refining real-time interfaces "feel direct and familiar," with more prototypes and tools to share.
Runway's Solaris study favored generated interfaces over coded ones, with text and accessibility still unsolved
Runway has worked on model-generated interfaces before. Its Solaris research post introduced Interface World Models, real-time models that generate the interface itself frame by frame with no code underneath.
Solaris builds on Runway's Gen-4.5 video architecture and generates 720p frames in under 500 milliseconds, and in a 250-person Runway study participants preferred it over Claude Opus 5-coded interfaces 61% to 24% for following the requested interaction, CMSWire reported. For natural behavior within the scene, the split was 71% to 21%.
Runway also acknowledged open problems for Solaris in stable, legible text and long-session coherence, plus accessibility, since a generated interface has no DOM for screen readers to hook into.
The Diffusion-UI thread doesn't say how the new application handles those limits.
Apple released FastFrames on Hugging Face, two autoregressive video diffusion models that generate 480p video at interactive latency, with the 1.3B FastFrames-S producing a frame every 36 ms on one B200 GPU and the 14…
Comfy added custom Builds to Comfy Cloud, letting a team package its own ComfyUI nodes, models, LoRAs and private node packs into one setup that every artist opens in the browser, as the company said on X. Studios now…
Pika introduced Enact, a research preview built on its Pika H3 Flex model that lets a creator interact with AI "actors," re-stage a scene and move the camera in real time, according to the company's announcement on X.…