Skip to content

How UI works ​

Plugins never ship UI. You describe what you want with the SDK's components, the SDK turns it into A2UI, the open standard for agent-built interfaces, and the app draws it natively. This page explains how that works, the two kinds of UI a plugin can have, and the few rules that keep a conversation fast.

Preview

The TypeScript SDK is being designed in the open. Details on this page may change before the first release.

Describe, don't draw ​

You write React components, but there's no browser underneath: no DOM, no HTML and no CSS. When your component renders, the SDK turns its output into A2UI: a flat list of components that refer to each other by id, plus a data model.

This card:

tsx
<Card title="Lisbon" subtitle="Clear" icon={Icon.CloudSun}>
  <Metric value="21°" label="Now" />
</Card>

is sent to the app as:

json
{
  "createSurface": {
    "surfaceId": "block-8f2c",
    "catalogId": "https://agentplaygrounds.com/a2ui/catalog/v0",
    "components": [
      { "id": "root", "component": "Card", "child": "metric-1",
        "title": "Lisbon", "subtitle": "Clear", "icon": "cloudSun" },
      { "id": "metric-1", "component": "Metric", "value": "21°", "label": "Now" }
    ]
  }
}

The app turns each component into a native view, built from the same design kit as every core feature. That's why plugins:

  • Look like they belong, with the app's type, spacing, colors and icons
  • Adapt automatically to light and dark mode, larger text sizes, Mac and iPhone
  • Stay fast, because scrolling, layout and animation are all native

There's no styling API. Components take a few meaningful options, like a tone of success, and the app decides exactly how that looks. A2UI works the same way: it leaves styling to each app's native look.

A2UI everywhere ​

Everything a plugin shows in the conversation or a surface is A2UI. That's a promise, and it has three consequences:

  • One format for plugins and agents. A card from a plugin and a card from an agent on a server are the same thing, drawn by the same code. Agents that speak A2UI work in Agent Playgrounds with no plugin UI at all.
  • What you build is portable. UI built from the basic catalog works in any app that supports A2UI. Our catalog adds components like Metric and Terminal on top.
  • What's saved is an open format. A block's snapshot is an A2UI surface, so conversations stay readable by other tools.

Slots, like a sidebar row or the buttons beside a message, are specific to Agent Playgrounds and aren't part of A2UI. See A2UI for the details of what we support.

Where your code runs ​

Each plugin runs in its own sandbox inside the app, using the JavaScript engine built into macOS and iOS. A sandbox has:

  • The SDK, and standard JavaScript like Promise, JSON and Intl
  • fetch and WebSockets, for the hosts your manifest allows
  • Storage for your plugin's own data

It doesn't have Node.js, a file system, or access to other plugins. Your code and the app talk only through the SDK.

Two kinds of UI ​

Blocks and slots: pure functions of data ​

Blocks in the conversation, and slots like a sidebar row, are pure functions of their data. A block renderer takes the block's payload and returns components, with no state and no effects:

tsx
export default function WeatherBlock({ payload }: BlockProps<Weather>) {
  return <Card title={payload.city}>…</Card>
}

The app renders a block once, then keeps the result, an A2UI surface, as a snapshot with the conversation. That's what lets a room with ten thousand messages scroll smoothly: the app draws snapshots and runs no JavaScript at all while you scroll. It also means:

  • Blocks look the same everywhere, on every device and for everyone in the room
  • Blocks still show while your plugin is loading, updating or turned off. If there's no snapshot yet, people see the fallback text.

To change a block, change its payload. The app re-renders it, saves the new snapshot and shows it to everyone. A payload can change in three ways:

FromHow
An actiononAction={async (block) => block.update(newPayload)}
An agentYield an update event for a block it sent
A plugin's command or background taskroom.updateBlock(id, newPayload)

Where this is headed

A2UI lets components bind to paths in a data model, like { "path": "/temperature" }, instead of holding literal values. A future SDK will compile blocks that way, so a payload change redraws the block natively without running your renderer at all. Your code won't need to change.

Surfaces, inspector tabs and view commands: live React ​

Bigger UI, like a surface beside the chat, a tab in the inspector or a command that opens a view, is a live React app. Hooks, state and effects all work, and you can fetch data, show a form or navigate between screens:

tsx
export default function PullRequests() {
  const { data, isLoading } = useFetch<PullRequest[]>('https://api.github.com/…')
  return (
    <List isLoading={isLoading}>
      {data?.map((pr) => <ListItem key={pr.id} title={pr.title} />)}
    </List>
  )
}

As your component re-renders, the SDK sends only what changed, as A2UI updates. Live UI mounts when it opens and unmounts when it closes, so it only uses memory while someone's looking at it.

Actions ​

Functions don't cross into the app. When you pass onAction, the SDK keeps the function in your plugin's sandbox and sends the app an A2UI action event with a name. When someone clicks:

  1. The app wakes your plugin if it isn't running.
  2. It sends the action back to your plugin, which calls your handler with context, like the block to update or the room it's in.
  3. Anything your handler changes is drawn as soon as it's done. While it runs, the app shows progress on the button.

Handlers can be async and take as long as they need. Each block's actions run one at a time, so a double click doesn't run them twice.

Some actions never need your plugin: openUrl, copyToClipboard and openSurface are functions the app runs itself, instantly. Use them when you can. When an agent sends a surface, its actions go back to that agent instead. See Agents and events.

Keep it fast ​

A few rules keep plugins fast in a busy conversation:

  • Render only. Don't fetch, compute or wait in a block renderer. Do that in a command, an action or an agent, and put the result in the payload.
  • Keep payloads small. A payload is saved with the conversation and synced. Link to images and large data by URL instead of embedding them.
  • Render quickly. Aim for a few milliseconds. If a render takes too long or throws, the app shows the block's fallback text, and in Developer Mode a warning tells you why.
  • Show loading in live UI. Set isLoading on a list instead of rendering it empty, so people know something's coming.

Where components can go ​

Not every component fits everywhere. A sidebar row is one line tall; a form belongs in a surface, not in the middle of a conversation. The Components reference marks where each one can be used:

PlaceAllowed
BlocksContent, layout and buttons: cards, text, metrics, lists, images, maps
Views: surfaces, inspector tabs, view commandsEverything, including forms and tabs
SlotsA small set sized for the slot: text, badges, icons, progress, buttons

If you use a component where it isn't allowed, the SDK warns you while you develop, and the app leaves it out.

All your Agents, playing nicely in one Playground.