Talking to the AI workspace instead of typing
A single click on a mic button on the AI workspace's chat composer starts it listening. Speak a request the way you'd say it out loud, and each pause gets detected and sent off on its own, landing in the chat as a real message, without switching your hands back to the keyboard. From there the agent takes over: it reads the voice prompt, decides what it's asking for, and the workspace updates itself accordingly, calendar, inbox, whatever the request called for, all driven from what was said, not typed or clicked.
Mic to message to a page that updates itself
The audio never touches the browser's own speech APIs. It goes out over a real WebRTC connection to a room, gets picked up server-side, chopped into utterances, and transcribed, with the text coming back over the same connection it left on, straight into the chat composer as a real message. The agent reads that message, decides what it's asking for, and the workspace updates on its own: ask to see the calendar and the calendar appears, ask for the inbox and the email list renders, with no click beyond the one that started listening.
Voice reaches the same agent, not a separate command layer
There's no separate list of voice commands to learn and no fixed set of recognised phrases. A spoken request goes through the exact same agent that handles typed chat, so whatever the agent can already do for a typed request (open the calendar, pull up an inbox, run a piece of research) it can do for a spoken one, and the workspace re-renders to show the result the same way it would from a click.
The stack behind it
LiveKit Cloud
WebRTC transport for the room the browser and the voice worker both join. Room tokens are scoped per organisation and minted by the backend, not handed out generically.
A standalone voice worker
A small Node.js service that joins the room server-side, subscribes to the mic track, and does nothing else: it doesn't touch the database or the rest of the backend directly.
Adaptive silence segmentation
Utterance boundaries are detected from a noise floor that adapts to the room, not a fixed volume cutoff, because real mics and rooms vary far more than a single threshold can cover.
A hosted speech-to-text service
Each segmented utterance is sent out for transcription and comes back as plain text, which is the only thing that ever reaches the chat composer, not raw audio.
Engineering details that surfaced under real conditions
A handful of issues only became visible once real microphones, real network conditions, and the actual hosting environment were involved. Each of the following was diagnosed and fixed against that reality, not assumed away.
Forced TURN relay
The host has no outbound UDP, so the connection is forced through a TLS relay rather than assuming a direct peer path will work.
Unpublish on stop, not mute
Clicking the mic button again explicitly ends the audio track instead of muting it, because a muted-but-open track left the worker waiting on audio that would never come.
Browser audio processing turned off
The browser's own echo cancellation and noise suppression were quietly stripping real speech down to near silence on some mic and OS combinations, so they're disabled in favour of the server-side segmentation.
Continuous sessions
One click starts a session that can carry several spoken sentences, each transcribed and delivered as its own utterance, instead of forcing a stop and restart between them.
Nothing sensitive in the logs
Tokens, room names, and transcript text are kept out of diagnostic logging on principle, not just for this feature.
Why bother with voice input at all
Typing a long freeform request into a chat box, then waiting for the right screen to appear, is friction a rep mid-call or away from the keyboard shouldn't have to deal with. Saying it instead removes that friction on both ends at once: the request lands as a real chat message, the agent decides what it needs, and the workspace, calendar, inbox, whatever the prompt called for, renders on its own, all starting from nothing but what was said out loud.