Tensorial T2 Live: a real-time multimodal conversation model

Tensorial T2 Live is our second-generation real-time multimodal conversation model: you speak, it sees through your camera, and it answers out loud — at the pace of a conversation between people. It runs in production as the engine behind Visionauta's real-time conversation, our visual assistant for blind and low-vision users, where a slow answer or a wrong perception has an immediate consequence in someone's day.

This post covers what changed from T1 to T2, and brings the thing we would rather show than describe: a live terminal, right here on this page. Talk to the model now.

Try it now

The demo connects to the same endpoint that serves Visionauta in production. Sessions are capped and capacity is shared with real users — if it is full, try again in a few minutes.

What a "live" model is

A live model is not a chatbot with a microphone. It is a system holding a continuous bidirectional connection where audio, video and text flow both ways at once. That changes the engineering requirements radically:

• You don't know when the user has finished speaking. Decide too fast and you cut people off; too slow and you create awkward silences.
• The user can interrupt the answer at any moment — and the model has to stop instantly, mid-sentence.
• The camera is live context, not an attachment: the scene changes while the conversation happens.
• Mobile connections drop — and the conversation cannot drop with them.

Latency is the product

In spoken conversation, the gap between the end of your sentence and the first sound of the answer decides whether the interaction feels like a conversation or like an interrogation over a walkie-talkie. It governs every architectural decision we make, and T2 attacked it from two directions worth describing.

Speculating through the silence

Every voice activity detector waits a few hundred milliseconds of silence before declaring the user done. In most systems that window is dead time. In T2 it is working time: the moment silence starts, recognition and generation are already running behind a send gate. If the person resumes speaking, all of it is discarded silently — they never find out. If the silence holds, the answer is already there. That is hundreds of milliseconds recovered on every single turn, at no perceptual cost.

Adaptive turn-taking

The waiting window is not fixed. When the partial transcript closes a complete thought, the turn ends sooner; when it stops mid-sentence, the system waits longer. Combined with instant barge-in, the result is a rhythm that does not sound like software taking its turn.

What is new in T2

T1 proved you could hold a conversation. T2 is about what happens between conversations, and beyond the question that was actually asked.

Memory that survives the session

T2 keeps long-term per-user memory — and deliberately not as a raw transcript of everything ever said. Between sessions the system consolidates what was discussed into a compact representation of what matters about that person, and that is what enters the next conversation. Talking to it on Tuesday continues what was left open on Monday, without anyone restating the context. Memory is optional, tied to an account, and the user can erase it at any time.

Directed proactivity

The hard capability is not answering: it is speaking without being asked, at the right moment. You say "tell me when the bus arrives" and T2 starts watching the scene in silence, deciding moment by moment whether it is time to speak. It is deceptively hard — speak early and it is a false alarm, speak late and it is useless, narrate constantly and it is unbearable. T2 treats this as an explicit gate, separate from answer generation, so that the decision of when to speak is not hostage to the model's urge to fill the silence.

Continuing without speaking

At the end of every answer, T2 offers three plausible ways forward. This is not autocomplete: they are questions derived from what was just said and from what the camera is seeing. For screen-reader users, tapping a suggestion is faster and more precise than phrasing the question out loud — and for anyone, it is what turns an isolated answer into a conversation.

Eight languages, nothing to configure

The model detects the spoken language and answers in it — Portuguese, English, Spanish, French, Korean, German, Italian and Japanese. There is no selector to get right before starting; the person just speaks.

Vision as live context

T2 takes continuous frames and treats them as part of the conversation rather than as an attachment: "what does this say?", "do these clothes match?", "is that gate open?". Redundant frames are dropped at the edge of the pipeline, so that cost grows with visual change, not with time — a still scene is nearly free. Beyond the camera, the conversation can also take the device's own screen, which opens up the apps and pages that are not accessible on their own.

Production robustness

• Session resumption: if the connection drops — lift, tunnel, network handover — the conversation resumes with its context intact.
• Tools mid-sentence: the model calls client-declared functions without breaking the audio stream.
• Polite endings: session limits warn before ending instead of dropping the connection.
• Explicit degradation: when a component fails, the system deliberately chooses what to sacrifice — and never silence, which to a blind user is indistinguishable from a broken app.

Works with the SDK you already use

T2 Live speaks the same bidirectional streaming contract that live-model SDKs already talk. In practice, migrating an existing application is pointing at the endpoint:

from google import genai

client = genai.Client(
    api_key=YOUR_T2_KEY,
    http_options={"base_url": "https://<your-t2-endpoint>"},
)

async with client.aio.live.connect(
    model="models/tensorial-t2-live",
    config={"response_modalities": ["AUDIO"]},
) as session:
    ...  # audio, video, tools — the same code as before

Why accessibility is the hard test

T2 is not a lab prototype: it is the production engine of Visionauta, used by blind and low-vision people to read letters, identify products, describe rooms and get through unfamiliar situations.

We chose that use case because it does not forgive. A beautiful late answer is useless. A confidently wrong description is worse than silence. A button that is not announced simply does not exist. None of these failures show up in a benchmark average — all of them show up in the first minute of real use. A model that survives this test tends to be easy everywhere else: support, assisted monitoring, education, field work, robotics.

Partners: request access

We are accepting partners to build on Tensorial T2 Live — companies and researchers who want to create real-time voice and vision experiences. Access is by whitelist: tell us what you would build and we will get in touch.