Voice and talk runtime

Orbz stays silent when connected. In @neongate-ai/orbz@1.0.3, the package supplies no greeting, persona or conversation: talk is an empty frozen object and DEFAULT_TALK_FLOW is an empty frozen array. The host supplies speech for one phrase or a typed talkFlow for a conversation, configures a voice engine, and starts it only after an explicit user action.

speech takes precedence over talkFlow. With neither supplied, startTalking() is a no-op. Starting a nonempty talk flow resets its runtime context; an ask step captures input under its declared capture key when the host calls receive(). A later step can interpolate that key. Orbz does not persist this context in cookies, local storage, IndexedDB or a backend.

Built-in talk data

import { DEFAULT_TALK_FLOW, talk } from "@neongate-ai/orbz";
 
console.log(talk); // {}
console.log(DEFAULT_TALK_FLOW); // []

Continue the flow

After the custom flow below has started and reached its ask step, pass submitted user input to receive(). The fullName key exists because that example declares capture: "fullName"; it is not built-in conversation data.

import "@neongate-ai/orbz/browser";
import type { OrbzElement } from "@neongate-ai/orbz";
 
const orb = document.querySelector<OrbzElement>("orb-z");
 
await orb?.receive("Jonatas");
console.log(orb?.talkContext.fullName);

English browser speech

WebSpeechAdapter defaults to pt-BR in the installed package. The example explicitly selects en-US because its sample sentence is English. For other languages, set both the host-owned text and the adapter language. The adapter waits for the asynchronous browser voice list and ranks available voices matching the requested language; installed voice availability still depends on the visitor’s environment.

import '@neongate-ai/orbz/browser'
import {
  WebSpeechAdapter,
  type OrbzElement
} from "@neongate-ai/orbz";
 
const orb = document.createElement("orb-z") as OrbzElement;
orb.speech = "Hello. This is an explicit speech example.";
orb.voiceEngine = new WebSpeechAdapter({
  language: "en-US",
  preferredVoices: ["Google US English", "Microsoft Aria Online"]
});
document.body.append(orb);
 
const startVoiceButton = document.querySelector<HTMLButtonElement>("[data-start-voice]");
 
startVoiceButton?.addEventListener("click", async () => {
  await orb.startTalking();
});

The actual installed voices still belong to the visitor’s browser and operating system. WebSpeechAdapter improves selection; it cannot turn a system voice into an OpenAI voice.

OpenAI-quality speech

OpenAISpeechAdapter defaults to gpt-4o-mini-tts, marin, MP3 and Brazilian Portuguese speaking instructions in version 1.0.3. This English example overrides instructions to match its host-owned text. The application must implement and protect the example endpoint.

import '@neongate-ai/orbz/browser'
import {
  OpenAISpeechAdapter,
  type OrbzElement
} from "@neongate-ai/orbz";
 
const orb = document.createElement("orb-z") as OrbzElement;
orb.speech = "Hello. This is an explicit speech example.";
orb.voiceEngine = new OpenAISpeechAdapter({
  endpoint: "/api/orbz/speech",
  instructions: "Speak in natural American English. Do not change the supplied text."
});
document.body.append(orb);
 
const startVoiceButton = document.querySelector<HTMLButtonElement>("[data-start-voice]");
 
startVoiceButton?.addEventListener("click", async () => {
  await orb.startTalking();
});

The endpoint is owned by the implementing application. It receives an OpenAI-compatible JSON body containing input, instructions, model, response_format, and voice, and returns the generated audio response. Keep the OpenAI API key on that server endpoint; never put it in browser code or in the npm package.

Applications using generated speech should clearly disclose that the voice is AI-generated.

Explicit activation and browser policy

Render a native <button> with a clear label such as Start voice, then call startTalking() from its click handler. Connecting the orb, assigning a voice engine, or navigating to a page never starts audio.

If the browser still rejects the requested audio with NotAllowedError, Orbz dispatches orbz-talk-error with the original error and retries the requested flow after the next pointer, keyboard, or touch interaction. Reset controls should not remount the element merely to unlock speech.

Supply a custom flow

import "@neongate-ai/orbz/browser";
import { WebSpeechAdapter, type OrbzElement, type OrbzTalkStep } from "@neongate-ai/orbz";
 
const flow = [
  { id: "welcome", kind: "say", needsAuth: false, text: "Hello." },
  { id: "name", kind: "ask", needsAuth: false, text: "What is your name?", capture: "fullName" },
  { id: "help", kind: "say", needsAuth: false, text: "How can I help, {{fullName}}?" },
  { id: "answer", kind: "respond", needsAuth: false, strategy: "openai", fallback: "I cannot answer that right now." }
] as const satisfies readonly OrbzTalkStep[];
 
const orb = document.createElement("orb-z") as OrbzElement;
orb.talkFlow = flow;
orb.voiceEngine = new WebSpeechAdapter({ language: "en-US" });
document.body.append(orb);
 
const startVoiceButton = document.querySelector<HTMLButtonElement>("[data-start-voice]");
startVoiceButton?.addEventListener("click", async () => {
  await orb.startTalking();
});

Assign voiceEngine, talkFlow, and intelligence before calling startTalking() so the explicit run uses them.

Optional intelligence

import type {
  OrbzElement,
  OrbzIntelligencePort
} from "@neongate-ai/orbz";
 
const intelligence: OrbzIntelligencePort = {
  async respond(input, context) {
    return productAgent.respond({ context, input });
  }
};
 
orb.intelligence = intelligence;

Events and visual state

While audio is playing, Orbz temporarily uses the speaking visual state and then restores the prior state.

The text-to-speech flows above do not capture a microphone. The host owns input controls, permissions, transcripts, product logic and calls to receive(). Orbz also exposes a separate Realtime conversation API; it is not started by these startTalking() examples.