On-device or cloud: where the conversation of an AI toy should run

Where the conversation runs is the biggest architecture decision in a talking toy, and it is rarely framed as a decision at all — it arrives hidden inside a platform choice. It deserves its own conversation, because it decides latency, battery, data, cost per unit sold, and what the toy becomes when the Wi-Fi goes down.

What the cloud gives, and what it takes

Cloud conversation is where the strongest models live: open-ended dialogue, current knowledge, easy content updates pushed to every toy overnight. The costs travel with it. Latency is physical — the answer lives one round trip away, and children notice hesitation faster than any adult. Connectivity becomes a dependency: the toy’s best feature dies with the router. And every spoken sentence crosses the network, which means the brand owns a data story it will be asked to tell — by parents, by reviewers, and eventually by regulators.

What on-device gives, and what it lacks

On-device conversation — small language models running on the toy’s own silicon — has crossed from novelty to practical for a useful band of behaviour: characterful barks, scripted-with-variance dialogue, simple Q&A, all with zero latency and zero network dependence. What it lacks is depth: the model that fits in a toy’s memory cannot match cloud models on open-ended conversation or current knowledge. Battery is the other tax — every conversation becomes a compute workload the cell must carry.

The hybrid is where the category landed

Almost every strong talking toy now ships hybrid: the character’s fast layer — greetings, reactions, in-character short dialogue — runs on-device, always available, instantly responsive. The open-ended layer runs in the cloud where the strong models are. The design question that matters is the third one most teams skip: what happens when the network drops? A toy that falls back to its on-device layer — maybe quieter, maybe simpler, but still itself — becomes “the toy that still works offline”. A toy that becomes mute plastic becomes a review headline.

The data story follows the architecture

Wherever the conversation runs, a sentence about it has to survive being repeated aloud. On-device processing lets a brand say “your child’s words never leave the toy”, which is the strongest sentence available in this category. Cloud processing requires a story about retention, purpose and deletion that must be true, short and public. Neither is a blocker; both are architecture consequences, which is why the data story belongs in the same meeting as the silicon choice, not in the legal review afterwards.

We build AI toys with the conversation architecture chosen deliberately — on-device, cloud, hybrid — and the fallback designed from the start. The OEM/ODM page outlines how we run development, or write to us with the conversation experience you want the toy to have.