Launching CueSense: A Health and Life Companion Where the AI Runs on Your Phone, Not Our Servers

CueSense runs small AI models on your phone to organise notes, plan and surface health patterns, with validated evidence and deterministic fallbacks.

Most "AI companion" apps have a server in the loop. You speak, your words travel to a data centre, a frontier model answers, and the reply travels back. That design is easy to build and hard to defend — because the moment a health note leaves the phone, you have to explain where it went, who can read it, how long it is kept, and what happens when the network is down. CueSense takes the opposite position. Every piece of AI that touches your words runs on the device in your hand. There is no cloud model, no "bring your own key" provider, and no server that sees a single capture. Today it goes live.

This is a short tour of how it works, and why we made the choices we did. The theme running through all of it is the same one we keep coming back to on this blog: the hard part is not getting an answer, it is getting one you can defend.

Capture a thought by voice or text. Open the screenshot to see the full screen. Cue brings conversation, notes and calendar actions together. Open the full screenshot.

Small models, close to the user

CueSense runs a stack of small, specialised models rather than one large general one. That is a deliberate architecture decision, and it falls straight out of the constraints of a phone.

• A streaming speech recogniser turns your voice into text with no audio ever leaving the device, and no audio ever stored.
• A tiny intent router — a few megabytes — reads each capture and decides what it is: a reminder, a shopping item, a health observation, a business note, an idea.
• A sentence-embedding model indexes your notes so you can later ask questions in plain language and get back the exact records that answer them.
• A compact language model — on the order of a 1-billion-parameter Gemma, quantised to four bits and about 584 MB on disk — does the reasoning, planning and extraction, with larger Gemma and Qwen options for phones that can carry them.

None of these is a frontier model, and that is the point. Small models are now good enough at narrow tasks that routing every request to a giant one would mean paying for latency, cost and a cloud dependency you do not need. A router that fits in a few megabytes classifies a note in the time it takes the screen to settle. A four-bit language model, handed a tightly scoped job, answers in a couple of seconds. The job of the system is to break the problem into pieces small enough that each one runs on the smallest model that reliably does it.

The AI selects; the app owns every fact

Here is the commitment that makes a one-billion-parameter model safe to ship next to health data: the model is never allowed to author a fact. It does not write the dosage, the date, the time, or the diagnosis. It selects.

When CueSense summarises how a symptom has changed over weeks, the model does not compose a sentence of medical prose. It is handed your confirmed records, each tagged with an opaque label, and it returns a structured choice — this symptom, these records, in this order. Deterministic code then writes every visible sentence from the underlying fields you already confirmed. The same pattern governs planning (the model ranks pre-computed options; it never invents a time), reminders (the date always comes from a deterministic parser, never the model), and answering questions about your notes (every answer must cite real record IDs, and a claim with no citation is discarded).

Ask your own records a question. Answers must cite matching records on this device. Open the full screenshot. The health timeline links observations to their underlying evidence. This is a preview of a long screen; open it to read the full timeline.

Then everything the model produces is validated before you ever see it. The output has to parse, the citations have to resolve to real records, the chronology has to hold, a claimed change has to correspond to an actual change in your data. If any check fails, CueSense does not show you a plausible guess — it falls back to an exact quote from your own note, or to a deterministic result. The worst a mis-generation can do is pick the wrong already-verified evidence, which the validator usually catches anyway. It cannot fabricate a number.

The app also refuses to practise medicine. Diagnosis, causation, treatment and urgency are not soft-discouraged by a prompt; they are structurally outside what the model is allowed to return. CueSense will tell you that a symptom you logged appears more often this month than last, and cite the entries. It will not tell you what it means. That boundary is a feature, and we would rather ship it than ship something that sounds more impressive and is less honest.

A goal becomes reviewed steps, with progress grounded in task records. Open the full screenshot. Planning starts from your calendar and available time. This is a preview of a long screen; open it to read the full calendar.

Making a phone behave like a reliable runtime

Running language models on consumer hardware is where the engineering gets unglamorous, and most of the work that made CueSense shippable lives here.

• Downloads survive real life. A 584 MB model is fetched by a background downloader that keeps going if you switch apps or the process is killed, lands in a temporary file, and is only committed once its SHA-256 matches. A truncated file never masquerades as a working model.
• The GPU is used, but never trusted blindly. On Android, inference asks for the GPU — roughly five times faster than the CPU — but a model's first attempt is treated as a probe. If the GPU driver throws, if the first reply fails, or if the app dies mid-load, that model drops to the CPU on that specific phone and stays there. A crash that would otherwise repeat on every launch happens at most once.
• Generation is serialised and deadlined. The runtime shares one cache, so requests run through a single queue; every call carries a timeout so no spinner can wait forever. Structured tasks decode greedily, so an extraction is repeatable; open conversation uses the model family's own sampling.
• Everything degrades toward a fallback. The router falls back to a rule-based classifier. The embedding model falls back to a deterministic vector. The GPU falls back to the CPU. Generation falls back to a quote. No single model failure takes a feature down with it.

The telemetry that tells us whether any of this is working is deliberately content-free: task name, latency, response length, failure code. It never contains a transcript, an answer, a note, or a line of audio. We cannot debug with your words, because we never have them.

Reminder times belong to the app's records, with explicit controls to edit or complete them. Open the full screenshot.

The factory stays offline

If the phone does all the inference, what is the backend for? It is a factory and a governance workbench, not a runtime. Models are trained there — a classical pipeline that generates synthetic data to prove the contracts, imports licence-pinned public corpora as auxiliary signal, trains, and exports a signed model artefact that the device verifies before it will load. Release gates use confidence bounds rather than point estimates, a human-adjudicated test set is evaluated exactly once through an append-only ledger, and consented training text is held under envelope encryption where withdrawing consent cryptographically erases that person's data without touching anyone else's.

We are candid in our own documentation about where this is still early: the current bottleneck is not model size but validation design — we need more consented, representative human data before any of these classifiers earns a production readiness claim. Shipping the app does not change that honesty. The on-device product is built to be safe by construction; the backend is built to make the models better without ever becoming a place your captures live.

What this adds up to

Strip away the specifics and a few transferable commitments remain. They are the same commitments whether you are building a health companion or a data agent:

• Put the model where the data already is. On-device inference is not only a privacy story; it removes an entire class of questions you would otherwise have to answer about a server.
• Use the smallest model that reliably does the job, and route deliberately between them rather than sending everything to the largest one.
• Let the model select, and let deterministic code own every fact the user sees — the number, the date, the claim.
• Validate before you render, and fall back to the truth — an exact quote, a deterministic result — rather than a confident guess.
• Make the boundaries structural. The things the system must never do should be impossible to express in its output, not merely discouraged in a prompt.

CueSense is live today. Capture a thought by voice or text, let it file itself, ask your own notes a question, and watch a small model on your own phone answer it — and tell you plainly when it cannot. That last part is not a limitation we are apologising for. It is the whole design.

Sources

• CueSense on-device model runtime and routing architecture.
• Cueset Labs, LLMs vs SLMs: A Practical Guide for Senior Engineers.
• Cueset Labs, Building a Data Analyst Agent: Grounding an LLM in Tools, Not Guesses.