Does Local AI Really Stay Offline? How To Prove It On Your Own Machine
Synergised Consulting Ltd
Last Updated: 5 November 2026
Local AI means the model runs on your hardware; it does not mean nothing leaves your machine. During actual inference, the news is good: a published 2026 packet-capture study found no traffic attributable to Ollama, LM Studio or llama.cpp carrying prompt data off the machine while they generated text [2]. But the same research shows what does leave: model-download records, update checks and, on disk, plaintext prompt histories that a forensic study recovered in full from common local tools [1]. The way to know what your own stack does is to watch it, and that takes about thirty seconds of packet capture while a prompt runs.
What "Local" Actually Promises
When a vendor says a model runs locally, the claim is about where computation happens: the weights sit in your memory and the inference happens on your processor, so no prompt is sent to a cloud API for generation. That is a real and valuable property. It is not the same claim as "no data leaves this machine", which is a statement about network behaviour, disk behaviour and default settings, and it is the statement buyers usually think they are hearing. A local stack still touches the network to download models, to check for updates and, depending on the serving engine and interface you put in front of the model, potentially to report usage. The distinction matters because the second claim is the one that decides whether local AI is appropriate for commercially sensitive work, and it is the one no product page will demonstrate for you.
Where Data Actually Goes In A Local Stack
The evidence from 2026 investigations separates cleanly into three paths. None of them is a prompt being uploaded during inference, which is the fear that drives most local-AI purchases, but all of them are data movements an owner should know about before asserting anything to a customer or a buyer.
Path 1: What crosses the wire
The packet-capture study watched the network interface of a machine running all three major local runtimes through idle periods, model downloads and real inference [2]. During generation, nothing attributable to any runtime went out. What did cross the wire was metadata: pulling a model contacted the vendor's registry, which records what you asked for and sees your IP address; update checks reported the operating system, processor architecture and application version; and one runtime checked in periodically rather than only at launch. This is a record of what you are building with, not a leak of what you are building, but it is a distinction worth making deliberately rather than assuming.
Path 2: What sits on the disk
A separate forensic study recovered plaintext prompt histories from local AI tools across Windows and Linux [1]. Prompts typed into the ollama run command line were recovered in full from a history file on disk. Interactive llama.cpp sessions left nothing recoverable, which shows how much the answer depends on which tool and which mode you use, not on "local" as a category. For an owner, the implication is concrete: the machine itself holds the sensitive material, so the confidentiality of a local AI deployment is governed by disk hygiene, access control and disposal as much as by the network.
Path 3: What the surrounding tools report by default
The model runtime is rarely the whole stack. Serving engines and web interfaces carry their own defaults: the vLLM project documents in its own manual that it collects anonymous usage data by default, describing the fields gathered and publishing opt-out environment variables [3], and Open WebUI's configuration reference similarly documents the telemetry-related variables an installation can set [4]. These are disclosed behaviours, and both projects make them switchable. But a default that is on unless someone turns it off is a decision nobody made, and in a small business that usually means nobody knew it existed.
The Test That Turns The Claim Into Evidence
For illustration, here is what proving it looks like on a machine you control. Start a packet capture on the machine's outbound interface, run a representative prompt through each tool in your stack, and stop the capture. If prompts are genuinely staying local, the capture shows no connection from the runtime during generation; if a tool is phoning home, its endpoint appears in the log with a timestamp you can match to your own actions. The same study that ran this properly notes it is roughly thirty seconds of work with standard tools such as tcpdump, plus a check of which processes hold open network connections while the stack sits idle [2]. The point is not the specific command; it is that the offline claim is checkable by the person making the purchase decision, on their own configuration, without trusting anybody's marketing.
What The Evidence Proves, And What It Cannot
A clean capture proves one narrow thing: during your test, on your configuration and version, no prompt data left over the network. That is worth having and worth filing. It does not prove the stack stays quiet after an update, which can change defaults; it does not cover the disk artifacts the forensic study documented [1]; and it does not say anything about who can access the machine. Owners planning to rely on local AI for sensitive work should treat the capture the way an accountant treats a reconciliation: dated, filed, and repeated whenever the configuration changes. An offline claim backed by a dated log is also something you can show to a sceptical counterparty, whereas the same claim without evidence is a promise you are asking someone to take on trust.
Where This Fits In The First 100 Days
In our own discovery conversations, owners evaluating AI tooling rarely arrive with the local-versus-cloud question framed this precisely; it usually surfaces as "we just don't want our data going anywhere", which is a privacy requirement stated as a deployment preference. The evidence-first habit this topic teaches generalises beyond local models: whether the stack is local or API-based, the useful move is the same, which is to know what leaves the building, prove it once, and keep the proof current. For a founder in the first 100 days of ownership, that habit costs minutes and produces exactly the kind of dated, specific evidence that makes an operating claim credible to everyone who was not in the room when it was made.
Sources
- [1] Forensic Implications of Localized AI: Artifact Analysis of Ollama, LM Studio, and llama.cpp, arXiv:2603.23996, 25 March 2026:
- [2] Is your local LLM actually local? We watched the wire, LocalModelBench, 3 October 2026:
- [3] Usage Stats Collection, vLLM documentation:
- [4] Environment Variable Configuration, Open WebUI documentation:
