Local AI vs On-Device, Offline, and Self-Hosted AI

Why these AI terms get confused

The phrases local AI, on-device AI, offline AI, private AI, and self-hosted AI are often used as if they were interchangeable. They are not. Each phrase answers a different question about a system. On-device describes where computation happens. Offline describes whether a connection is required. Self-hosted describes who operates the infrastructure. Private describes how data is handled across the whole product, not only where one model runs.

The confusion matters because a product can satisfy one definition without satisfying the others. A chatbot may run a model on a server in your home, which is self-hosted but not on the phone in your hand. A phone may use an on-device model for rewriting while sending analytics and account identifiers to remote services. A browser app may work offline after setup but still require the internet for its first visit and model download.

Android's official AI guidance separates on-device inference from cloud inference and says on-device options process input locally, can work without a network, and avoid per-request cloud inference cost. Those are meaningful properties, but they do not automatically describe every surrounding feature in an app. [1]

A precise definition for each term

On-device AI means inference executes on the user's current device. The model weights, runtime, prompt processing, and token generation stay on that phone, tablet, or computer for the local path. Google describes Gemini Nano as an on-device model that can deliver features without sending data to the cloud. [2]

Local AI is broader and less standardised. In everyday product language it usually means inference runs on hardware controlled by the user rather than a provider's shared cloud service. That can mean the current phone, a laptop on the same desk, or a private workstation reached over a local network. Ask where the model process actually runs instead of relying on the label.

Offline AI means the relevant task can complete without an active network connection. An app can be on-device but not offline-ready if its interface, runtime, tokenizer, or model has not been downloaded. A feature can also be offline for generation but require a connection for search, sync, updates, or a newly selected model.

Self-hosted AI means the user or organisation operates the inference service and its infrastructure. The service might run on a home server, office workstation, private cloud account, or rented machine. Self-hosting provides more operational control than a managed public endpoint, but prompts still travel across a network when the client and server are separate.

Private AI should describe the complete data boundary. It includes prompts, responses, files, indexes, identifiers, logs, analytics, backups, reports, and optional tools. A local inference engine can reduce disclosure, but privacy still depends on product design and user choices.

The question each AI label should answer

Common combinations in real products

A fully on-device chat app can also be offline and private for ordinary conversations. This is the clearest local arrangement: the model and runtime are present on the phone, the prompt stays inside the app process, and no server is needed to generate a reply. It still needs a defined policy for model delivery, updates, crash reporting, and any optional online actions.

A self-hosted home server is local in the sense of ownership, but it is not on-device from the phone's perspective. The phone sends a prompt over Wi-Fi to the server. That may be a good trade when the server has more memory and compute, but the connection, server logs, router, and access controls join the privacy boundary.

A cloud privacy proxy can hide the user from the underlying model provider or reduce retention, yet inference still happens remotely. Such a service may offer strong contractual privacy, but it does not become on-device or offline. Conversely, an on-device feature can have weak overall privacy if the application sends detailed telemetry elsewhere.

A Progressive Web App can be both installed and local. Service workers can make the shell available offline, and browser storage can retain model artifacts and user data. web.dev explains that PWA assets are still web assets and must be cached or stored before they can be served without a network. [3] Installation alone therefore does not prove offline readiness.

How to verify which architecture you have

Start with the product's own interface and documentation. Find the exact model name, artifact size, download state, runtime, and selected execution path. If the app never exposes a model download or system model but claims unlimited offline use, ask what is actually doing the inference. A clear product should distinguish a model that is available in a catalogue from one that is downloaded, verified, and active.

Then perform a controlled offline test. Complete initial setup, activate one model, restart the app, enable airplane mode, create a fresh conversation, and send a unique prompt. A new reply shows that this generation path can operate offline. It does not prove that earlier traffic was absent or that search, reports, sync, and model changes are offline too.

For a self-hosted system, identify the network destination. If the phone stops working when the home server is off, inference is not on the phone. Inspect whether the server is limited to the local network or exposed to the internet, whether transport is encrypted, and whether request logs are retained. Control over a server is valuable only when its configuration is understood.

1. Identify the machine that loads the model weights.
2. Confirm which device processes the prompt and generates tokens.
3. Test the core action after a restart with all network paths disabled.
4. Test optional search, backup, sync, and reports separately.
5. Review local storage, server logs, analytics, and deletion controls.
6. Record the app version, model artifact, device, and test date.

Which architecture fits which task

On-device AI is a strong fit for personal drafting, local notes, document questions, and dependable use during travel or service outages. The primary constraints are device memory, storage, battery, heat, browser support, and model quality. Smaller models may be less capable than large hosted systems, so local execution should not be confused with correctness.

Self-hosting can suit a household or team that wants one more capable machine to serve several devices. It can also support models too large for a phone. The trade is operational responsibility: updates, authentication, encrypted transport, backups, monitoring, remote access, and incident response all belong to the operator.

Cloud AI remains useful for tasks requiring very large models, current provider-side tools, shared workspaces, or capacity that a phone cannot supply. The decision is not a moral ranking. It is an architecture choice based on data sensitivity, capability, connectivity, cost, maintenance, and the user's ability to verify the boundary.

Hybrid products can combine these paths honestly. A local model can handle ordinary chat while an optional online search runs only after the user reviews the query. The essential design requirement is visible consent and accurate state. A product should never silently switch from local inference to a hosted model while continuing to display an offline or private label.

The CuriousLM boundary

CuriousLM uses on-device inference for ordinary chat and keeps conversations, projects, imported files, and indexes in local application storage. It does not require a CuriousLM account or a hosted conversation database for the normal local workflow. The chosen model must be downloaded or otherwise available, verified, activated, and supported by the current runtime.

The network boundary is not absolute. The first app load, updates, model delivery, and some runtime assets require connectivity. Optional Tavily search sends a reviewed query only after user confirmation and uses the user's key. A response report sends previewed excerpts and an optional comment after a separate confirmation. These are named network actions, not hidden extensions of local chat.

That distinction is why the most defensible description is a private local AI workspace with explicit network boundaries. It says what happens in the default workflow and leaves room to describe exceptions precisely. Labels should help a user predict behaviour, not replace evidence.