Does an AI App Send Your Prompts to a Server? How to Check

Why this question needs a feature-by-feature answer

An AI app rarely has one network state. Opening the app may fetch code or an update. Installing a model may contact a model publisher. Ordinary inference may run locally. A search button may send a query to a third party. A feedback form may upload selected text. Saying “the app goes online” or “the app works offline” does not tell you which of those events happened.

The goal of a privacy check is therefore not to produce a magical yes-or-no badge. It is to build a small data-flow map: action, data, destination, purpose, trigger, retention, and user control. NIST’s Generative AI Profile emphasizes privacy, security, transparency, and measurement across an AI system’s lifecycle. A repeatable check follows the same spirit by connecting claims to observable evidence. [1]

Do this investigation before using real confidential information. Create synthetic prompts and harmless sample documents. A network monitor can help you learn where an app connects, but it does not undo a disclosure already made during testing.

Start with a written boundary

Read the current privacy notice, product documentation, app-store data disclosure, and any separate terms for optional services. Search for explicit answers to five questions: where inference runs; whether prompts, responses, documents, or images leave the device; who receives them; how long remote data is kept; and what the user can delete.

Watch for wording that answers a different question. “History stays on your device” describes storage, not inference. “We do not sell personal information” does not say whether a provider processes prompts. “No account required” does not eliminate IP addresses, session identifiers, cookies, or remote logging. “End-to-end encrypted” needs endpoints: encrypted between which parties, and who can decrypt data for inference?

Translate marketing language into testable questions

Build a test matrix before opening network tools

List the actions you actually intend to use. Test them one at a time so a model download is not mistaken for a prompt upload. Record the time, app version, operating system, selected model, connection state, and expected recipient. If the product changes, the old result may no longer describe the current build.

• First launch and ordinary reopen
• App shell or software update
• Model catalogue view, model download, verification, and activation
• New chat, regeneration, stop, and reopen of history
• Document, image, camera, OCR, and local retrieval
• Web search or other connected tool
• Feedback, response report, diagnostics, or crash reporting
• Export, backup, deletion, and uninstall

Run an offline smoke test

Install the app and its chosen model while connected. Confirm that the model is activated, close the app, enable airplane mode, reopen it, and submit a harmless prompt such as “Reply with exactly OK.” Repeat the test with any local document feature you depend on. If output appears, you have evidence that this tested path did not require a live inference server at that moment.

Airplane mode is valuable but limited. An app could have uploaded content before the radio was disabled, use a model only for certain prompts, queue a request for later, or fail because an un-cached interface asset is missing even though model inference is local. Conversely, a visible connection does not prove prompt upload; it may be an update check. Record exactly what the test demonstrates.

Google’s official AI Edge Gallery project provides a useful reference claim: model inference happens on the device and no internet is required once the model is available. That still leaves installation and model delivery as connected actions, which is why the sequence matters. [2]

1. Use a fresh, non-sensitive test chat and harmless test file.
2. While connected, finish app setup and the explicit model download.
3. Close the app completely, then enable airplane mode.
4. Reopen it and send a deterministic short prompt.
5. Close and reopen again to test offline persistence.
6. Test each claimed local feature separately and record failures.

Observe network requests where practical

For a web app or PWA, the browser’s developer tools provide a direct starting point. Chrome’s Network panel records requests while the tools are open and can show URLs, domains, methods, timing, initiators, headers, and response details. Preserve the log, clear it between actions, and filter by fetch or XHR when looking for application requests. Chrome documents the panel and its limitations in its official developer documentation. [3]

A native mobile app requires a device-level method, such as operating-system data-use views, an approved local VPN/firewall monitor, a controlled Wi-Fi gateway, or developer tooling on a test device. Use tools you trust: a monitoring VPN may itself gain visibility into destinations, and installing a custom certificate to inspect encrypted traffic changes the device’s security. Do not weaken a daily-use phone merely to satisfy curiosity.

Transport encryption also limits what you can conclude. Seeing a destination and request size can establish that a connection occurred, but not necessarily which fields were inside it. Not seeing a request during a short test does not prove the app will never connect under another feature, error, timer, or operating-system condition. Network observation is one evidence layer beside documentation, source code where available, and repeatable offline behavior.

Classify every destination

When a domain appears, do not jump directly to “spyware.” Identify its owner and purpose. A request to the app’s origin may deliver a static interface; a model host may deliver a multi-gigabyte artifact; an error service may receive a stack trace; an inference endpoint may receive prompt content. The initiator, timing, method, payload shape, privacy notice, and repeatability help distinguish them.

Pay particular attention to third-party analytics, crash reporting, advertising, remote configuration, and content delivery. Ask whether they start automatically, what legal basis applies, whether a simple opt-out exists, whether identifiers are stable, and whether a prompt or file name could appear in a URL, event label, log, or error message. URLs can be copied into histories and logs; sensitive content should not be placed in a query string.

Check storage, retention, and deletion too

A prompt can remain private from an inference server and still be exposed through weak local storage. Determine whether saved history, documents, retrieval indexes, credentials, and exports are encrypted. Check whether protection depends on a device screen lock, app passphrase, operating-system keystore, or merely ordinary browser storage. Also ask what remains visible while the application is unlocked.

For browser applications, offline storage is controlled by both the application and the browser. The web platform provides Cache Storage and IndexedDB for offline assets and data, but users and operating systems can remove site storage. Persistent storage can reduce eviction risk without making it impossible. A privacy-preserving local design therefore needs both protection and a backup story. [4]

Test deletion with the same precision as chat. Does “delete conversation” remove attachments and indexes? Does “delete account” apply when no account exists? Can the provider delete a local vault it never possessed? Are optional remotely submitted reports covered by a separate retention and deletion process? A good notice acknowledges these distinctions.

A worked example: CuriousLM

CuriousLM’s documented normal path is local: ordinary chat does not require an account, a CuriousLM-operated inference server, or hosted conversation storage. Conversations, projects, approved memory, documents, images, extracted text, OCR results, retrieval indexes, citations, and selected model inference stay on the device. Private PWA records and files use a local encrypted vault; Android uses encrypted native storage protected by Android Keystore and device authentication. [5]

Expected delivery traffic includes the app shell or update and a model the user explicitly chooses. The browser, app store, hosting provider, or model publisher receives ordinary request metadata such as IP address, user agent, time, and requested file. CuriousLM says prompt and response text is not included in those delivery requests. [5]

The two main optional content paths require a preview and confirmation. A Tavily search sends the confirmed query directly to Tavily using the user’s key. A response report sends only the reviewed excerpts and optional comment, plus app, platform, runtime, and model versions and a random idempotency key, to reports.ai.teda.dev. The notice says reports exclude stable device identifiers, API keys, complete conversations, and unrelated diagnostics; diagnostics are off by default. [5]

A sensible CuriousLM test should therefore expect traffic during setup, updates, and model installation; expect no prompt-bearing network request during an ordinary local chat; and expect a new, named recipient only after confirming search or reporting. Any contradiction should be documented with version, platform, action, time, and destination and reported without softening the finding.

What a responsible conclusion looks like

Prefer a bounded conclusion: “In version X on device Y, after setup, ordinary text chat and this document test completed in airplane mode; observed connections during the connected run matched the documented app and model delivery hosts.” That is more useful than “100% private” and easier for someone else to reproduce.

Recheck after major updates, when enabling a new feature, or when the privacy notice changes. If you cannot identify a destination or a product’s behavior contradicts its documentation, stop using sensitive data and contact the publisher with a minimal reproduction. Privacy testing is strongest when it remains factual, scoped, and open to correction.