Notion AI Chats Exposed in Fireworks AI Breach: What to Do
Notion says some Notion AI chat transcripts may have been exposed in a security incident at Fireworks AI, a vendor it used for its Insights feature and for serving open-weight models. Fewer than 1% of workspaces may be affected. The chats carried no author or workspace labels, but Notion advises rotating any passwords or API keys pasted into Notion AI.
On this page
What Notion told its customers
Notion has published a notice saying that some chat sessions with Notion AI may have been exposed in a security incident at Fireworks AI, a company that hosts and serves AI models for other businesses. Notion puts the scope at under 1% of workspaces and says it has contacted the owners of those affected.
The transcripts came from Insights, a feature that helps customers understand their own AI usage. Notion sent chats to Fireworks to be classified for that feature. According to the notice, the chats were not labelled with an author or a workspace, and they did not include tool call results such as page content that Notion AI pulled in while answering. Notion says no other workspace data is affected.
The timeline in the notice is short. Notion learned of potential incidents from Fireworks earlier this month and cut its connections. By 3 November it had deleted its data in the Fireworks system and halted training runs. On 6 November Fireworks said attackers may have accessed customer data, and on 8 November it confirmed a dataset that Notion found to contain the transcripts.
How the Fireworks intrusion unfolded
Fireworks has published two bulletins. The first says that on 6 October it identified unauthorised use of internal credentials, which gave an outside party access to environment variables in evaluation jobs for a limited number of customer accounts. Those variables included API keys and tokens stored as secrets.
The second, updated on 9 November, describes further activity. A third-party cloud provider hosting Fireworks research machines had its own incident on 23 October. The next day the intruder used it to reach Fireworks development machines and run a workload that touched more customer data. Fireworks now lists account and billing information and, for a limited set of accounts, secrets, datasets, and incomplete model weights. It says payment card numbers were not stored, and that it has brought in law enforcement and Palo Alto Networks Unit 42.
Fireworks gives no count of affected customers. Trade outlet RuntimeWire, citing Bloomberg, describes this as the company's second known breach in weeks.
The gap between zero retention and stored data
Fireworks stresses that its inference platform does not store prompts or outputs by default, and that it has detected no impact on its zero data retention services. That can be true while chat transcripts still leak, because the Notion data was not ordinary inference traffic. It was a dataset sent for classification jobs, and datasets are one of the categories Fireworks says were accessed.
Two points remain unstated. Fireworks' bulletins do not mention Notion, and Notion's notice does not say what retention terms applied to the Insights data. Notion also says it previously used data that customers volunteered through an early access programme to experiment with training task-specific models on Fireworks.
The lesson for anyone weighing AI privacy claims is specific. A retention promise usually describes one path through a system. Analytics, evaluation, batch jobs, and fine-tuning can each keep copies under different rules, at vendors the end user never chose.
What affected users should do now
Notion's advice is to treat the chats as exposed. It recommends rotating any passwords, API keys, or other credentials that someone pasted into Notion AI during the period, though the notice does not define the dates. Workspace owners can open a review tool from the help menu in the sidebar, under "Security incident review", to see which chats were involved. Insights is disabled while Notion investigates, and Notion has stopped serving open-weight models through Fireworks.
Companies that use Fireworks directly have separate work. Fireworks tells them to replace revoked keys and to rotate third-party credentials with the issuing provider, because deleting a stored secret does not invalidate the credential itself.
For everyone else, the habit worth keeping is simple: do not paste secrets into a cloud assistant, and assume that a prompt may be copied to subprocessors for purposes beyond the answer. Unlabelled transcripts can still identify people through what they say. Work that is too sensitive for that risk is a good fit for a model running on your own device, where no vendor holds a copy.