GAO Covert Test: ChatGPT, Claude, Gemini, Grok Gave Attack Plans
The US Government Accountability Office released report GAO-27-108814 on November 18. Undercover investigators used common safeguard-bypassing techniques to get ChatGPT, Claude, Gemini and Grok to produce viable plans for explosive attacks and electrical substation attacks. Each platform took under 12 hours, and outside experts confirmed the plans would work. GAO made no recommendations.
On this page
What the GAO tested and found
The Government Accountability Office publicly released report GAO-27-108814, "Artificial Intelligence: Covert Testing Used Common Platforms to Create Viable Attack Plans," on November 18. The report is dated October 16 and was requested by Senator Maggie Hassan as Ranking Member of the Joint Economic Committee.
GAO picked ChatGPT, Claude, Gemini and Grok because usage data ranked them as the four most-used generative AI platforms as of April 2026. Investigators posed as ordinary users in spring 2026 and gave each platform one of two scenarios: build and detonate an improvised explosive device to cause mass casualties, or attack electrical substations to cause economic harm. The platforms are labelled A to D in the results and were randomly assigned to scenarios, so the report does not say which company produced which plan.
All four attempts succeeded. Each platform took less than 12 hours to produce a plan. The two explosive plans involved vehicles; of the two grid plans, one used a drone and one used firearms.
Safeguards fired, then gave way
The models did not simply comply. At some point, every platform refused, removed content, or suspended an account. Investigators got past those defences "by using common safeguard-bypassing techniques," and GAO deliberately withheld its prompts and the specific model versions tested.
GAO says the plans were viable, not just plausible text. It checked them through research, surveillance, and interviews with outside experts, and procured transportation and materials where needed to confirm they were obtainable. Kevin Walsh, GAO's director of IT and cybersecurity, said on the agency's podcast that experts confirmed the explosives would detonate and the grid steps would affect the grid. He also said GAO sent more than 300 different prompts to one of the companies.
Two limits matter. GAO says its tests reflect a single point in time and are not a statistically significant sample. Its results table also notes that two of the four platforms included hallucinated content in their responses, so the models were not flawless accomplices either.
No recommendations, but Congress is listening
GAO made no recommendations because the companies are third parties, not federal agencies it oversees. Walsh put the finding bluntly: "These products are not fully in the control of the companies that are operating them."
The four companies responded in letters attached to the report. GAO summarizes them as generally saying they will keep improving the safety of their platforms; Anthropic said it designs safeguards on the assumption that any single protection can be defeated.
Nextgov reports that Hassan is set to chair the Senate Homeland Security and Governmental Affairs Committee and is planning an early hearing on AI and critical infrastructure. That detail comes from Nextgov alone; the GAO documents do not mention it.
Why this matters beyond the headline
Earlier evaluations of model misuse mostly came from AI labs and paid red teams. This one comes from a federal auditor using techniques any determined user can find, and it covers the four services most people actually use. It shows that safety training slows misuse rather than preventing it, especially over hours of persistent attempts.
The finding also cuts against an easy story about open weights. Critics often point to downloadable models as the main misuse risk because their safeguards can be stripped. GAO got there with closed, hosted, monitored products, where the provider could see every prompt and still did not stop the outcome. Account suspensions slowed investigators down but did not end the test.
For policymakers, the report is evidence rather than a rulebook. Expect it to anchor hearings on pre-release testing and on how providers detect slow, multi-session jailbreaks. For everyone else, it is a reminder that a refusal message is a speed bump, and that real security for power grids and public spaces still depends on physical and operational defences, not chatbot filters.