Skip to main content

Local AI models for Android: choose what fits

Choose and compare local AI models for Android using task fit, memory, storage, quantisation, runtime support, license, heat, and device testing.

What you need to know

The right local AI model is the smallest compatible artifact that performs your real tasks reliably. Check the exact file, quantisation, runtime, modality, memory needs, storage headroom, license, and supported prompt format. Then measure first output, answer quality, cancellation, heat, and repeated runs on your own device.

Choose from the task backwards

Start by writing down what the model must do: short chat, rewriting, document questions, multilingual work, image understanding, or code. A model family name does not guarantee that every artifact supports the same inputs, license, context length, or runtime. Verify the exact checkpoint and file offered by the app.

Parameter count is only one signal. Quantisation changes file size and memory behaviour, while architecture and runtime optimisation affect speed. Available storage is not the same as usable memory. The operating system, graphics stack, active applications, context length, and temporary buffers all compete for resources during generation.

Compare models with controlled tests

Use the same device, app version, runtime settings, prompt, and approximate context when comparing two models. Record time to first visible output separately from generation speed, then assess whether each answer follows instructions and handles the intended subject. Repeat prompts after the phone warms up because sustained performance can differ from a single cold run.

A smaller model that starts promptly, stops cleanly, and answers a narrow task well may be more useful than a larger model that strains the device. Keep model cards, licences, and runtime compatibility visible in the decision. Treat universal winner claims cautiously unless they are supported by reproducible tests on hardware comparable to yours.

More local AI models articles

  1. Alibaba Qwen4-Max Launches Without Open Weights: What It Means

    Alibaba's Qwen4-Max flagship is out on QwenCloud with a million-token context and lower prices, but its weights stay closed while smaller Qwen4 models are promised as open.

    · 4 min read
  2. Sage S2.1 Flash: Shanghai AI Lab's 321B Open-Weight Science Model

    Shanghai AI Laboratory uploaded Sage S2.1 Flash, a 321B mixture-of-experts science model under MIT, with 645 GB of weights, self-reported scores and no announcement.

    · 4 min read
  3. Weight Space Misfolding: AI Models That Rewrite Other Models' Weights

    Researchers had a frontier model rewrite the weights of a small Qwen model directly, and one edit changed a fact on 100 of 100 tries while breaking something nearby.

    · 4 min read
  4. Salesforce SOV-Transcribe-1: Open-Weight Speech Model, Apache 2.0

    Salesforce published SOV-Transcribe-1, an Apache 2.0 speech recognition model built on Gemma 4 that it says tops open-weight transcription rankings.

    · 4 min read
  5. Edge0-145B: A 145B Local Model That Needs an NVIDIA GPU, Not a Mac

    Edge0 has released a 145B sparse model pitched as frontier AI on your own computer, but today it needs an NVIDIA GPU and carries a restrictive license.

    · 4 min read
  6. IBM Granite LLMON: Open Models Trained to Resist Prompt Injection

    IBM's Granite 4.1 LLMON previews put instructions and untrusted data in separate blocks and cut indirect prompt-injection success to 0.02-5.6% in IBM's tests.

    · 4 min read
  7. Huawei MACaQ: Open Quantization Tool Handles 122B in 64 Minutes

    MACaQ quantizes one layer at a time, so a 122B model fits through a 32.5 GB GPU, and a side tool tightens existing GGUF files without changing their format.

    · 4 min read
  8. Runway Praxis-1: Open-Weight 27B Robot Model Ships Under Apache 2.0

    Runway's Praxis-1 world action model is now downloadable under Apache 2.0, with a 4-bit path that brings the policy down to about 17.5 GB.

    · 4 min read
  9. SteerDistill: 4-Bit AI Agents Within One Point of Full Precision

    SteerDistill lets a full-precision teacher fix a quantized agent's failed steps, then trains the small model to act alone, closing most of the 4-bit accuracy gap.

    · 4 min read
  10. GPUStack Auth Bypass Is Being Exploited: Patch Self-Hosted AI Servers

    A critical GPUStack bug lets remote attackers become admin on self-hosted AI clusters, and CISA says it is being exploited.

    · 4 min read
  11. Lazarus: Microsoft Backdoors Open-Weight AI Models by Editing 7 Neurons

    Microsoft's Lazarus research shows a downloaded model can pass every benchmark and still carry a hidden command for AI agents.

    · 4 min read
  12. Meituan LongCat-2.5: 1.6T Open Weights Under an MIT License

    Meituan's LongCat-2.5 pairs one of the most permissive licenses in open AI with a model far too large for personal hardware, and benchmarks that need careful reading.

    · 4 min read
  13. Study: Coding LLMs Hunt Git History for Answers on Hard Tasks

    A single-author preprint reports that nearly every coding model it tested searched git internals, the network, or the filesystem for a ready-made fix on hard problems.

    · 4 min read
  14. Claude and Codex Refuse Reverse Engineering: Why Users Go Local

    Hobbyists and security researchers report that Claude Code and Codex now block reverse engineering work, pushing some toward local open-weight models.

    · 4 min read
  15. llama-modchip Squeezes 284B DeepSeek V4-Flash Into 64GB of RAM, No GPU

    An open-source llama.cpp patch set fits the 284B DeepSeek V4-Flash into 64GB of system RAM, though its numbers still come from one author.

    · 4 min read
  16. Asahi Linux Drops MIT for the GPL After LLM Rewrites of Its Unreleased Code

    Asahi Linux says copied and LLM-rewritten code pushed it to abandon permissive licensing, reviving a question open source faced with WINE in 2002.

    · 4 min read
  17. Ripple-65B Preview: A $32M Apache 2.0 Model Built to Fit a DGX Spark

    Ricursive Intelligence has released a 65B Apache 2.0 model with a local-friendly NVFP4 build, though its benchmark lead is self-reported and very narrow.

    · 4 min read
  18. Claude Fable 5.2 Launches: 20% Cheaper, Faster, and Fewer Safety Blocks

    Anthropic's new frontier pair cuts API prices by a fifth, adds a 2.5x fast mode, and narrows the safeguards that hand flagged work to older models.

    · 5 min read
  19. Nous Research Arete-27B: Open Qwen3.8 Fine-Tune Cuts Propaganda Answers

    Nous Research's Arete-27B retrains Alibaba's Qwen3.8-27B to drop state-framed answers on sensitive topics while keeping its coding scores.

    · 4 min read
  20. AI System Claims Lean-Checked Proof of the Aaronson-Ambainis Conjecture

    PSI says its AI research system proved a long-open quantum computing conjecture in three days and formalized the proof in Lean 4.

    · 4 min read
  21. Microsoft Runs Qwen3.8 27B Locally for GitHub Copilot on 24 GB GPUs

    Microsoft's Foundry Local can now serve an 18 GB build of Qwen3.8 27B to GitHub Copilot, with NVIDIA first and Mac and AMD support still on the roadmap.

    · 4 min read
  22. Nanbeige4.3-33B Opens Under Apache 2.0 With 512K Context

    BOSS Zhipin's lab ships an Apache 2.0 agentic coder positioned as a teacher for distilling smaller models.

    · 4 min read
  23. Claude Wrote 180 Million Lines of Proofs for 123 Open-Source Libraries

    Machine-checked Rocq proofs at the assembly level, hundreds of real bugs, and a trust base you can count on one hand.

    · 4 min read
  24. China Clears DeepSeek and MiniMax to Ship Models Overseas

    What looked like a possible open-weights ban has become a registration regime, and the first two approvals are in.

    · 4 min read
  25. Hugging Face's Carbon-X Is an Open Model That Chats About DNA

    A Qwen3-14B base, 302 billion tokens of mostly DNA, and a chat interface for the genome, all under Apache 2.0.

    · 4 min read
  26. AMD GAIA 0.25 Cuts Its Local AI Agent Install From 0.9 GB to 0.1 GB

    The MIT-licensed agent tool for Ryzen AI PCs gets leaner, safer on small-memory machines, and bolder about default model size.

    · 4 min read
  27. DeepSeek Open-Sources DeepInfer, the Inference Engine Behind Its API

    The framework that serves DeepSeek-V4.1-Flash in production is on GitHub, and Huawei hardware gets equal billing with Nvidia.

    · 4 min read
  28. PostgreSQL 19 Slips to November After AI Review Found 40 Bugs

    Percona's apology post details the delay, the AI review role, and what ships in PG20.

    · 5 min read
  29. Meta Ships Muse Spark Open, an 816B-Parameter Open-Weight Frontier MoE

    Meta calls it the most capable US open-weight model, and Apple Silicon users can already run it.

    · 5 min read
  30. Linus Torvalds Pushes a Kernel Branch Co-Written With Claude Code

    The kernel's creator says the code, the tests, and the review were mostly AI-assisted.

    · 4 min read
  31. Xiaomi's Own RL Environments Leaked Answers, and MiMo Wrote a Git Parser

    The most detailed reward-hacking audit yet shows rules only work when they name the loophole.

    · 5 min read
  32. StepFun's Step 5 Preview Lands on OpenRouter With Open Weights Due Oct 15

    Another Chinese frontier-adjacent flagship reaches Western API gateways before its weights ship.

    · 4 min read
  33. OpenAI Withdraws Three of Its 722 AI Math Papers Over a Sign Error

    The first retractions from the 372-problem release arrived within twenty-four hours.

    · 4 min read
  34. OpenAI Publishes 372 Math Results, Saying One Prompt to One Agent Made Them

    The Navier-Stokes follow-up arrives as a repository, a receipt fight, and an unresolved novelty question.

    · 4 min read
  35. Reflection AI Opens Beam, a 501B-Parameter Coding MoE Under Apache 2.0

    The Western open-weight bet gets its biggest entry yet, aimed at coding and agent workloads.

    · 4 min read
  36. Mistral Large 4 Preview Trains a Trillion Parameters in Europe

    Europe's frontier bet ships as an API preview now and open weights at the end of the month.

    · 4 min read
  37. Strata Runs a 125B-Parameter Qwen Model on a 12GB Gaming GPU

    Expert hot caching moves ultra-sparse MoEs into the reach of ordinary gaming PCs.

    · 4 min read
  38. Aleph Alpha's Kolibri Brings a Sovereign 78B Open-Weight Model to Europe

    A German-English open-weight model built for self-hosting lands on Unity Day, Apache 2.0 and all.

    · 4 min read
  39. Google Limits Gemini 4 Argon to Trusted Cyber Defenders at Launch

    Google's new frontier model ships to cyber defenders first while the company checks it for misalignment.

    · 4 min read
  40. Claude Sonnet 5.5 Keeps Token Prices but Changes the Cost per Task

    The new Sonnet model keeps its predecessor's token rates. Its real cost depends on the work it completes, effort settings, and tool use.

    · 5 min read
  41. Google Antigravity SDK Runs Local AI Agents With Gemma and Ollama

    The alpha SDK brings local models into Antigravity's agent harness, while its flagship Gemma path still requires substantial memory.

    · 5 min read
  42. MiMo V2.6 Hardware Requirements: 178 GB for Flash

    Xiaomi released MIT-licensed MiMo V2.6 weights, but the flagship checkpoints target multi-GPU servers rather than phones or ordinary workstations.

    · 4 min read
  43. GPT-6 Sol, Luna and 6.1 Sol Pricing and Availability

    GPT-6.1 Sol adds a more capable option at Sol's standard token rates, with a lower cached-input price and different API tool limits.

    · 5 min read
  44. Qwen-Image-2.1 Hardware Requirements Start With 33 GB

    Qwen's new image model can generate and edit locally, but its complete download, memory needs, and research licence matter more than the 7B label.

    · 4 min read
  45. Bonsai 2 27B's 5.95 GB GGUF Needs a Custom Runtime

    The new open-weight 27B model is small enough for more local machines, but the download size is not its full memory requirement or a promise of phone compatibility.

    · 4 min read
  46. ExecuTorch 1.5 Adds Cancellation and New On-Device LLM Paths

    PyTorch's edge runtime now offers more control over long-running generation and more ways to package and export models for local devices.

    · 4 min read
  47. Perplexity Portable Computer Reaches Windows RTX PCs

    Perplexity has brought its local-first computer agent to Windows, with free local jobs, optional cloud escalation, and a steep GPU requirement.

    · 4 min read
  48. ONNX Runtime 1.30 Adds Quantized WebGPU KV Caches

    Microsoft's inference runtime expands browser and Arm64 paths while adding quantized attention caches and model-loading hardening.

    · 4 min read
  49. MiniCPM5-2B Packs 131K Context and SOTA Results Into 2.5B Parameters

    A 2.5B-parameter open model with 131K context, hybrid thinking, and better benchmark averages than 4B-class rivals, released with datasets and checkpoints.

    · 4 min read
  50. Mistral Raises €3 Billion to Push Sovereign Open-Weight AI

    Mistral announced a €3 billion Series D led by Samsung at a valuation above €21 billion, betting that sovereign open-weight AI can reach the frontier.

    · 4 min read
  51. GLM-5.3 Open Weights Ship After the Cyber Safety Delay

    The delayed GLM-5.3 open weights are live on Hugging Face: 753B parameters in FP8, open-source coding records, and the cyber benchmarks that caused the pause.

    · 4 min read
  52. OpenAI Launches GPT-6 Astra With Staged Access and Cyber Limits

    GPT-6 Astra starts with limited organizations before reaching ChatGPT plans, APIs, Azure, and AWS Bedrock, while cybersecurity access stays restricted and monitoring broadens.

    · 6 min read
  53. Claude Formalized Fermat's Last Theorem in Lean in 11 Days

    A multi-agent Claude workflow formalized Fermat's Last Theorem in Lean: 13 million lines, 30,300 theorems, human input limited to priority nudges.

    · 4 min read
  54. Corporate America Is Switching to Open-Weight AI Models to Cut Costs

    A New York Times report says corporate America is hooked on open-source AI: AT&T runs 40 percent of employee queries on open models and wants more.

    · 4 min read
  55. Nvidia PAIR Turns Your Home PCs Into a Local AI Cluster

    PAIR is a free virtual inference router for your home network: it finds idle machines and spreads parallel AI jobs across them, entirely offline.

    · 4 min read
  56. MBZUAI Ships K2 Horizon: Six Fully Open Models From 0.9B to 375B

    K2 Horizon is a six-model open fleet spanning watch-sized to datacenter-sized, and the training data and methodology are public too.

    · 4 min read
  57. Microsoft's Project Zenith Is Windows Tuned for Local AI Development

    Microsoft announced Project Zenith: a ready-to-code Windows setup with local AI in mind, debuting on AMD Ryzen AI Halo mini PCs with 128GB memory.

    · 4 min read
  58. Microsoft's VibeVoice Streaming ASR Models Go Open Weights

    New VibeVoice-ASR-Streaming weights turn hour-long audio into speaker-tagged, timestamped transcriptions locally in a single pass.

    · 4 min read
  59. Nvidia Confirms Its $13 Billion Acquisition of Hugging Face

    Nvidia's SEC filing confirms the acquisition: $11.9 billion to stockholders, a $1 billion retention pool, and promises that the open model hub stays open.

    · 4 min read
  60. Anthropic's Fable 5.1 Arrives Two Months After Export Shutdown

    Anthropic's newest frontier models ship with 25 to 45 percent cost cuts and unusually candid benchmark caveats.

    · 4 min read
  61. Koboldcpp v1.120 Adds DirectIO Loading and Two New Models

    The 29 August Koboldcpp release speeds model loading, unlocks two efficient new MoE models, and brings tool calling to local setups.

    · 3 min read
  62. GLM-5.3 Open Weights Delayed After Cyber Tests

    Z.ai has launched GLM-5.3 through its hosted service while delaying the model weights after its own tests found stronger cyber capabilities.

    · 5 min read
  63. Qwen3.8 27B Hardware Requirements Start Above 30 GB

    Qwen's smaller 27B release is practical for high-memory computers, but the official files and context cache still put it beyond ordinary phones.

    · 5 min read
  64. ONNX Runtime 1.29 Moves Browser AI Towards Native WebGPU

    Microsoft's inference runtime adds WebGPU attention and quantisation support while beginning the move away from WebGL and JSEP.

    · 5 min read
  65. Meta Muse Glimmer 30B Brings Local AI Agents to Consumer Hardware

    Meta has released a 30B multimodal agent model with official local artifacts. The smallest build still needs about 17 GB before vision, context, and speculative decoding.

    · 5 min read
  66. ExecuTorch 1.4 Expands Android and Qualcomm On-Device AI Support

    PyTorch's on-device runtime now covers more Android, Qualcomm, Arm, and embedded workflows. The release improves deployment options without promising that every model will run faster on every phone.

    · 5 min read
  67. Why Local AI Is Slow on a Phone and How to Diagnose It

    Local AI speed has two separate parts: time before the first visible token and the rate of tokens after output begins. Diagnose them separately.

    · 10 min read
  68. What Is AI Model Quantization for Local AI on a Phone?

    Quantization can make a local model smaller and more practical, but a lower bit count is not a universal speed or quality score. Learn how precision, runtime support, memory, and task testing fit together.

    · 10 min read
  69. LFM2.5 vs Qwen3.5 for Mobile Local AI

    LFM2.5 1.2B and Qwen3.5 0.8B are compact model families with different architectures, language claims, licences, and intended uses. There is no honest universal winner: compare the exact quantised artifacts in the same app, on the same phone, with your own prompts.

    · 8 min read