What you need to know
The right local AI model is the smallest compatible artifact that performs your real tasks reliably. Check the exact file, quantisation, runtime, modality, memory needs, storage headroom, license, and supported prompt format. Then measure first output, answer quality, cancellation, heat, and repeated runs on your own device.
Choose from the task backwards
Start by writing down what the model must do: short chat, rewriting, document questions, multilingual work, image understanding, or code. A model family name does not guarantee that every artifact supports the same inputs, license, context length, or runtime. Verify the exact checkpoint and file offered by the app.
Parameter count is only one signal. Quantisation changes file size and memory behaviour, while architecture and runtime optimisation affect speed. Available storage is not the same as usable memory. The operating system, graphics stack, active applications, context length, and temporary buffers all compete for resources during generation.
Compare models with controlled tests
Use the same device, app version, runtime settings, prompt, and approximate context when comparing two models. Record time to first visible output separately from generation speed, then assess whether each answer follows instructions and handles the intended subject. Repeat prompts after the phone warms up because sustained performance can differ from a single cold run.
A smaller model that starts promptly, stops cleanly, and answers a narrow task well may be more useful than a larger model that strains the device. Keep model cards, licences, and runtime compatibility visible in the decision. Treat universal winner claims cautiously unless they are supported by reproducible tests on hardware comparable to yours.