local-llm

MLX is Apple's open-source ML framework optimized for Apple Silicon. It exploits unified memory to run and fine-tune LLMs on Macs.

Ollama runs open-weight LLMs like Llama 3 and Qwen locally with one command. Models, hardware needs, API, benchmarks, and alternatives explained.

Quantization shrinks LLMs 2-4x by storing weights in fewer bits. Learn INT8, INT4, GGUF, GPTQ, AWQ, and which quant to choose for your GPU.

Run open-source AI on your computer with Ollama. Llama 3.1, Mistral, and Phi compared, plus real hardware requirements and setup steps.