Practical AI and machine learning engineering. LLM inference, tokenization, RAG pipelines, model deployment, vector databases, and the infrastructure behind modern AI applications.
54 articles
Benchmarks and architecture comparison of Ollama, vLLM, and llama.cpp. Tokens/sec at 7B through 70B, quantization trade-offs, concurrent throughput, VRAM requirements, and a clear decision framework for local dev, production, and edge.
Q8 KV cache halves VRAM with under 0.1% perplexity cost. Q4 K-cache is OK, Q4 V-cache hurts. Asymmetric Q4-K + Q8-V is the magic combo.
RTX 5090 unlocks Qwen 3.5 32B at Q5_K_M with 16K context. NVFP4 native gives 60-80% inference speedup over RTX 4090. Real benchmarks and build guide.
A practical guide to monitoring LLM applications in production -- input/output logging, cost tracking, quality metrics, and a comparison of LangSmith, Langfuse, and Arize.
Below 3M tokens/day, the API wins. 3-30M, cloud GPU wins. Above 30M sustained, hardware pays back in 18-24 months. Real 2026 numbers.
End-to-end guide to deploying ML models -- from ONNX export and FastAPI serving to Kubernetes GPU workloads, canary deployments, and Prometheus monitoring.
Three frontier open-weight models compared for coding in April 2026. Qwen wins on consumer GPUs, GLM-5.1 leads SWE-Bench Pro, DeepSeek V4 has 1M context.
A practical guide to choosing between prompt engineering and fine-tuning for LLMs -- techniques, costs, LoRA/QLoRA, and a decision framework for production systems.
Q5_K_M is the sweet spot for Qwen 3.5 GGUF. Full perplexity table, K-quants vs IQ-quants, NVFP4 on Blackwell, and picks by VRAM tier with framework flags.
New articles delivered to your inbox. No spam.