Tag: vllm

2 articles

Fix: Outlines Not Working — Backend Setup, Pydantic Schemas, Regex, Choice, and Slow Sampling

May 20, 2026

How to fix Python Outlines errors — model backend missing, JSON schema vs Pydantic, regex pattern compilation slow, choice list timing, vLLM/Transformers/Ollama wiring, and streaming structured outputs.

python outlines llm constrained-generation pydantic vllm

Fix: vLLM Not Working — CUDA OOM, Model Loading, and API Server Errors

Apr 9, 2026

How to fix vLLM errors — CUDA out of memory during model load, tokenizer mismatch with HuggingFace, tensor parallel size does not match GPU count, KV cache exceeds memory, OpenAI API compatibility issues, and max_model_len too large.

python vllm llm inference machine-learning gpu debugging