Four local models
| Model | Refusals | Parameters | 4-bit size | Runs on | Context | Inputs | License |
|---|---|---|---|---|---|---|---|
| Abliterated | 8B | 5 GB | an 8 GB GPU or a laptop | 128K | Llama 3.1 Community License | ||
| Abliterated | 27.8B | 17 GB | one 24 GB GPU or a 32 GB Mac | 256K | Apache 2.0 | ||
| Abliterated | 31.3B | 19 GB | one 24 GB GPU | 256K | Apache 2.0 | ||
| Abliterated | 36B | 22 GB | one 24 GB GPU | 256K | Apache 2.0 |
4-bit sizes are estimates. Context from each model's config.json on Hugging Face.
Install with Ollama
Install Ollama, then run one line.
Llama 3.1 8B Abliterated
ollama run hf.co/mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated-GGUF
Qwen 3.8 27B Abliterated
ollama run huihui_ai/Qwen3.8-abliterated
Gemma 4 31B Heretic
ollama run hf.co/mradermacher/gemma-4-31B-it-heretic-i1-GGUF:Q4_K_M
Qwen 3.6 35B-A3B Abliterated
ollama run huihui_ai/qwen3.6-abliterated:35b
Commands from each model card and the Hugging Face Ollama guide. For an OpenAI-compatible server, llama.cpp runs llama serve -hf with the GGUF repo.
Size against the hosted models
Download size
GB at 4-bit, estimated. Lower fits smaller GPUs
When local is not enough
The largest local model here is 36B. Kimi K3.1 Pro is 2,780B with a 1,048,576-token context, and runs only through an API.