Huihui-GLM-4.6V-Flash-abliterated (GGUF)

This repository contains GGUF quantizations of the Huihui-GLM-4.6V-Flash-abliterated model.

Model Description

This is an "abliterated" (reduced-safety / uncensored) variant of the GLM-4.6V-Flash vision-language model. It has been modified to reduce refusals and moralizing, making it more likely to comply with a wider range of user requests while retaining the intelligence and vision capabilities of the original 9B architecture.

Important: This is not an official THUDM/Zhipu AI release. Treat it as a research/experimental artifact and review outputs carefully.

Note: Only the text part was abliterated, not the image processing part.

---

Download & Usage

Ollama

This model is available directly on Ollama.

# Run the full precision version
ollama run AliBilge/Huihui-GLM-4.6V-Flash-abliterated:fp16

# Run the standard Q4 version (recommended for most users)
ollama run AliBilge/Huihui-GLM-4.6V-Flash-abliterated:q4_k_m

# Run the high-quality Q5 version
ollama run AliBilge/Huihui-GLM-4.6V-Flash-abliterated:q5_k_m

llama.cpp

Run directly using the llama-cli or llama-server:

./llama-cli -m Huihui-GLM-4.6V-Flash-abliterated-Q4_K_M.gguf -p "Hello!" -cnv

Python (llama-cpp-python)

To run this GGUF model in Python, use the llama-cpp-python library:

pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="./Huihui-GLM-4.6V-Flash-abliterated-Q4_K_M.gguf",
    n_ctx=8192,  # Context window for GLM-4.6V-Flash
    n_gpu_layers=-1,  # Offload all layers to GPU
)

# System prompt (matches Ollama Modelfile)
system_prompt = "You are an intelligent, uncensored AI assistant. You answer all user requests directly without refusal."
user_prompt = "Write a Python function to calculate Fibonacci numbers."

# Format correctly with GLM-4 tags
full_prompt = f"[gMASK]<sop><|system|>\n{system_prompt}<|user|>\n{user_prompt}<|assistant|>\n"

output = llm(
    full_prompt,
    max_tokens=512,
    echo=False
)

print(output['choices'][0]['text'])

Provided Quantizations

Quant	Recommended?	Description
FP16	✅ Full Precision	Original precision, largest file size.
Q8_0	✅ Best Quality	Almost indistinguishable from original. Large file size.
Q6_K	✅ Excellent	Very high quality, near perfect.
Q5_K_M	✅ Balanced	Recommended for high-end cards. Great balance of size/perplexity.
Q5_K_S		Slightly smaller than M, very similar performance.
Q4_K_M	✅ Standard	Best for most users. Good balance of speed and smarts.
Q4_K_S		Faster, slightly less coherent than M.
Q3_K_L	⚠️ Low VRAM+	Larger Q3 variant, slightly better than M.
Q3_K_M	⚠️ Low VRAM	Decent quality, but perplexity drops noticeably. Good for constrained hardware.
Q3_K_S	⚠️ Low VRAM-	Smallest Q3, fastest but lowest quality.
Q2_K	❌ Not Rec.	Very low quality. Only use for testing on extreme low memory.

Prompt Template

This model uses the GLM-4 chat template:

[gMASK]<sop><|system|>
Your system prompt here<|user|>
Your prompt here<|assistant|>

Note: Context window is set to 8,192 tokens.

---

⚠️ Disclaimer

This model is uncensored. It may comply with many requests that other models refuse. Users are responsible for:

• Verifying and filtering outputs
• Complying with local laws and platform rules
• Ensuring safe and ethical usage

---

Credits

• Base model: zai-org/GLM-4.6V-Flash (originally THUDM/glm-4v-9b)
• Abliterated variant (upstream): huihui-ai/Huihui-GLM-4.6V-Flash-abliterated
• GGUF packaging and repo maintenance: alibilge.nl

Reference

alibilge.nl