Quatfit Mini

8B Multimodal Foundation Model · Up to 4× Faster Inference · 131K Context

Quatfit Mini is an 8-billion-parameter multimodal foundation model built on the Quatfit 1 Architecture. It natively understands text, images, and audio, supports 131K token context, delivers up to 4× faster inference than comparable 8B models, and is optimized for agentic AI workflows — all on consumer GPUs.

Native Transformers support. Quatfit1 is integrated directly into Hugging Face Transformers via Auto Classes — no trust_remote_code=True required.

Model Summary

Property	Value
Parameters	8B
Architecture	Quatfit 1 (dense decoder-only + ViT + Conformer)
Context Length	131,072 tokens
Precision	BF16
Vocabulary	262K tokens
Language	English, Hindi, multilingual
Modalities	Text, Image, Audio

Intended Uses

Primary use cases

Agentic AI and autonomous workflows
Multimodal reasoning (text + image + audio)
Long-document analysis and codebase understanding
Tool calling and orchestration
Coding assistance and API development
Visual Q&A, OCR, diagram understanding
Audio transcription and reasoning
Research copilots and productivity automation

Out-of-scope

Repository-scale software engineering
Competitive programming
Enterprise-scale refactoring
Performance-critical code synthesis
High-stakes domains (legal, medical) without additional safeguards

How to Use

Installation

pip install git+https://github.com/Jatinverma0786/transformers.git

Once PR #47081 merges, use the official package: pip install "transformers[torch]".

Load Model

import torch
from transformers import AutoProcessor, AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained(
    "Quatfit/Quatfit-Mini",
    torch_dtype="auto",
    device_map="auto"
)
processor = AutoProcessor.from_pretrained("Quatfit/Quatfit-Mini")

Text Generation

messages = [{"role": "user", "content": "Write a Python function to merge two sorted lists."}]

inputs = processor.apply_chat_template(messages, tokenize=True, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0]))

Image Understanding

messages = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "Explain this image in detail."},
            {"type": "image", "image": "diagram.png"}
        ]
    }
]

inputs = processor.apply_chat_template(messages, tokenize=True, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0]))

Long Context (131K)

with open("long_document.txt") as f:
    text = f.read()

messages = [{"role": "user", "content": f"Summarize:\n\n{text}"}]
inputs = processor.apply_chat_template(messages, tokenize=True, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=1024)
print(processor.decode(outputs[0]))

Performance

Configuration	Relative Speed	VRAM
Standard 8B (reference)	1×	~16 GB
Quatfit Mini BF16	2.5×	~16 GB
+ Speculative Decoding	3.9×	~16 GB
GGUF Q4_K_M	4.1×	~5 GB

Benchmark Scores

Domain	Accuracy
Overall	89.1%
CLI	95.0%
Exams	93.3%
Coding	92.5%
Agentic Tasks	92.5%
Science	91.7%
Finance	90.0%
Security	90.0%
Social Intelligence	90.0%
Reasoning	88.9%
Expert Knowledge	83.8%
Mathematics	81.3%

Architecture

Text Decoder

Component	Value
Parameters	8B
Layers	42
Hidden Size	2560
Attention Heads	8
KV Heads	2 (grouped query attention)
Shared KV Layers	18
Feed Forward	GeGLU
Precision	BF16
Context Length	131,072

Vision Encoder

Vision Transformer (ViT), 16 layers, patch size 16×16
280 visual tokens, pan & scan high-resolution support

Audio Encoder

Conformer, 12 layers, streaming compatible, causal chunk attention

Key Optimizations

Flash Attention 3
Sliding window / Grouped query attention
KV cache sharing
Speculative decoding

Training

Trained on approximately 10 trillion tokens:

Data	Description
Web	Large-scale web corpus
Code	Python, JavaScript, and others
Academic	Mathematics, scientific literature
Books	Wikipedia and book corpus
Multilingual	Including Hindi
Vision	Image-text pairs
Audio	Speech transcriptions

Post-training: SFT → RLHF → Constitutional AI alignment.

Cross-Platform Support

Quantization	VRAM	Platforms
Q4_K_M	~5 GB	Ollama, LM Studio, llama.cpp, Jan, Open WebUI
Q5_K_M	~6 GB	Ollama, LM Studio, llama.cpp
Q6_K	~7 GB	llama.cpp, Jan
Q8_0	~9 GB	Full precision alternative

Hardware

Recommended: NVIDIA RTX 3090, RTX 4090, A6000, H100
Minimum (GGUF Q4): Any GPU with ~5 GB VRAM
CPU: GGUF via llama.cpp / Ollama

Responsible AI

Quatfit Mini may produce unfair, unreliable, or offensive outputs. Be aware of biases, factual inaccuracies, and quality gaps in non-English languages.

Mitigations: Review outputs for safety-critical use, apply RAG for factual grounding, and use application-level filters.

Citation

@article{quatfitmini2026,
  title={Quatfit Mini: A Compact Multimodal Foundation Model with Up to 4× Faster Inference},
  author={Quatfit AI Research},
  year={2026}
}

License

Quatfit Non-Commercial License v1. Commercial licensing available through Quatfit AI Research.

🤗 View on Hugging Face · 📄 Read Technical Report

Downloads last month: 460

Safetensors

Model size

8B params

Tensor type

F32