Gemma-4-E4B
Vision Language Model

Mixed-precision 4-bit quant (6.234 bpw) · 8B parameters · MLX · Apache 2.0

MLX Apple Silicon 6.16 GB Q4_K_M mixed

Try the Demo

This interactive demo showcases what Gemma-4-E4B VLM can do. The actual model runs locally on Apple Silicon with mlx-vlm. Below is a sandbox that simulates the interaction pattern.

Drop an image here or click to browse

PNG, JPG, WEBP

Upload an image and ask a question to see the model's response.

In demo mode, the response is simulated. Run locally with mlx-vlm for real inference.

About This Model

Model

Base
google/gemma-4-E4B
Fine-tuned
google/gemma-4-E4B-it
Further fine-tuned
llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic
Quantized by
zecanard (via mlx-vlm 0.6.3)
Parameters
8B
License
Apache 2.0

Quantization

Format
MLX 4-bit mixed
Bits per weight
6.234 bpw (effective)
Quant method
--quantize --q-group-size 32 --quant-predicate mixed_4_6
File size
~6.16 GB
Precision
Mixed: 4-bit where possible, higher precision where it matters

Capabilities

  • Visual question answering — ask questions about image contents
  • Image description — detailed captioning of scenes, objects, text
  • OCR-like reading — extract and interpret text from images
  • Visual reasoning — infer relationships, count, compare
  • Tool calling — bugfixed in this quant
  • Thinking — enabled by default (can be disabled in chat template)

Customization Notes

  • Thinking enabled by default. The chat template sets enable_thinking = true. Delete that line or set to false to disable.
  • Reasoning section parsing should recognize <|channel>thought as Start String and <channel|> as End String.
  • Bugfix included for tools calling.
  • Aware of the current date.

Run Locally on Apple Silicon

This model is MLX-only. It runs on Macs with Apple Silicon (M1/M2/M3/M4). Install mlx-vlm and generate directly from the command line.

1

Install mlx-vlm

pip install -U mlx-vlm
2

Generate (vision + text)

mlx_vlm.generate \
  --model zecanard/gemma-4-E4B-it-ultra-uncensored-heretic-MLX-4bit-mixed_4_6 \
  --max-tokens 100 \
  --temperature 0 \
  --prompt "Describe this image." \
  --image /path/to/your/image.jpg
3

Chat (interactive)

mlx_vlm.chat \
  --model zecanard/gemma-4-E4B-it-ultra-uncensored-heretic-MLX-4bit-mixed_4_6 \
  --image /path/to/your/image.jpg

Multi-turn conversation about the image.

Requirements: macOS 14+, Apple Silicon (M1 or later), Python 3.10+, ~7 GB free RAM for the model (6.16 GB weights + runtime overhead).

Example Prompts