immiq/Qwen3.5-4B-Orinfer-E8P-Q4A8
image-text-to-text model by immiq
Older snapshot: check the original source for current information.
From the publisher
Model card & documentation
Source previewRead the publisher’s intended use, setup instructions, evaluations and limitations. The original model card is the source of truth.
中文 · Hugging Face · ModelScope A quantized version of Qwen/Qwen3.5-4B for Jetson AGX Orin 64GB / CUDA SM87, deployed with Orinfer v0.2.0. It supports text, images and multiple images, streaming Chat Completions, native MTP, prefix caching, continuous batching and resumable JSONL offline inference. Translation requests use enablethinking: false; thinking follows the original model's chat template. Two-stage residual E8P Q4 with coefficients [4, 1], signed-block128 rotation and beam-8 encoding over the original BF16 weights, paired with INT8 computation. Sensitive tensors retain the validated W8/FP16/BF16/FP32 mixed-precision policy. Deployed read-only GPU data, including scales, derived constants and precision exceptions: 2,608,622,592 bytes, approximately 4.478 bits per recorded GPU weight parameter. Stored safetensors: 2.61 GB across 4 shards. Context capacity: 262,144 tokens; KV cache is INT8 and persistent GDN state is FP32. Quantization seed: 20261002. The repository contains HF configuration/tokenizer assets, the native chat template, standard safetensors containers and cache/model.json. The descriptor fixes physical…
Read the full model card ↗ · Preview checked 2026-10-28T01:24:52.554Z
Inside the original model card — Document outline
Headings are captured from the source. Links open the publisher’s document, not a locally hosted copy.
Links from the model card
References supplied by the publisher, not independently verified endorsements. Check the destination before downloading files or entering credentials.
Documentation belongs to its respective authors. Reported project/model license: apache-2.0. A listing is not a grant of reuse or training rights. Confirm the document’s own terms at the source.
Model overview
immiq/Qwen3.5-4B-Orinfer-E8P-Q4A8 is a image-text-to-text model repository published by immiq on Hugging Face. A library was not reported.
This page summarizes Hub metadata. For intended use, training data, evaluation results and limitations, consult the original model card.
Model facts
- Task
- image-text-to-text
- Library
- Not reported
- Recent downloads (30 days)
- 0
- Cumulative likes
- 0
- Hugging Face trending score
- 0
- Reported safetensors parameters
- Not reported
- Architecture
- Qwen3_5ForConditionalGeneration
- License
- apache-2.0
- Access
- Not gated by Hugging Face
- Created
- 2026-10-19T06:03:32.000Z
- Last modified
- 2026-10-19T06:34:00.000Z
Compatibility and lineage
Language coverage was not reported.
Reported base models: Qwen/Qwen3.5-4B
Parameter count is not a RAM/VRAM requirement. Check precision, quantization, context length and runtime compatibility in the model card; no hardware or API-cost claim is inferred here.
Use and evaluate
Check license terms and access requirements first. Review the model card’s documented loading instructions, evaluations and safety limitations. Benchmark scores are not imported by this directory, and popularity is not an accuracy ranking.
No model weights are downloaded or executed by AltAPIs. Never enable remote model code without reviewing it.
Source and freshness
Source: Hugging Face Hub. Metadata observed 2026-10-19T06:43:13.200Z. Daily imports are snapshots, not real-time monitoring.
Popularity and source listings do not establish security, suitability, licensing rights or benchmark performance.