An XDA writer reports running the Q4_K_M quantization of Qwythos 9B v2, based on Qwen3.5, on an 8 GB RTX 3070 at roughly 20–22 tokens per second with about 14,000 tokens of usable context. The model card describes reduced repetition and restored multi-token prediction, but some coding results remain below the base model. The developer’s claim of training on a large Claude-conversation dataset has not been independently validated, and does not mean access to Claude’s private internal reasoning. A claimed million-token context was not demonstrated in this 8 GB test. The result is evidence of practical small-model deployment, not broader equivalence.
Back to Tech
Tech
Qwythos 9B v2 runs on an 8 GB GPU, with important limits
A quantized RTX 3070 test shows practical local use, not verified Claude equivalence or million-token context.