Qwen 3.8 Flash Next (125B) Achieves 100 Tokens/Sec on Consumer RTX 4090 Hardware
A significant inference optimization milestone has emerged within the open-source community, demonstrating that the 125-billion parameter Qwen 3.8 Flash Next model can run on consumer-grade hardware—specifically a single NVIDIA GeForce RTX 4090—at speeds reaching 100 tokens per second. By combining aggressive quantization techniques with advanced offloading and memory-efficient runtime architectures, this development drastically reduces the cost and infrastructure requirements for local execution of frontier-class open-weight models, challenging the necessity of dedicated enterprise datacenter GPUs for high-throughput inference.