The short URL: https://ask.yugatech.com/w67a

Alibaba’s Qwen team has released Qwen3.8-Flash-Next, an open-weight multimodal model that previews the architecture planned for the Qwen4 family. The experimental model uses 125 billion total parameters but activates about 6 billion for each token, aiming to reduce inference costs while retaining capability for coding, tool use, and long-context tasks.

Qwen describes the release as a research preview rather than a finished Qwen4 model. Its Hugging Face release includes weights and configuration files compatible with Transformers, vLLM, SGLang, and TokenSpeed. Developers can also run quantized versions through tools such as Ollama, although the largest local packages require substantial memory.

Qwen3.8-Flash-Next uses a mixture-of-experts design. Instead of running the entire 125B-parameter network for every token, a router selects a small group of experts. Qwen says the model has 512 experts, with 10 routed experts and one shared expert active per token.

The model also adds 51 billion parameters through an n-gram embedding table. This stores representations for short sequences such as pairs and triplets of tokens. Qwen presents the approach as another way to scale model capacity without adding the same level of computation required by a larger dense model.

Its attention system combines Gated DeltaNet layers with Qwen Sparse Attention. QSA processes context in micro-blocks rather than selecting individual tokens, which Qwen says can reduce latency when handling long inputs. The model has a native 256K-token context window and can be extended to 1 million tokens with YaRN.

Multimodal and agent-focused capabilities

Qwen3.8-Flash-Next is a causal language model with a vision encoder. It accepts text and images, and its architecture is designed for agentic coding, tool use, and other tasks that require multiple steps rather than a single response. Thinking mode is enabled by default, with the option to disable it for requests that need faster answers.

Qwen’s published comparisons report a 62.5 score on SWE-bench Pro, 81.0 on SWE-bench Multilingual, 73.9 on CoWorkBench, and 91.7 on GPQA Diamond. It also reports 91.9 on LiveCodeBench v6. These are vendor-provided results, and benchmark scores can vary with the harness, prompt, sampling settings, and hardware. They should not be treated as independent performance verification.

Qwen3.8-Flash-Next is available as an open-weight release through Hugging Face, with additional formats and quantizations appearing through community distribution channels. Qwen says the production Qwen3.8-Flash API model is based on this architecture and adds features such as a 1 million-token context window and built-in tools.

Source

Qwen3.8-Flash-Next specs:

  • Type: Causal language model with vision encoder
  • Parameters: 125B total, approximately 6B activated per token
  • Additional parameters: 51B n-gram embedding and 4B multi-token prediction
  • Context: 256K tokens natively, extendable to 1M with YaRN
  • Modalities: Text and image input
  • Architecture: Mixture of experts with Qwen Sparse Attention and Gated DeltaNet
  • License/availability: Open-weight release through Hugging Face
  • Production successor: Qwen3.8-Flash API model based on this architecture