- What: Xiaomi’s MIT-licensed MiMo-V2.6-Flash-RL (309B params, 15B active) runs on AWS EC2 today.
- Why it matters: Bedrock Custom Model Import rejects its architecture and its 1M-token context. EC2 is the only AWS route.
- What to do: Size a 4x H200 node. Pull the vLLM mimo-v26 image. Budget for a 173GB checkpoint plus KV cache.
- Key stat: Minimum viable load is roughly 208GB of VRAM at FP8, before a single token of context is cached.
Running MiMo V2.6 on AWS today means self-hosting on EC2, not Bedrock. Xiaomi’s MiMo-V2.6-Flash-RL is a 309-billion-parameter MoE model with 15 billion active parameters. Bedrock Custom Model Import doesn’t support its architecture and caps context at 128K tokens, well under MiMo’s 1M-token window. A 4x H200 EC2 node is the practical minimum for a working deployment.
MiMo V2.6 is Xiaomi’s second-generation open-weight foundation model family. It was released September 22, 2026 under an MIT license. It ships in two sparse mixture-of-experts configurations: MiMo-V2.6-Pro-RL (1.02 trillion total parameters, 42 billion active per token) and MiMo-V2.6-Flash-RL (309 billion total parameters, 15 billion active per token). Both are natively omnimodal across text, image, video, and audio, with a 1-million-token context window.
Xiaomi dropped MiMo V2.6 on Hugging Face yesterday. Within hours it was the top story on Hacker News, with over a thousand points. The pitch is hard to ignore: an MIT-licensed model that scores 89.9 on Terminal Bench 2.1, ahead of Claude Opus 5’s 89.1. The weights are free to download and run. Naturally, PHP and Node teams already living on AWS want to know if they can get MiMo V2.6 on AWS the easy way: through Bedrock, with a managed endpoint and a per-token bill. I tested this the morning the model shipped. The honest answer is not yet, not through Bedrock. I hit the 128K context ceiling and an unsupported-architecture error on my first Custom Model Import attempt, then pivoted to EC2. Here’s what actually worked, what it costs per hour, and the AWS-specific gotchas that will burn you if you skip straight to the vllm serve command.
What Is MiMo V2.6 and Why Does It Matter for AWS Developers?
MiMo V2.6 matters because it beats several frontier closed models on agentic benchmarks, under a permissive license you can run in production. On AutomationBench v1.0.6, MiMo-V2.6-Pro-RL scores 53.1, per Xiaomi’s published model card. Claude Opus 5 scores 50.3 and GPT-5.6 Sol scores 45.8. That’s a meaningful gap on a benchmark built to measure real tool-use and multi-step automation, the exact workload most backend teams care about.
For developers already running PHP or Node services on AWS Lambda, the appeal is obvious. There’s no per-token API bill, no rate limits, and full control over the inference stack. The catch: MiMo V2.6 is a mixture-of-experts model, not a dense one. Self-hosting means provisioning serious GPU infrastructure, not spinning up a small container.
The Flash-RL variant is the one most teams should target. It trades some benchmark points for a much smaller footprint: 309B total parameters against Pro-RL’s 1.02T. It stays close on agentic tasks (Toolathlon-Verified: 73.6 vs 76.9). On a billed-by-the-hour AWS deployment, that gap is the difference between a $30/hr instance and a multi-node cluster.
Can You Run MiMo V2.6 on AWS Bedrock?
No, not as of this writing. Amazon Bedrock Custom Model Import only accepts a fixed list of architectures: Mistral, Llama, Qwen, Flan, GPTBigCode, Mixtral, and GPT-OSS. MiMo V2.6’s hybrid sliding-window/global-attention backbone and its DFlash speculative decoder aren’t on that list. The import job fails at the architecture-validation step, before it even touches the weights.
Context length is the second blocker. Bedrock Custom Model Import caps context length at 128K tokens for any imported model. MiMo-V2.6-Flash-RL’s whole reason for existing is its 1M-token window, built for long repositories and multi-session agent runs. Importing it into Bedrock would silently throw away 87% of that context, even if the architecture check passed.
Weight size isn’t the clean disqualifier you might expect. Bedrock caps imported weights at 200GB for text-only models and 100GB for multimodal ones. MiMo-V2.6-Flash-RL’s 173GB checkpoint would clear the text-model cap but blow past the multimodal one, and MiMo is natively omnimodal. Either way, architecture support and context length are the real blockers, and Xiaomi’s model fails both. Self-hosting on EC2 is currently the only way to get MiMo V2.6 on AWS.
How Much GPU Memory Does MiMo V2.6 Need?
MiMo-V2.6-Flash-RL needs roughly 208GB of VRAM just to load its weights at FP8, before any KV cache is allocated. The checkpoint stores weights as mxfp4 and computes in FP8 (block-wise e4m3, 128×128 blocks). That keeps the on-disk size to 173GB for a model with 309 billion total parameters.
Why FP16 Won’t Fit on a Standard 8-GPU Node
Loading the same model at FP16 needs roughly 618GB of VRAM for weights alone. An 8x H100 node has 640GB raw, or roughly 576GB usable after a 90% utilization cap. That’s not enough. The load fails with an out-of-memory error before a single request is served. This is the most common mistake teams make when they try to reuse an existing H100 fleet: MiMo V2.6 simply requires FP8 or lower precision to fit.
At FP8 on an 8x H100 node, usable KV cache headroom comes out to roughly 267GB total, about 33GB per GPU. That’s enough for meaningful concurrent context, but not enough to serve many simultaneous 1M-token requests. Xiaomi’s own vLLM recipe recommends 4x H200 GPUs, each with 141GB of HBM3e, as the practical minimum for one Flash-RL replica with reasonable KV headroom. AWS announced H200-based P5e instances specifically for this class of memory-hungry inference workload.
How Do You Deploy MiMo-V2.6-Flash-RL on AWS EC2?
The fastest working path is a single p5e.48xlarge instance. It runs the prebuilt vLLM Docker image with tensor parallelism across four of its eight H200 GPUs. Here’s the sequence that got a working endpoint up in under 40 minutes on launch day.
- Launch a
p5e.48xlargeinstance (8x H200, 141GB each) in a region with P5e capacity, using an EC2 Capacity Block if on-demand capacity is scarce. - Attach at least 250GB of gp3 EBS storage for the 173GB checkpoint plus vLLM’s compilation cache.
- Pull the model-specific image:
docker pull vllm/vllm-openai:mimo-v26. Stable vLLM 0.29.0 and below cannot load the mxfp4-stored weights, so a generic vLLM install fails silently on model load. - Download the checkpoint from Hugging Face:
hf download XiaomiMiMo/MiMo-V2.6-Flash-RL. This pulls the full 173GB snapshot to local disk. - Launch the server with tensor parallelism set to 4. That leaves four GPUs free for a second replica, or for the DFlash drafter’s speculative decoding path.
vllm serve XiaomiMiMo/MiMo-V2.6-Flash-RL \
--tensor-parallel-size 4 \
--trust-remote-code \
--gpu-memory-utilization 0.95 \
--max-model-len auto \
--reasoning-parser mimo \
--tool-call-parser mimo \
--enable-auto-tool-choice \
--generation-config vllmOnce the server is up, test it against http://localhost:8000/v1/chat/completions using a standard OpenAI-compatible chat completion call. Xiaomi’s official vLLM recipe recommends temperature 1.0 and top_p 0.95. Those defaults matched the RL training configuration, and produced noticeably more consistent tool-call formatting than lower temperatures in my own testing.
What Does Self-Hosting MiMo V2.6 on AWS Actually Cost?
A single p5e.48xlarge instance runs $39.80/hr on-demand for all 8 H200 GPUs, whether you use four of them or all eight. Left running continuously, that’s roughly $28,656/month. That’s the real number teams need to budget against, not the per-GPU headline rate.
AWS doesn’t sell a standalone 4-GPU H200 instance, so you pay for the full 8-GPU node either way. The efficient move is running two Flash-RL replicas at TP4 each on the same p5e.48xlarge. That effectively halves the per-replica cost to roughly $19.90/hr, or lets you serve two independent workloads instead of over-provisioning one.
Compare that to the cheaper H100-based p5.48xlarge, at $55/hr for the full node. Despite the higher headline price, H200’s larger 141GB-per-GPU memory pool gives FP8 MiMo V2.6 far more KV cache headroom per dollar than an equivalent H100 setup. That directly affects how many concurrent 1M-token sessions you can actually serve.
What’s the Cheaper AWS Option for MiMo V2.6?
The g6e.48xlarge instance is the practical budget route, at roughly $30.13/hr on-demand. It packs 8x NVIDIA L40S GPUs at 48GB each, for 384GB total. That’s 24% cheaper than the H200 node. The 384GB pool comfortably holds the 173GB FP8 checkpoint, with around 211GB left over across eight GPUs for KV cache.
The trade-off is interconnect bandwidth. L40S GPUs on g6e communicate over PCIe, not the NVLink fabric that ties H200s together on p5e. Tensor-parallel throughput per token will be lower, especially at longer context lengths where cross-GPU attention traffic increases. For workloads that don’t need MiMo’s full 1M context, this is still a reasonable trade.
Community quantizations push the cost down further. NVFP4 and GGUF builds of MiMo-V2.6-Flash-RL landed on Hugging Face within a day of release. Those could shrink the footprint enough to fit on a single g6e.12xlarge (4x L40S), for teams willing to accept a further accuracy hit. It’s the same tradeoff curve covered in our server vs. serverless architecture comparison: more control and lower marginal cost, against more operational ownership.
How Does MiMo V2.6 Compare to Claude and GPT-6 on AWS-Relevant Benchmarks?
MiMo-V2.6-Pro-RL edges out Claude Opus 5 and GPT-5.6 Sol on agentic and terminal-automation benchmarks, but loses ground on coding and security tasks. On Terminal Bench 2.1, MiMo Pro scores 89.9. Claude Opus 5 scores 89.1 and GPT-5.6 Sol scores 88.8. On ProgramBench, a pure coding benchmark, MiMo Pro scores just 26.5 against Claude Opus 5’s 37.0 — a gap large enough to matter for teams building coding agents specifically.
Security-focused benchmarks show the widest gap. On ExploitBench, MiMo Pro scores 47.9. Claude Opus 5 scores 70.0 and GPT-5.6 Sol scores 78.5. If your AWS workload involves security research or red-team automation, the closed models still lead by a wide margin. That gap alone may justify Bedrock’s managed Claude access over self-hosted MiMo V2.6.
Comparing Your AWS Options at a Glance
| Option | Cost | Complexity | When to Use |
|---|---|---|---|
| Bedrock (Claude/Llama/Qwen) | Per-token, no fixed cost | Low — managed endpoint | Production workloads on supported architectures today |
| EC2 p5e.48xlarge (8x H200) | ~$39.80/hr ($28,656/mo) | High — manual scaling, TP config | Max context, multiple concurrent replicas |
| EC2 g6e.48xlarge (8x L40S) | ~$30.13/hr ($21,696/mo) | High — lower interconnect bandwidth | Budget-conscious teams, shorter context needs |
| Quantized (NVFP4/GGUF) on g6e.12xlarge | ~$7-8/hr estimated | Very high — accuracy tuning required | Dev/test, latency-tolerant internal tools |
The NexGismo verdict: Run MiMo-V2.6-Flash-RL on a p5e.48xlarge if you need the full 1M context. Share the node across two TP4 replicas to justify the cost. If your workloads rarely need more than 128K tokens, skip the self-hosting complexity. Wait for Bedrock or Qwen-class Custom Model Import support to catch up instead. For most PHP and Node teams already on AWS, that wait will be shorter than building reliable GPU-fleet operations from scratch.
- MiMo-V2.6-Flash-RL is a 309-billion-parameter MoE model with only 15 billion parameters active per token, released under an MIT license on September 22, 2026.
- Amazon Bedrock Custom Model Import does not support MiMo V2.6’s architecture and caps imported context at 128K tokens, versus MiMo’s native 1M-token window.
- The FP8 checkpoint for Flash-RL is 173GB on disk. Loading the same model at FP16 requires roughly 618GB of VRAM and will OOM on a standard 8x H100 node.
- AWS’s p5e.48xlarge (8x H200) costs $39.80/hr on-demand, while g6e.48xlarge (8x L40S) costs roughly $30.13/hr — a 24% saving with a PCIe interconnect trade-off.
- MiMo-V2.6-Pro-RL scores 89.9 on Terminal Bench 2.1, ahead of Claude Opus 5’s 89.1, but trails significantly on security benchmarks like ExploitBench (47.9 vs 70.0).
- Stable vLLM releases at or below version 0.29.0 cannot load MiMo V2.6’s mxfp4-stored weights. A nightly build or the prebuilt mimo-v26 Docker image is required.
Frequently Asked Questions
Is MiMo V2.6 free to use commercially?
Yes. Xiaomi released MiMo V2.6 under an MIT license. That permits commercial use, modification, and redistribution of the weights, without royalty payments or usage restrictions from Xiaomi.
What GPU do I need to run MiMo V2.6?
MiMo-V2.6-Flash-RL needs roughly 208GB of VRAM at FP8, practically a minimum of 4x H200 GPUs. The larger Pro-RL variant, at 1.02 trillion total parameters, needs substantially more and typically runs across 8 or more H200s.
Can I run MiMo V2.6 on Bedrock?
Not currently. Bedrock Custom Model Import only supports Mistral, Llama, Qwen, Flan, GPTBigCode, Mixtral, and GPT-OSS architectures, and caps context length at 128K tokens. MiMo V2.6’s architecture and 1M-token context window fail both requirements.
How much does it cost to run MiMo V2.6 on AWS?
A p5e.48xlarge instance (8x H200) costs about $39.80/hr on-demand, or roughly $28,656/month running continuously. A cheaper g6e.48xlarge (8x L40S) option runs about $30.13/hr, with reduced interconnect bandwidth.
Is MiMo V2.6 better than Claude or GPT-6?
It depends on the task. MiMo-V2.6-Pro-RL beats Claude Opus 5 on Terminal Bench 2.1 and AutomationBench, but trails noticeably on coding benchmarks like ProgramBench and security benchmarks like ExploitBench.
MiMo V2.6 on AWS is possible today, but it’s an EC2 story, not a Bedrock one. Xiaomi shipped a genuinely competitive open-weight model. The MIT license removes the usual legal friction around self-hosting. What it doesn’t remove is the GPU bill. A real deployment needs a 4x H200 minimum, a specific vLLM build, and a clear view of the 128K-token wall that keeps it off Bedrock for now. If your context needs fit under that ceiling, wait for managed support. If you genuinely need the 1M-token window, the p5e.48xlarge path above gets you a working endpoint in under an hour. Check that cost against your real token volume first.