Moe

AI models may get lighter by carrying only the modules they actually need

EMO tries to turn MoE modularity from a theoretical compute advantage into a practical tool for smaller, domain-focused deployments.

DeepSeek is trying to make long AI work cheaper

DeepSeek V4 arrives in Flash and Pro versions with a 1M-token context window, a MoE architecture, and a claim that it is closing in on leading closed models.

Transformers are the new coal plants of AI

Meta’s latest 175B-parameter LLaMA 3 model required a training run that consumed 1.2GWh—enough to power a Tesla Gigafactory for a day.

LiME shows how expert AI models can learn without copying adapters

LiME uses one shared PEFT module and lightweight expert vectors to cut MoE-PEFT parameters by up to four times.

Arcee’s Trinity: Open Reasoning or Just Open Marketing?

Gemma 4’s real trick: Squeezing more IQ per byte

Trillion-parameter models now fit in laptops. So what?

MoE's 1-trillion-parameter model now runs on a 96GB MacBook Pro.

Mistral wants one AI model to do three jobs, but the hardware bill still matters

Mistral quietly shipped Small 4, a 119B-parameter MoE model that collapses Magistral, Pixtral, and Devstral into one 6B-active-weight binary — and for the first time, the unified architecture actually works in production.

MoE-SpAc’s speculative bet: Lookahead or just more hype?

The MoE-SpAc team repurposed Speculative Decoding—a technique normally used to speed up LLMs—as a memory oracle for edge devices, betting it can predict expert activation before the model stumbles.

Open AI agents get a stronger engine, but deployment is the real test

Nemotron 3 Super combines 120B parameters, Mamba, and MoE for a new open-agent push.

YuanLab’s trillion-parameter AI is really a story about the bill

YuanLab’s model emphasizes MoE pruning and expert rearrangement, making it a compute-economics story rather than only a size story.