AI Smells the Difference—But Can It Tell Chanel from Cheetos?

A new benchmark, dubbed Olfactory Perception (OP), claims to test how well large language models can reason about smell. The paper, posted on arXiv arXiv:2604.00002v1, presents 1,010 questions across eight categories—from odor classification to olfactory receptor activation—using two prompt formats: compound names and isomeric SMILES. The results? Compound names consistently outperformed SMILES, but even the best models scored well below human-level accuracy.

This isn’t just academic curiosity. The benchmark’s creators tested 21 model configurations, including major players like GPT and Llama, and found them struggling with basic scent identification—let alone complex judgments like intensity or pleasantness. The gap between the demo and real-world utility is glaring. While the paper frames this as a step forward, the actual performance suggests LLMs are still sniffing around the edges of olfactory understanding, not mastering it.

The irony? The benchmark itself is a clever piece of work, but the marketing hype around AI’s sensory breakthroughs hasn’t matched the results. If LLMs can’t reliably tell lavender from limburger, their utility in fragrance development, food science, or even AI-driven perfumery remains speculative at best.

So why does this matter beyond the lab? The OP benchmark highlights a broader trend: AI’s struggles with multimodal reasoning beyond text and images. Smell, like taste and touch, remains a sensory frontier where models lag behind even average human abilities. The paper’s authors acknowledge this but stop short of addressing the real-world implications—for instance, how many industries are banking on AI to parse sensory data that it can’t yet handle.

The competitive angle is equally revealing. The fact that compound names outperformed SMILES suggests that simpler, human-readable inputs still have an edge over raw molecular data. This could shift how AI is applied in chemistry, fragrance, and food tech, where accuracy isn’t just about benchmarks but about real product decisions. Companies betting on AI-driven scent analysis—or worse, hyperlocalized marketing based on olfactory data—may need to recalibrate their expectations.

The developer community’s reaction has been muted, with most discussions focused on the benchmark’s methodology rather than its practical applications. GitHub and technical forums show little buzz, underscoring that this is still a niche research project, not a breakthrough ready for deployment. For now, the OP benchmark is a reminder that AI’s sensory capabilities are still more about potential than performance.