AI can know the answer forward and still fail the same fact in reverse
Symbolic neural network paths show how a fact can be easy in one direction and fragile in reverse.š· AI-generated / Tech&Space
- ā Autoregressive models still struggle when a known fact is queried in reverse.
- ā Bidirectional training improves reversal behavior without proving full conceptual understanding.
- ā The test matters for AI tools that must verify identities, authorship, and linked facts rather than autocomplete text.
The reversal curse sounds like a narrow benchmark problem, but it hits the center of what people call knowledge in language models. If a model learns āA wrote B,ā it should also answer āwho wrote B?ā According to a new arXiv paper, standard autoregressive models still often slip on that reversal.
This is not just a trivia failure. Autoregressive training optimizes the next token in one direction, so a model can complete a sentence fluently without storing the relation as a genuinely two-way link. Syntax moves forward. Fact checking often needs a map that can be turned around.
Autoregressive models can recite a fact forward, then stumble when the same fact is asked backward.
Bidirectional training changes the test from a memory trick into a stronger check of relational learning.š· AI-generated / Tech&Space
The important signal in the paper is that bidirectionality helps. Models trained to use context from both sides have a stronger basis for retrieving reversed relations. That does not prove full understanding. It shows that architecture and training objective change whether a fact becomes a relation or just a phrase the model can continue.
For AI products, the lesson is operational. Systems that search documents, verify authorship, or connect entities should not accept a single fluent answer as evidence of knowledge. They need tests in both directions. If a fact collapses when the question is reversed, it is not stable knowledge. It is a memorized path with no return lane.
