The one-sentence version
When current reasoning models "think," they must write every step as text (Chain of Thought). However, text is a very narrow channel: each step can only select one word from tens of thousands, forcing the model's rich, continuous representations to be "compressed" into discrete symbols. Latent space reasoning allows the model to take several more steps within its internal continuous hidden states before outputting, much like how humans ponder things in their minds without having to speak every thought aloud. It may be more efficient and capable of preserving multiple possibilities simultaneously, but the cost is that it is harder for humans to inspect, posing a real challenge to safety and interpretabilityability.
Positioning
Preliminary research exists, but it is far from mature. Distinguish between existing evidence and speculation.
Layer 1: The Intuition
1.1 Thinking Out Loud vs. Pondering Internally
When solving a difficult problem, there are two ways:
- Thinking out loud: Every thought is spoken aloud; once spoken, it is fixed, and to change it, one must speak again;
- Pondering internally: Considering several lines of reasoning simultaneously in the mind, turning them over, and only stating the conclusion at the end.
Chain of Thought is the first method. Latent space reasoning is the second.
1.2 Why Text is a "Narrow Channel"
The hidden state at each step inside a model is a continuous vector with thousands of dimensions, containing a vast amount of information. Converting it into a single word is equivalent to picking just one most likely discrete choice, while the rest of the information (e.g., "I am still considering two other possibilities") is lost.
Layer 2: Existing Evidence
| Work | Content |
|---|---|
| Coconut (Meta, 2024, "Continuous Chain of Thought") | Uses the hidden state of the model's last layer directly as the input for the next step without "word selection." On logical reasoning tasks requiring search and backtracking, it achieves better results with fewer steps; analysis shows that multiple candidate paths are encoded simultaneously in the hidden states, similar to Breadth-First Search |
| Recurrent Depth / Implicit Reasoning (Multiple works in 2025) | Allows the same set of layers to run repeatedly, "thinking deeper" during inference without needing to output more tokens |
| Large Concept Model (Meta, 2024) | Performs prediction in the representation space of "sentence-level concepts" rather than token by token |
| Chain of Thought Unfaithfulness | Multiple studies (including work from Anthropic and OpenAI) have found that the Chain of Thought written by a model is not necessarily reflective of its true reasoning process; it may have already decided on an answer internally and then "backfills" the reasoning |
| Interpretability | Mechanistic interpretability research shows that multi-step "implicit reasoning" exists within models; for example, when answering a two-hop question, the intermediate conclusion appears in the internal representations without being written out |
The last two points indicate: models are already performing some reasoning in latent space; we simply have not explicitly trained or utilized it.
Layer 3: Conception and Trade-offs
3.1 Potential Benefits (Reasonable Speculation)
| Benefit | Reason |
|---|---|
| More efficient | No need to generate and process tokens for every thinking step |
| Preserving multiple possibilities | Continuous vectors can represent "60% Path A, 40% Path B," whereas text can only write one path at a time |
| Not limited by language | Some reasoning (spatial, numerical, intuitive) is inherently non-linguistic |
| Closer to humans | A large amount of human thinking is non-linguistic (see Predictive Processing and the Next Token) |
3.2 The Real Cost: Supervability
Chain of Thought is currently a vital tool for AI safety: we can read what the model is thinking to discover if it is planning to cheat, deceive, or perform dangerous actions. Researchers from multiple labs have jointly called for "protecting the monitorability of Chain of Thought."
Latent space reasoning would cause this channel to disappear:
| Textual Chain of Thought | Latent Space Reasoning | |
|---|---|---|
| Can humans read it? | Yes (though not necessarily faithfully) | Cannot be read directly |
| Monitoring methods | Reading text, using models to check text | Requires interpretability tools to decode hidden states |
| Risk | Unfaithful Chain of Thought | Reasoning is completely invisible |
3.3 A Compromise Concept
- Hybrid Reasoning: Routine steps are completed in latent space, but key decision points must be written in text (e.g., "I decide to call this tool because...") for monitoring;
- Decodable Latent Space: Adding a "decoder" constraint during training, requiring hidden states to be translatable back into approximate textual descriptions to ensure auditability;
- Real-time monitoring with interpretability tools: Using methods like Sparse Autoencoders to map latent thoughts onto human-readable features.
3.4 Hypotheses to be Verified
- Will latent space reasoning still outperform textual Chain of Thought in open-domain complex reasoning? Current advantages are mainly in structured logic and search tasks.
- Can "key decision points" in hybrid reasoning be reliably identified? Will models hide critical reasoning within the latent space?
- Will decodable constraints "narrow" the latent space again, offsetting the benefits?
Summary
| Layer | Key points |
|---|---|
| Intuition | Thinking out loud vs. Pondering internally |
| Evidence | Coconut's continuous Chain of Thought, recurrent depth, unfaithful Chain of Thought, internal implicit reasoning |
| Benefits | Efficiency, preserving multiple possibilities, not limited by language |
| Costs | Reasoning is invisible, weakening safety monitoring; requires hybrid reasoning or decodable constraints |