Rethinking Software Architecture: Context as the Key to Agentic AI

In a thought-provoking podcast, Michael Stiefel and Baruch Sadogursky delved into the evolving role of context in agentic AI architectures. They argued that large language models (LLMs) can act as reasoning engines—despite their inherent stochasticity—when guided by rigorous context artifacts. This insight flips traditional software development on its head: specifications become the permanent source of truth, while code is reduced to a disposable intermediate language. Below, we explore the core ideas from their conversation.

1. How does context drive the agentic architecture revolution?

Context is the linchpin that enables LLMs to function reliably in agentic systems. Without it, the model's stochastic nature leads to unpredictable outputs. By providing rich, structured context—like detailed specifications, past interactions, or domain rules—developers anchor the LLM's reasoning. Sadogursky emphasizes that context artifacts act as guardrails, transforming a probabilistic text generator into a deterministic-like reasoning partner. This shift allows AI agents to interpret human ambiguity, execute multi-step tasks, and adapt to changing conditions. In essence, context replaces guesswork with guided inference, making agentic architectures both powerful and controllable.

Source: www.infoq.com

2. Why can LLMs be considered reasoning machines despite their stochastic behavior?

LLMs are fundamentally stochastic: they generate responses by predicting the next token probabilistically. Yet, when supplied with sufficient context, their outputs exhibit structured reasoning. Sadogursky explains that the key is the interplay between the model's training and the artifact's constraints. The LLM's vast internal knowledge, combined with explicit context, allows it to simulate logical deduction, plan steps, and synthesize information. It's not that the model "thinks" in a human sense—it's that context makes its stochastic sampling mimic reasoning. This works because the context narrows the probability distribution toward relevant, coherent answers.

3. What are context artifacts, and how do they control LLM reasoning?

Context artifacts are carefully curated pieces of information that guide an LLM's output. They can include software specifications, design documents, API definitions, or even error logs. These artifacts serve as an external memory and reasoning scaffold. By feeding them as part of the prompt or through retrieval-augmented generation, developers steer the model toward specific reasoning paths. For example, a specification artifact might define the exact behavior of a function, preventing the LLM from inventing arbitrary implementations. Sadogursky stresses that the rigor of these artifacts is crucial: vague context leads to vague reasoning, while precise artifacts yield dependable, production-ready outputs.

4. How do software specifications become the source of truth?

In traditional development, the code is the ultimate source of truth—specifications often drift as bugs are fixed and features evolve. Sadogursky proposes flipping this: treat the specification as the definitive, immutable document. The LLM uses that spec to generate code, test cases, and documentation. If a change is needed, you edit the specification, not the code. The code is regenerated accordingly. This ensures alignment between intent and implementation. It also makes auditing easier: you review the spec to understand what the system should do, rather than reverse-engineering from code. This paradigm shifts the developer's role from coding to curation of specifications.

Rethinking Software Architecture: Context as the Key to Agentic AI — Source: www.infoq.com

5. Why is code described as a disposable intermediate language?

If specifications are the source of truth, then code is merely a transient artifact—a translation of the spec into a machine-executable form. Sadogursky argues that code should be treated as disposable because it is generated automatically from the spec. As specifications evolve, old code is discarded and regenerated. This contrasts with today's approach where code is carefully preserved and refactored. The advantage is reduced technical debt: there's no legacy code to patch. However, it demands that specifications be precise, complete, and versionable. This model works especially well with agentic AI, where the LLM can instantly produce new code that reflects the latest spec, eliminating the need for manual synchronization.

6. What are the practical implications for software teams?

Adopting this specification-first, code-disposable model requires a cultural and tooling shift. Developers must become adept at writing clear, unambiguous specifications—almost like formal contracts. Version control shifts from code repositories to spec repositories. Testing also changes: you now validate that the code produced matches the spec, rather than testing the code itself for logic errors. Sadogursky predicts that this approach will accelerate development cycles and reduce bugs, but it also demands a higher level of domain knowledge from everyone involved. Teams will need to invest in context management and prompt engineering skills. Ultimately, the role of "coder" evolves into "specification engineer," focusing on intent rather than implementation details.