LLMs Transform Knowledge Capture: AI Interviewers Replace Manual Documentation

Breaking: New Technique Uses AI to Interview Humans for Complex Tasks

A groundbreaking approach called 'interrogatory LLM' is reshaping how organizations feed context to large language models. Instead of humans writing lengthy documents, the LLM itself interviews the human to gather all necessary information. This method, gaining traction in software development and knowledge management, promises to streamline workflows and reduce barriers for non-writers.

LLMs Transform Knowledge Capture: AI Interviewers Replace Manual Documentation — Source: martinfowler.com

“The LLM asks me all the questions it needs to create appropriate context,” explains Martin Fowler, Chief Scientist at ThoughtWorks. “I feed much of the information it needs, and tell it other sources to consult. Once done, it creates a context report for another session.” This technique was first described by Harper Reed, who insists the LLM should ask only one question at a time—a detail that often requires frequent reminders.

Background: The Context Problem

Performing complex tasks with an LLM often requires extensive context—descriptions of user interfaces, implementation guidelines, external system details. Typically, a human writes this context as several pages of markdown. But many people find writing difficult, leading to rushed or missing documentation.

The interrogatory LLM offers an alternative: let the AI conduct a structured interview to extract expertise. This is especially useful when the human is a domain expert but not a skilled writer. The resulting output may carry an “AI-writing tang,” Fowler notes, “but that’s better than not having the information itself.”

Two Primary Use Cases

1. Gathering Context for New Tasks

When designing a feature, an LLM interviews the human about desired appearance, implementation guidelines, and external systems. It then compiles a context report for another session—perhaps with a different model—to execute the next step. This method can be applied iteratively, with one LLM building a document and others reviewing it with different experts.

2. Reviewing Existing Documents

Alternatively, the LLM can take an existing document—like a software specification—and interview a human expert to verify its accuracy. “People often find reviewing hard,” Fowler says. “A conversation with an LLM might be more fruitful, particularly if the document isn’t well-written.” This shifts the burden from reading dense text to answering focused questions.

Broader Applications Beyond LLM Context

The technique is not limited to preparing context for AI tasks. For individuals who struggle with writing—even when they possess deep knowledge—an interrogatory LLM can serve as a knowledge extraction tool. “I’ve become a natural writer; writing is essential to thinking,” Fowler reflects. “But different people are different. Many find writing hard, often very hard. Maybe such people would find it easier to ask an LLM to interview them.” This could capture expertise that otherwise remains locked in someone’s head.

What This Means

The interrogatory LLM represents a shift in human-AI collaboration. Instead of humans preparing data for AI, the AI actively elicits data from humans. This reduces the need for written documentation, potentially accelerating knowledge capture in fast-moving teams. However, it requires careful prompting and adherence to single-question discipline to avoid confusion.

As organizations increasingly rely on LLMs for complex, context-dependent tasks, this interview-based approach may become standard. It bridges the gap between expert knowledge and AI execution, making expertise accessible even when the expert is not a proficient writer. The next step is refining the questioning process and integrating these interviews into broader AI workflows.