I’ve stopped thinking of my work with Claude as prompting. I think of it as building a world for the agent to work inside. The agent revolution, of course, has changed how we think of LLM applications such as Claude and ChatGPT. Increasingly, it doesn’t make sense to think of them as “chatbots,” and increasingly the framing of “agents” makes more sense. This doesn’t mean, however, that they are independent of human agency. I find that the more I know about a particular domain, the easier it is for me to get stuff done together with agents. The more I know, the easier it is for me to build a context (or a world) for the agent to operate within. This is a critical starting point for the human-embedded agentic design (HEAD) framework that I am developing.
The context is the full set of text, code, images, video, audio, and other documents that is provided to an agent. In principle, it is all part of the prompt that is sent to the LLM, but because such contexts can be quite large, these documents are often stored separately and not entered through the chat window per se but rather added to the prompt by the harness itself. Because contexts can be so complex, I think of them as a “context stack.” The stack contains everything the agent needs to orient itself, arranged in a hierarchy that runs from broad and persistent at the top to specific and ephemeral at the bottom.
The Structure of a Context Stack
At the top are your goals: the system prompt and the skills that describe how you work across a whole domain. A researcher might keep a skill spelling out how they approach theory, what counts as a contribution, how they decide whether an argument holds. Below the goals are your standards: templates, and the patterns and anti-patterns, in code and in prose, that encode what good output should look like. Below those are the resources: the literature, data, and code a particular task happens to need. At the very bottom sits the chat prompt, the thing most people just call “the prompt”: the in-the-moment request that activates the whole stack.
The utility of thinking of this as a stack rather than a single prompt is that the various aspects of the context stack can be switched out and recombined depending on what you are working on. While T\the system prompt loads every time, different style guides can be loaded depending on what you are writing and for which audience. Similarly, different datasets need to be loaded for different projects or analyses.
Being specific about how you set up your context can be quite tricky, and in my experience, it has required a fair amount of experimentation. I often find that I underspecify the context, and thus gets output that is generic, or that I overspecify the context in a way that becomes overly narrow. Once I had Claude reframe a whole paper using dance metaphors, and the agent went completely overboard, which was kind of funny, but also not useful.
Research shows that LLMs tend to forget about things which are placed in the middle of a context. This means that the way in which you structure your context is important both in terms of what you put in there, as well as where you put it. Using Claude’s API you can control how content is placed in different slots of the context, and while increasingly there are various computational techniques for dealing with the “forgotten middle,” being strategic about how you place your content throughout your context stack can be helpful.
Source: Liu, Nelson F., Kevin Lin, John Hewitt, et al. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics 12 (2024): 157–73.
Now, it is not only building the agent’s context that matter—you also need to build a parallel context in your own mind that helps you both specify the agent context, as well as understand inputs to the context provided by an agent working with RAG to identify new forms of content. As an example, for this post I conducted several micro-literature reviews that helped me identify interesting papers to read. To make sense of what the agent was suggesting I do with these papers, I had to read them myself (or at least select parts of them, such as abstract, introduction, and discussion) to see how they would fit within the larger argument of what I was writing.
The inescapable conclusion, therefore, is that you still must read (I love reading, by the way, so I’m game). You cannot substitute your own cognitive processing for the agent’s computational processing. The context you assemble for your agents is only as good as the context you are holding in your own head.
If you are not clear about what your “cognitive context” is, then you risk anchoring yourself in something suggested by the LLM, which may or may not be a good idea. Indeed, research by Chen et al. (2024) suggests that for experts, using LLMs as “ghost writers” may lead to decreases in creative performance, while for non-experts using LLMs as “sounding boards” may help to increase creative performance. If you have a firm grasp of the knowledge domain you are operating in, and if you are clear about what you want to achieve, then it tends to be easier to achieve it through leveraging agents.
The Parallelism of Cognitive and Computational Context
So the context stack and how you build it for a particular project is a crucial leverage point for governing what the agent is going to produce. It won’t help you get out of the work of actually learning the knowledge domain surrounding the project yourself, but once you do, you can get a lot of leverage out of a clearly specified context helping agents to deliver what you expect out of them. Hence, it is less about what “prompts” you write for your agents, and more about the world of goals, standards, and resources that you construct around your agents.