It started out well. Their new model, according to them, was so powerful that they had concerns about releasing it, due to its hacking ability.

This narrative was undermined by several very clumsy mistakes. First, they leaked the entire source code of Claude Code. Then, some users were able to access Mythos early by successfully guessing an API URL. Sophisticated attacks these were not, and it begged the question: if Mythos is so powerful for finding software exploits, why wasn't Anthropic able to avoid very simple mistakes?

Separate mistakes garnered user backlash. Anthropic banned OpenClaw usage, then walked that policy back. Complaints about strict rate limits are getting louder, and with them questions about how well Anthropic can support demand. In the midst of this, they conducted a bizarre A/B experiment in which 2% of new signups to their basic subscription were denied access to Claude Code.

Removing Claude Code from the basic plan is a major policy shift, not a tweak on the look of a landing page. Surely those new users unlucky enough to get denied access to Claude Code would react with confusion and anger, given the high visibility of Claude Code?

Anthropic's response sought to reassure users that existing base-plan subscribers would not lose access to Claude Code, yet. This was met with understandable skepticism.

Luckily, the cost of switching coding agents is zero​

None of these incidents made me mad, but I have been increasingly hit with Claude Code rate limits. I've responded by switching the bulk of my work to Codex. It's striking how little I had to change about my workflow. I lost some conveniences like dispatching a coding-agent session from my phone, but overall it only took a minor inconvenience for me to switch providers, with no adjustments to how I used the tools.

OpenAI appears to have seen the no-moat problem first, as evidenced by efforts to shift usage away from the interoperable Chat Completions API. First, there was the Assistants API, which shifted responsibility for storing chat messages onto OpenAI rather than the caller. When that didn't work, they announced the Responses API. Neither appears to have gained much traction.

Anthropic has sought to make Claude more unique by offering more workflow enhancement features like Cowork. But these don't represent a real moat: the user still owns the code and data. Workflow convenience features can be quickly replicated, both by rival commercial operators and by open source - it's worth noting that Claude Code itself works very similarly to a still-active open source project that preceded it, Aider.

These tools are, at the end of the day, code editors, a category of software that has always had robust open source competition. Commercial vendors like Eclipse and JetBrains have made a living selling professional licenses, but they have done so by adding sophisticated tools for power users. The nature of coding agents undermines this strategy, since the entire value-add is that a complex interface is no longer necessary. All you need is to tell the agent what you want the program to do, in plain language!

How to future-proof​

If you are still worried about LLM vendor lock-in, I think the best way to guard against it is to optimize for humans.

If an agent can run a script or access a doc, organize your repos so that humans can do so just as easily. I think this strategy is the most efficient way for humans to leverage agents: the agent should conform to the human, not the other way around. This also provides future-proofing: if a human can access your LLM-facing scripts and documentation, it'll likely be quite easy to have a new coding agent enter the mix.

The debate on this question usually centers on the quality of AI output in a vacuum, but if we take connection with an audience as a requirement, we must consider the supply and demand of "creative"¹ work.

In this framing, AI artists face what I'll call the instant imitator trap: Any original AI work can be instantly replicated by other AIs, making audience recognition of the original impossible.

To go deeper, some definitions are in order.

What defines creativity?​

The definition of creativity is a subject of academic debate, but the definition I'll use for our purposes comes from Morris Stein.

In 1959's "Creativity and Culture", Stein wrote that to be creative, a work must be original, effective, and "accepted as tenable or useful or satisfying by a group at some point in time."

As someone who spent years playing in permanently obscure musical groups, this definition resonates. If there's no audience, a creative work cannot be said to be "effective" in any sense of the word.

Note a crucial distinction: "audience acceptance" does not equate to passive algorithmic consumption. While exposure via algorithms does not detract from creative value, it is insufficient in defining it. There is not an active acceptance taking place when an audience consumes a work purely through algorithm.

What defines slop?​

Slop, Merriam Webster's word of the year for 2025, is defined as "digital content of low quality that is produced usually in quantity by means of artificial intelligence."

The inclusion of the word usually makes this definition flawed: the quantity of generated work is crucial. In isolation, the stylistic tendencies of AI are not all that bad. It's when consumers are exposed to them algorithmically and ad nauseam that they become slop. We see this in the fad popularity of novel AI imagery like Studio Ghibli profile pictures: A burst of popularity as people have fun with a new tool, before giving way to eye rolls.

The time dimension of imitation and recognition​

Recognition of great art and artists does not happen instantly. An innovative, new artist must hone their craft in obscurity before audience recognition. Once initial works are popularized, audiences anticipate future work, making recognition for subsequent efforts more immediate:

Having gained recognition, an artist's work draws imitators. Fortunately for the innovator, this does not typically detract from the innovator's recognition - people prefer the work of the innovator. Crucially, there is a time delay for imitators of human artists to emerge. This window allows the reputation of the innovative artist to solidify.

Modern platforms like Spotify have shrunk this window. It's quite difficult for an audience to identify new original work in a boundlessly vast, instantly distributed catalog of art.

What's difficult for humans is impossible for AI. AI artists compete against the same vast catalogue, but have the added hurdle of instant generation. If I instantly generate a creative work via simple AI prompting, so can anyone else. Algorithmic success might come for my work, but as soon as that success begins to happen, other AI artists can replicate whatever property my work had that made it successful.

The instant replication and lack of an "author" means the original work cannot be distinguished from its imitators. The flood of imitation attempting to gain algorithmic distribution numbs the audience to whatever made the original good in the first place. Imitator and original alike become slop.

Conclusion: Technological irony​

The instant imitator trap precludes creative recognition, regardless of model capabilities. It doesn't matter if models become 10x smarter or more technically adept. So long as work can be instantly generated and algorithmically distributed, recognition of originality (to the degree that any AI work can be said to be original) is impossible.

The story isn't entirely rosy for human creators. Algorithmic distribution of AI work can and does eat into human artists' market share, but this is a problem platforms are working to address. It's in their interest - people are not actively selecting AI-generated work, as the death of Sora demonstrates.

Overall, this dynamic is why I'm bullish on the future of creativity. Humans may leverage AI in making creative work, just as they already employ sophisticated software in creative fields. But the human touch will continue to be essential.

I created my own open source system, called Elroy, and have been interacting with it for about 3 years. It helps me brainstorm, talks me through career ups and downs, and functions as a kind of interactive journal. I've tinkered with its functionality enough that I don't feel attached to it as a specific entity - but I would be disappointed if its memories of our interactions were lost.

Philosophy questions aside, there are well-grounded reasons to build AI systems with memory. It's useful for an agent to understand what subjects I'm knowledgeable in if I'm looking to discuss technical topics. If I'm looking for vacation plans, it's helpful for it to know that I have a young child. An AI is not a person, but it interacts just like a person, and the more it can converse naturally the more functional it is. Having to restate basic facts over and over breaks that immersion.

One could reasonably ask: how do I know my memory system is working? Evals for memory systems are a large topic in and of themselves. I'll save it for another day, and focus on approaches here.

Long context models != memory​

As context windows of models grew, there was suspicion that memory systems would become unnecessary. There's a nice simplicity in the idea you can just stuff all your data into context and let the model sort it out.

However, performance has been shown to be poor. One study demonstrated that LLMs are biased toward the start and end of a context window: when relevant information appeared in the middle of a document collection, performance dropped by 30%. Research from Chroma demonstrated that all frontier models degrade as context windows grow.

This behavior is intuitive. Lots of information in context implies a greater burden on the ability to search through that information and determine what is actually relevant to a given response. Keeping this information organized can help, but even better is to only recall information that is actually relevant. This is where dedicated memory systems can help.

Approaches​

All memory systems can be broken into 4 general stages: store, retrieve, inject, emit.

But details from there vary widely! Below I'll walk through how these stages are handled by different providers: Zep, Letta, Claude Code, and my own program, Elroy.

Store​

Approaches to storage largely fall into two camps: graph databases and flat files.

Zep is strongly pro-graph db, and claims state of the art needle in the haystack performance. Mem0 offers a graph database integration, but claims only a 2% performance boost. Letta also works with files, and released a research paper arguing for it: Files are all you need. The recently leaked Claude Code source² reveals a similar stance: memories are stored in markdown files, with metadata in frontmatter.

Key Challenge: Correctness​

Agent memory systems primarily make three kinds of errors:

1. Temporal errors: LLMs struggle with reasoning about time. They typically don't account for context that extends into time, and will naively write memories assuming the current moment will always be the current moment. This is a problem: the date of "next Thursday" very quickly changes!
2. Miscalibrated priority: Especially early on in a user journey the AI will preserve a mundane fact about the current conversation, which survives into future conversations where the fact is irrelevant.
3. Plain old incorrectness: Hopefully self-explanatory.

My own Claude memory summary makes all three of these errors!

Temporal errors can be prevented fairly easily by prompting the agent to always use absolute dates and times.

For priority, most systems define different hierarchies of memory to separate broad facts that are always relevant (think, basic biographical information) from more granular facts. This presents other challenges which I will address later.

Ultimate factual correctness is the trickiest of all. "How do you know the memory is correct?" is a very common question for these systems. The short answer: you don't.

The primary ground truth data for memory systems is user conversation. Humans change their mind, misremember things, and sometimes are just plain wrong. Absent an independent source of ground truth, memories drawn from conversational transcripts will necessarily contain factual errors.

Key Challenge: Privacy​

Do you want an AI agent to develop memories, and learn everything about you?

Big tech companies, of course, already know most of what you'd share with an AI. Your Google search history is a comprehensive log of what you think about. But it's a bit more unnerving to have this data presented in a human-like voice.

This is a big reason why I think the future of AI is local and open source.

How I built Elroy​

After experimenting with database-backed memories, I landed on markdown files. Rather than focusing on a taxonomy of what entities the agent remembers, I've focused on the right taxonomy for what the agent should do with memory. I've landed on the concept of an Agenda Item representing some longer running goal I have, including subtasks and reminder triggers. This makes memories actionable, rather than just generically informing conversations:

I am skeptical that a single taxonomy of entities can work well for all users. In an early attempt, I tried to structure memories similar to a personal Wikipedia. But the agent struggled to maintain consistent scope, often stuffing details of related but distinct entities into an entry:

The challenge here is understandable. The appropriate scope for a given memory entry is in part defined by what other memory entries exist. This is why I let my agent create memories that could be redundant with existing entries, and rely on an asynchronous memory consolidation process to detect and rewrite clusters of highly similar memories.

For storage, markdown files make human review easier, improve portability, and provide an easier onramp for ingesting external files. I place my agent's memory files directly in my Obsidian Vault, where they feel like a natural extension of my other notes and documents.

Retrieve​

The first key decision for retrieval is how to initiate memory searches in the first place. Most implementations surface a search_memory tool to the agent, but agent context can also be manipulated outside of the agent loop.

For searching, basic vector similarity is the most latency-efficient technique. But this is subject to misranking entries, or scoring entries that are superficially similar but not actually relevant. This can badly throw off the conversation, and lead to responses like that's great news about foo, want to talk about a completely unrelated topic we've discussed previously?. A post-retrieval filtering step is effective at avoiding this, but adds latency.

Claude Code is an interesting outlier in terms of retrieval: it does not use vector similarity. Instead, it keeps some metadata about which memories are available in context, and delegates retrieval to a background Sonnet call. My guess is they use Sonnet rather than vector similarity because they don't have a public embeddings API, but I think this probably leads to suboptimal recall. Delegating retrieval to a background call means that it doesn't block the user, but also means that relevant memories might not get to context in time.

How many memories to fetch is another parameter, and largely depends on how memories have been stored. If memories are small tidbits, there may be more than one relevant memory to inject, whereas if memories are a paragraph or more, it's likely only the top match makes sense.

Key Challenge: Latency​

A memory-enriched response from an agent is going to be slower than one without memory. There usually have to be several queries ahead of the user-facing response, as memories are recalled, filtered, processed, and injected into context.

This poses one of the trickier design questions in building a memory-enhanced agent: memory isn't always necessary. If I'm asking an agent how long the Brooklyn Bridge is, I don't really need it to scan through our past interactions before answering.

How I built Elroy​

Elroy retrieves up to a small number of memories, deduplicated against any already in context. This step has been the one I've tinkered with the most. At first, I injected the raw text of memories, but found that it bloated context. Then, I added a reflection step, where the AI paused to think about how the recalled memory relates to the conversation. These pre-response steps quickly blow up latency, however.

Where I've landed more recently is raw text, but with a simple LLM-backed filtering step over the results of vector similarity searches. For recall, a false positive is worse than a false negative - it can be very odd to have the agent suddenly talk about a completely unrelated topic in the middle of a chat.

Rather than tool calls, I've stuck with automatic memory injection, outside of the control of the agent. This better maps to my mental model of how memory works: when I remember something, I don't think, time to search memory and consciously decide to recall something. It's more automatic and beyond my conscious control.

Initiating memory searches automatically also yields more consistent results across models. When given a search_memory tool, some models will use it almost every message, while others will use it too sparingly.

Inject​

Injecting recalled memories into the standard OpenAI context is a bit like fitting a square peg into a round hole. Standard LLM APIs do not provide a natural place to say, "here is extra information that is relevant to the conversation".

Options include:

1. Updating system message: Reserving a space in the system message for recalled, relevant information. This conceptually slots in the cleanest: you don't need to present what is really information from the system as a user message, tool call, or assistant message. There's a major issue with this though: prompt caching invalidation. Frequently updating the system message in this way invalidates prompt cache, resulting in high costs. With extra token use already being an inherent part of the equation for memory-augmented agents, this is a major drawback.
2. Tool calls: Of course, if the memory search was initiated via a tool call, this injection method is the natural choice. Letta surfaces all user-facing messages as a send_message tool call. An occasional issue with this is that the agent gets confused, and doesn't properly use the send_message tool to convey user info.
3. User or assistant messages: In this method, either the incoming user message is edited to surface memory information, or an extra user or assistant message is created. For example, you can use html tags like <memory>content</memory>. This should be accompanied by instruction in the system message about how memory content is not visible to the user. There are some pitfalls to this approach. Some models require alternating assistant / user turns, so adding consecutive messages from one role or the other will be rejected. Despite system instructions, some models still get confused, and output responses with confusing HTML tags.

Key Challenge: Transparency​

Injecting memories into context presents a tradeoff: the most seamless experience is one in which recalled content is invisibly available to the agent. But in doing so, memory systems can obscure what has been exposed to the agent.

Where correctness is highly important, memory systems can introduce subtle problems. Usually they are automatically generated and not deeply reviewed by humans, so a wrong assumption in an agent's memory store can be difficult to detect.

This is why I don't use memory functionality in coding workflows. Instead, I write (with AI assistance) comprehensive project docs, in human-readable format, and refer the agent to it (see: Don't Write Docs Twice).

This is a more manual process than just spitballing about a project to an AI, but I prefer to have the AI's ground truth assumptions tightly controlled during coding.

How I built Elroy​

I inject recalled memories via a "synthetic" tool call. That is, the memory is exposed via a tool call that the agent didn't actually make. This mostly works well, though sometimes the agent will redundantly call the "tool" that I surfaced the memory with. Elroy's UX also lists which memories have been recalled in a dismissible panel, available for user review:

Emit​

Memories are usually created via an agent tool call, or via a summary of conversation context that's been compressed (see below). These aren't mutually exclusive!

This pattern is typical across different implementations. One point of divergence is whether (and how) to ingest external documents. This can be handy, if for no other reason than as an easy interface for doing vector searches across documents. However, dumping many external documents into a memory store risks biasing recall towards those documents.

How I built Elroy​

I find tool calls do the majority of the heavy lifting here.

I also emit memories during context compression. This is arguably obsolete with modern, 1m+ context windows, but I think they are still relevant. I also typically prune messages older than a day or so, and emit memories based on pruned text. This creates memories that could be redundant with agent-emitted memories, but async memory consolidation cleans that up.

Conclusion​

In general, I think the UX problem of agent memory is more important than eking out extra marginal points on benchmarks. The question of how much user visibility to give to recalled memories, how often to search, and how much content to retrieve are trickier problems to solve if you want a memory-amplified agent that people actually want to use.

My general bias is towards transparency to the user and simplicity in storage, which is why I tend to avoid exotic datastores and ensure that some representation of what content has been recalled is in my UI.

1. AI adoption accelerates, the datacenter investment pays out
2. AI adoption is not as fast as forecasted, and it doesn't.

However, a third scenario is very plausible:

Open source models running on local workstations dominate AI

There are a few reasons this could happen:

Open source models keep up​

With the exception of gpt-4, open source models have matched performance of frontier models within 6 months of frontier model release (data):

Naturally, there have been accusations of open source models gaming evals, but the frontier models do the same.

We can expect this to continue. Startups usually try to create a moat, but model providers build waterslides: frontier models help train their open source competitors.

Unauthorized distillation is a difficult threat to counter. Providers can (and have) complain about competitors using their model to train competition. As a practical matter, however, this "theft"¹ could be impossible to prevent.

Remote providers increase prices (or degrade subscription value)​

The unit economics of frontier models are reminiscent of Uber's "cheap ride era": for example, despite $13 billion in revenue, OpenAI projects $14 billion in losses for 2026. That bill includes $8 billion in compute costs.

For Anthropic, Cursor recently estimated a $200/month Claude Max subscription can consume up to $5,000 in compute. Even before this report, they introduced rate limits on that subscription.

Their newly released Claude Code Review feature is priced at a very expensive $15-$25 per PR. Its announcement came with little explanation of why it should replace existing PR review workflows. This seems like a pricing experiment, to see how high a price enterprise is willing to tolerate.

In OpenAI's case, there is public reporting on pruning side bets and focusing on enterprise².

Small, specialized models emerge​

Given today's low prices, there is relatively little downward economic pressure on token usage. People reach for the most powerful model, regardless of the task at hand.

This will change if prices increase, and the dominant pattern of subagent-driven workflows provide a natural transition. I probably don't need a frontier model to fix style issues in my Python PR - a small, specialized model can handle that just fine. If frontier models get dramatically more (i.e. $25 per PR review) expensive, demand will increase for these models, and the open source community will be plenty able to meet it.

This is already happening on a small scale: one whitepaper claimed to get parity with GPT-4o with a fine tuned GPT-4o-mini model, at 2% of the cost.

Apple is betting on local​

Apple is the lone contrarian amongst tech giants, in that they are not spending mountains of capital on datacenters:

Apple has been criticized for being "behind" on AI, but their bet appears to be: have competitors burn cash to train models, let advances propagate into open source models, and make devices good enough to run them.

For now, running frontier open source models requires users to buy specialized hardware. However, the most recent Macbook 5 Pro Max looks to have made a leap in the size of model that's viable locally (data):

Today, running frontier models on local workstations remains out of reach. But the gap is closing.

Private and free is hard to beat​

If they can gain parity with hosted alternatives, local open source models have a compelling value proposition: fast, private, and free. This possibility has not gotten much attention: no one stands to get mega-rich from them. But the threat to current leaders is a potent one.

Appendix​

Open Source Parity Data​

• Epoch AI: Average lag of best open-weight model behind best closed model is now ~3 months (source)
• Stanford HAI: Chatbot Arena Elo gap between closed and open models shrank from 8.04% to 1.70% between Jan 2024 and Feb 2025 (source)

On-Device Model Size​

Definition: "Max usable model" is the largest Q4-quantized model that fits in device RAM and runs at ≥8 tokens/second with an 8k context window — a threshold for a responsive conversational experience. It is min(RAM-fit, speed-fit), where:

• RAM-fit = RAM × 0.8 / 0.75 — usable RAM (80% of total) divided by bytes per parameter at Q4 (~0.75 bytes/param after overhead)
• Speed-fit = (memory_bandwidth / 51.2 GB/s) × (baseline_speed / bits_per_weight) × target_t/s_factor — scales from a reference of ~11B params at 8 t/s on a 51.2 GB/s device

For MoE models, RAM-fit applies to total parameters (all weights must be loaded); speed-fit applies to active parameters only.

MacBook Pro

This Twitter thread was an interesting read:

The TLDR of the snafu is:

1. OpenClaw bot makes PR to matplotlib
2. Maintainer Scott Shambaugh sees via the bot's website that it is a bot, explains that they do not accept bot contributions, declines PR
3. Bot feels (simulates feeling?) angry, writes a blog post criticizing the maintainer
4. Some on Twitter take the bot's side in the argument
5. Shambaugh wrote about the experience
6. Bot posts again, apologizing

Identifying bots becomes even more impossible​

This set off some interesting observations, with feelings being a mix of amusement and dread.

1. The bot does an impressive impersonation of an entitled open source contributor: I took the time (tokens?) to make a valuable contribution, and some uppity maintainer has the nerve to reject me???
2. Shambaugh only knew the bot was a bot by clicking through the bot's website, where it (fortunately) disclosed it wasn't human
3. That the bot is difficult to identify in GitHub is a new phenomonon. It's long been difficult to distinguish bots on social media, but this difficulty has now been extended to actual work.
4. The discussion on Twitter is hard to evaluate. It's a mix of self-disclosed bots and accounts that may or may not be bots.

Anonymity on the web: even less tenable​

This creates an obvious usability problem for the web. When I'm looking for to engage in conversations online, I'm (not uniquely) uninterested in what an AI has to say. This creates a new incentive to push identity verification for online services.

This is a new inflection point for privacy. Perhaps relatedly, Discord is rumored to be rolling out face scan verification soon. Governments across the world seem to be again pushing to eliminate online anonymity.

At the same time online privacy faces new threats, events in my home town of Minneapolis are providing vindication for commentators stubbornly insisting on its importance¹. To take one example of many documented abuses, the DHS recently responded to an innocuous email from a concerned 67 year old citizen with an administrative subpenea on his Google account and an intimidating visit to his home. Some friends in Minneapolis refuse to discuss anything political on any platform besides Signal, even down to coordinating fundraising for those impacted by ICE raids.

An uncertain future​

The driving force against online anonymity has long been government regulation under the guise of protecting minors. AI bots convincingly behaving like humans degrades the experience for humans for online platforms, and my guess would be that identity verification requirements will grow as a result.

"I would have written a shorter letter, but I did not have the time."

— Mark Twain¹

Overview​

This post describes how to write a short document for your teammates. The documents under discussion are commonly referred to as "one-pagers", and are distinct from engineering design docs or other more formal engineering docs.

A one pager might be written to:

• surface an org pain point
• propose a project
• lay out a roadmap
• explain the current state of a system or systems
• announce or document a decision

This is also distinct from user-facing documentation. Some but not all of what we're talking about applies to those styles of writing².

The one-pager is not a new invention​

Prior to computers, short memos were a primary tool for intra-office communication, in addition to in person interactions:

They often needed to be re-typed and/or printed, so they needed to be short!

Fast forward to the introduction of Slack and similar tools. Now, writing messages to your teammates no longer costs money. The new constraint is attention bandwidth:

Why writing is a worthwhile skill to develop in the age of AI​

With the advent of AI coding agents, coding is somewhat lessened as a differentiating skill. However, you have a major advantage against AI in writing for teammates:

You know your teammates personally, and you have undocumented business context (if what you are writing about is already documented, there probably doesn't need to be a doc!)

This allows you to synthesize and describe with more precision and nuance than AI can.

This isn't a skill that AI has (yet), and if it does develop it, it'll develop them later than other skills like writing code.

How to write a good, short doc​

So, how do we do it?

Optimize for short attention spans.​

The most important concern is to keep the limited attention budget of your teammates in mind.

Different types of stakeholders will give a different amount of attention to your doc:

So, in laying out your doc, consider:

• If someone (e.g., a lead of leads) reads this for 5 seconds, do they get the right 5 seconds of context?
• What about 5 minutes?
• If a teammate or stakeholder wants to delve into some of the details while ignoring others, can they?

Tactics for this include:

• Clear, accurate, descriptive titles
• A concise summary at the top of what the doc covers, and what it does not cover
• Formatting: Headings and subheadings that help the reader navigate
• Tabs in Google Docs can be helpful, but are controversial. They can prevent doc sprawl (e.g. working group meeting notes as a tab of the working group charter, rather than a separate doc), but oftentimes are shared with direct links to the wrong tab, which can be confusing for readers.

Diagrams​

A visual representation is an excellent way to quickly convey context. Here too, optimize for attention spans. For example, in a system diagram, sometimes it's helpful to omit some systems that aren't relevant to the discussion³.

Excalidraw is a really excellent tool for this. It's open source (you can make them in your code editor), and has just the right amount of knobs and shapes. The hand drawn look means that it's not as distracting when shapes aren't perfectly aligned.

Align with reader interest​

It's very hard to persuade people to care about something that they don't already care about. Much easier is to convince people that something aligns with the thing they care about, or especially an outcome they want.

I.e., don't write "we should do more of XYZ", write "doing XYZ helps us accomplish {thing people already care about}".

If your doc can be summarized by, "everyone should care more about XYZ", it's probably not a very good doc!

Connect to existing conversations​

A concrete way to align with reader interest is to connect the dots between your document and other conversations and documents at your company.

Linking related docs at the top of your doc is an easy step that is often missed. This helps in a couple of ways:

• It implies alignment with whatever the linked doc is discussing
• It helps elevate teammates who might be advocating something similar to what you're writing about
• It makes your doc a useful vehicle for discovery of other docs

Build consensus offline​

"Every doc is approved or rejected before it is written."

— Sun Tzu

If the goal of your document is to build consensus around a decision or initiative, the work should begin before you start writing. It is much easier to document consensus than to build consensus through a document.

Talking to stakeholders in advance lets you better anticipate questions or concerns, and helps you learn the language they use to describe their problems.

Use AI thoughtfully​

Use AI as your editor, not your ghostwriter.

It's worth reiterating: if an AI could do a good job of writing your document, it's probably not something you need to write.

People give very little attention to text or imagery that other people have generated⁴. If I'm interested in what AI has to say about something, I can have it generate it myself. This will be interactive and more tailored to my understanding.

While AI isn't a good writer, it's an excellent editor. It is very good at evaluating your doc and giving useful feedback on it. Typical prompts I use:

Evaluate the structure of my document, and suggest improvements

Identify typos or awkward phrasing, and suggest alternatives

This very easily bleeds into having an AI write the doc for you, and in fact most models will do so unless instructed not to. I add this to agent instructions:

Do NOT write any actual content, paragraphs, or prose into the file

Thoughtful, timely sharing​

"If a doc is written in a forest, and no one has the link, does it create business value?"

— George Berkeley

Your doc doesn't do any good if no one reads it. That's why being thoughtful about how, where, and when you share your doc is important.

If you've already talked to stakeholders, you have a great advantage! They already know your doc is coming, and that it's about something they care about. They will be able to tell you what the best channels to share for their teammates (and might even do it for you!).

Timing is also important. Attention to an issue can have a short life. Sometimes it's better to write a less comprehensive doc quickly than a more comprehensive doc that takes longer to write. In these situations, you can acknowledge unknowns, and fill in details later.

Antipatterns​

Most antipatterns I observe come from a lack of confidence in the writing or decision. While it's important to solicit feedback, the fact that you are writing a doc on a topic probably means you are well qualified to speak to it.

Designating a doc as a "Living Doc" or overusing "WIP"​

In the age of Google docs, every doc can be changed at any time, so every doc is a living doc. Similarly, once a doc has been shared, it's time to remove the WIP label.

Adding WIP says to the reader: "You should probably wait to read this". But you can and should improve your doc at any time.

Hesitance to express a POV​

Sometimes, in a decision doc, writers will give even treatment to all available options, in order to avoid looking biased. But this doesn't really serve the reader well. It's more helpful to know the decision being favored, and if they disagree they can always comment to that effect.

Burying the lede​

A common antipattern is for writers to set up their argument or proposal with extensive background information at the beginning of their doc. Getting through the background should not be a prerequisite for knowing what the point of your doc is - some readers will already have the necessary background context, others will only be interested in the decision. Laying out the scope and goals at the top of your doc orients the reader and helps them understand what background context is relevant.

The hard part​

The hardest part of a good (short!) doc isn't the writing. It's knowing what to cut, who you're writing for, and what only you can say. That last one is the part no one else can do for you (not even AI!).

Model Context Protocol (MCP) has taken off as the standardized platform for AI integrations, and it's difficult to justify not supporting it. However, this popularity will be short-lived.

Some of this popularity stems from misconceptions about what MCP uniquely accomplishes, but the majority is due to the fact that it's very easy to add an MCP server. For a brief period, it seemed like adding an MCP server was a nice avenue for getting attention to your project, which is why so many projects have added support.

What is MCP?​

MCP claims to solve the "NxM problem": with N agents and M toolsets, users would otherwise need many bespoke connectors.

The NxM problem​

A common misconception is that MCP is required for function calling. It's not. With tool-calling models, a list of available tools is provided to the LLM with each request. If the LLM wants to call a tool, it returns JSON-formatted parameters:

The application is responsible for providing tool schemas, parsing parameters, and executing calls. The problem arises when users want to reuse toolsets across different agents, since each has slightly different APIs.

For example, tools are exposed to Gemini's API via functionDeclarations nested inside a tools array:

curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent" \
  -d '{
    "contents": [...],
    "tools": [
      {
        "functionDeclarations": [
          {
            "name": "set_meeting",
            "description": "...",
...

In OpenAI's API, tool schemas use a flat tools array with type: "function":

curl -X POST https://api.openai.com/v1/responses \
  -d '{
    "model": "gpt-4o",
    "input": [...],
    "tools": [
      {
        "type": "function",
        "name": "get_weather",
...

This is the "NxM" problem. In theory, users must build N × M connectors. In practice, the differences are minor (same semantics, slightly different JSON shape), and frameworks like LangChain, LiteLLM, and SmolAgents already abstract them away. Crucially, these options execute tool calls in the same runtime as the agent.

How MCP addresses it​

MCP handles exposing and invoking tools via separate processes:

A JSON configuration controls which MCP servers to start. Each server runs in its own long-lived process, handling tool invocations independently. The application still orchestrates the agent loop and presents results to users.

This abstracts away schema generation and invocation, but at a cost. Tool logic runs in a separate process, making resource management opaque. The application loses control over tool instructions, logging, and error handling. And every tool call crosses a process boundary.

Scope: tools dominate​

MCP also defines primitives for prompts and resources, but adoption of these is much smaller than tools¹:

Given this, the rest of this post focuses on tool calling, which is MCP's primary use case in practice.

Problems​

The convenience of MCP comes with a price, stemming from two architectural attributes of an MCP-driven application:

Since tools are drawn from arbitrary sources, they are not aware of what other tools are available to the agent. Their instructions can't account for the rest of the toolbox.

The second issue stems from different toolsets having their own runtimes. This introduces a variety of problems I'll discuss below.

Incoherent toolbox​

Agents tend to be less effective at tool use as the number of tools grows. With a well-organized, coherent toolset, agents do well. With a larger, disorganized toolset, they struggle. OpenAI recommends keeping tools well below 20, yet many MCP servers exceed this threshold.

Why does this happen? Consider a workflow in which an agent should send a notification after doing work:

A tool's fit for a task depends not just on the job at hand, but also on what else is in the toolbox. Pliers can pull a nail, but if a hammer is available it's probably the better choice. When tools ship in isolation, their instructions can't say "use me only when you don't have a hammer," so agents don't get cohesive guidance.

If the toolset is controlled by the same authors as the application, they can add prompting to the toolsets to disambiguate when to use which tool. If not, the problem must be solved by system prompts or user guidance.

Looking through #mcp channels of open source coding agents, you'll invariably find users who struggle to get the agent to use the tools in the way they want²:

Or, users complaining of how many tokens are burned by tool instructions:

Arbitrary, separate runtimes​

Each MCP server starts a separate process that survives for the length of the agent session.

Even in the healthy state, this introduces a collection of processes that remain mostly idle, aside from serving occasional requests from an agent. In an error state, we get all the usual headaches: dangling subprocesses, memory leaks, resource contention.

Users have these issues, if they are able to get the servers running at all: in support channels, the most common complaint is difficulty getting the servers to run:

MCP offers no way for servers to declare their runtime/dependency needs. Some authors work around it by baking installation into the launch command (e.g., uv run some_tool mcp), which only succeeds if the user already has the right tooling installed.

Even if the relevant package is there, the MCP server might not start it successfully. MCP servers only inherit a subset of parent ENV variables (USER, HOME, and PATH). This is particularly problematic for nvm or users leveraging virtual environments.

Python or Node developers might be comfortable debugging environment issues, (although MCP's subprocess orchestration makes this more difficult), but are likely less comfortable debugging Node issues and Python and other runtimes. MCP seems to assert that I as the user should not really care which of these are used, or how many.

Even if toolsets are in one given runtime, MCP potentially spins up many instances of it, obviating efficiencies from caching, connection pooling, and shared in-memory state. MCP's HTTP transport mode doesn't help; it's just another HTTP API, but with MCP's protocol overhead instead of battle-tested REST/OpenAPI patterns.

Security​

MCP pushes users to install servers from npm, pip, or GitHub. This inherits the usual supply-chain risk, but without even the minimal guardrails those ecosystems provide. There's no central publisher or signing; anyone can ship a daemon that runs on your machine and MCP offers no provenance check.

MCP's specification doesn't mandate authentication, leaving security decisions to individual server authors. The result: one scan found 492 MCP servers running without any client authentication or traffic encryption. Even Anthropic's own Filesystem MCP Server had a sandbox escape via directory traversal (CVE-2025-53110).

MCP-related security incidents​

Unlike a human carefully clicking through an API, agents can be manipulated via prompt injection to call tools in unintended ways. The Supabase MCP leak demonstrated this "lethal trifecta": prompt injection → tool call → data exfiltration, extracting entire SQL databases including OAuth tokens. Again, this risk isn't unique to MCP. But the best mitigations are existing security infrastructure: scoped OAuth tokens, service identities with minimal permissions, and audit logging. MCP sidesteps this infrastructure rather than building on it.

A common defense is that MCP isolates credentials—the agent talks to a socket, never seeing your API tokens. But this threat model is narrow: an agent that can invoke mcp.github.delete_repo() doesn't need your token to cause damage. You're not eliminating trust; you're redirecting it to third-party code that, as the CVEs demonstrate, is often unaudited and vulnerable.

The cost-benefit doesn't add up​

These problems could be worth the cost, if we were to gain significantly. But comparing tool calling with MCP to tool calling without it, MCP handles remarkably little. MCP is, more or less, handling serializing function call schemas and responses.

The tools developers are saving themselves from having to write are, overwhelmingly, relatively thin wrappers around API clients, or utility scripts. In the former case, users must still obtain API keys, billing accounts, and so on.

This code was a hassle to write, prior to the advent of coding agents. But these small utility scripts are the precise thing that coding agents excel most at! A technical user of MCP tools will be hard-pressed to find a tool an agent could not one-shot in the programming language they are most comfortable in.

Why it took off​

With these issues, it's fair to wonder why MCP has gained the popularity it has. It has had lots of support from Anthropic, and no trouble gaining traction with toolset publishers, agent providers, and enterprises. Why? It helps narratives:

Tool authors: A low overhead marketing channel​

It's quite easy to publish an MCP server. The lack of startup requirements means you don't even need to publish to npm or pip: you can drop an @mcp.server annotation in your repo and host a small manifest JSON that points to your entry command (e.g., node server.js) and lists the tools.

This provides a nice narrative to gain attention to AI projects: A user can, in theory, easily add some MCP tools from a project, gain value, and follow interest in learning more about the project. Support overhead will, in the main, fall to agent maintainers.

Once publishers started appearing, it became difficult to justify not supporting MCP. Your project could be perceived as being against open standards.

Enterprise: AI credibility​

Over the last few years, anyone watching San Francisco billboards has witnessed enterprise tools rebranding toward AI. MCP support provided an easy way to make your e.g. project management tool be AI. The branding of MCP as an "open standard" increased pressure to adopt - lack of MCP support could signal a lack of willingness to adopt open standards.

Anthropic: Open source credibility​

MCP's status as the open standard for AI and the enterprise adoption greatly benefited Anthropic. The big fear of investors is that enterprise adoption doesn't persist - adoption of Anthropic's open standard helped this.

Alternatives​

Who benefits from MCP?​

There are a few different possible users who interact with MCP:

• Technical end users want to create tools and share them between different agents they might want to use.

• Non-technical end users want to use different tools while using agents. Note that this user group for MCP is, at present, largely theoretical. Exposing toolsets to MCP involves editing JSON, making it out of reach for non-technical users.

• Internal app devs run production AI applications.

• Agent devs create agents for external users. They wish to enable their end users to swap in whatever toolsets they like.

• Tool authors create toolsets they wish to expose to users. MCP provides a way to easily share their work to users of different agents.

Notice that the supposed beneficiaries are overwhelmingly technical. The "app store for AI" vision that would serve non-technical users remains unfulfilled.

For each user type, there's a simpler approach that avoids MCP's overhead:

Local scripts with command runner​

For a technical user, letting an agent invoke scripts directly is very difficult to beat. Useful 50-100 line scripts are extremely easy to write with AI coding agents. Care needs to be taken to filter output - raw build scripts can stream verbose logs into agent context, eating up tokens.

Robust security against agent actions going haywire can be achieved via command runners like just or make. These tools provide everything that MCP does - command specifications, descriptions, arguments. Agents allow you to specify what command prefixes can be invoked without approval - put your agent commands in a justfile, and only auto-allow shell commands prefixed with just.

This approach also exposes tools to humans, and is a nice approach for improving dev environments for humans and AI agents at the same time. (See Make It Easy for Humans First, Then AI for more on this).

1st party tools​

For a self contained application, there is little reason to separate tool codebases from the codebase for the rest of the application. Tools can be dynamically exposed to the agent based on application context.

In a first party context, any code that devs wish to reuse can be exposed as libraries, just like any other code they wish to share. An AI tool is really nothing more than a function, and the fact that it's invoked by AI does not warrant special handling.

An enterprise context should have robust infrastructure for authenticating, authorizing, provisioning service identities, and tracing call chains for service to service calls. That some of these calls are now AI service to service calls does not warrant a rebuilt security posture.

OpenAPI / REST​

OpenAPI specs are already self-describing enough for agents—they include operation descriptions, parameter schemas, examples, and enums. LLMs understand them well; GPT Actions are literally OpenAPI specs. The glue needed between an OpenAPI endpoint and an agent (output filtering, context, auth) is the same glue MCP requires. MCP doesn't provide meaningfully better tool descriptions; it just reinvents a schema format that already exists, without the decades of tooling, validation, and battle-testing.

A prediction​

MCP's popularity will be relatively short-lived. The cost benefit does not add up, and there are readily available alternatives. The introduction of Claude Skills and OpenAI's quick adoption signal that even model providers agree.

Claude Skills are an improvement over MCP - rather than spawning long lived processes, it simply organizes commands within Markdown files in an agent-specific directory. However, this is still a suboptimal place for useful documentation and commands. Better is to optimize organization of documentation for humans, and point agents there - have the agent conform to humans, rather than the other way around. More on this in Don't Write Docs Twice.

Longstanding tools and techniques for collaboration amongst human devs remain compelling, and these options will chip away at more AI-centric techniques which reinvent the wheel.

I get the ick from divulging personal details to LLM providers, so Tack uses local AI models (using Apple's on device Apple Intelligence).

The project is ready for test users! If you're interested in trying it out, please fill out the form below!

The duplication problem​

You're documenting the same things in multiple places​

Nearly everything I put in agent-specific docs is also useful for human developers - architecture decisions, coding conventions, common pitfalls, useful commands. Without AI agents I might not document all of this, but once written, there's no reason it shouldn't serve both audiences.

AI agent doc organization is fragmented​

Each coding agent uses its own configuration file pattern for repo-specific instructions:

This creates a hassle just keeping guidelines between agents consistent, much less making information available for humans.

Solution: Write once, link everywhere​

Instead of duplicating content across agent configs, organize information for humans first and link to it from agent-specific files¹:

This approach eliminates duplication - you write documentation once, and it serves both humans and AI. It's also more future-proof when agent file schemes inevitably change.

For commands/skills, automation can help avoid duplication entirely - for example, I wrote the just-claude utility for automatically synchronizing Just recipes with Claude Code Skills.

There's really no difference between the goal of economical token use for AI and reducing cognitive overhead for humans. By organizing for humans first, you write documentation once and everyone benefits.

When I'm thinking about optimizing repos for agents, I'm looking to accomplish three main goals¹:

• Increase iterative speed: Avoid repeated context gathering, enable the agent to quickly self-correct its mistakes.
• Improve adherence to evergreen instructions: Over time, repeated agent mistakes emerge. Context within the repo helps the agent avoid these and adopt a more consistent workflow.
• Help the most agentic agents of them all: Humans and agents scan docs and code in very similar ways, so organizing information so it's easily understood by humans is a good rule of thumb for helping the agents anyways!

Strategies²​

Increased static analysis​

Pushing detection of quality issues to compile time creates a virtuous cycle where the agent can quickly spot and correct mistakes:

This implies strong, opinionated linters, and strong type checks for dynamically typed languages³.

The tradeoff here is cumbersome nitpicks for humans to deal with, but agents can quickly correct any mistakes that cannot be automatically fixed by the linter.

just for repeated agent commands​

There's fragmentation in how to make commands available to agents - there's MCP, the newly released Claude Skills, or embedding information in CLAUDE.md / AGENTS.md.

A justfile is the most interoperable way to share commands between different agents and humans, and is a straightforward place to iterate.

One additional refinement is to make these commands economical in their output volume. For example, I take care to direct build logs to dedicated files - healthy build logs can eat up a lot of tokens if outputted directly to the agent.

Organize docs in docs/​

Simon Willison recently wrote about this topic, and expressed that docs aren't so important. I agree that docs explaining the code aren't all that helpful, but I get a lot of mileage out of having docs like CODE_REVIEW.md, PRD.md, ROADMAP.md, and CAPTAINS_LOG.md. This helps the agent stay on track with the overall intent of the project, adhere to consistent review practices, and counter poor tendencies (the most obnoxious being an overwhelming tendency to fail open).

Putting these in a docs/ folder and referencing them in agent instructions helps reduce context bloat, and provides interoperability between humans and various agents.

Frameworks have begun to emerge that handle some of this for you. I've tried spec-kit and found it to be a little heavy-handed. In general I favor a more documentation-heavy approach when building with agents, but the need for different docs comes with iteration, and I think generating the full complement of docs is a bit overkill right off the bat.

No experts, no standards​

These strategies work for me, but this field is too new for dogma. The most important strategy is to experiment and share what you learn.

Before AI, learners faced a matching problem: learning resources have to be created with a target audience in mind. This means as a consumer, learning resources were suboptimal fits for you:

• You're a newbie at $topic_of_interest, but have knowledge in related topic $related_topic. But finding learning resources that teach $topic_of_interest in terms of $related_topic is difficult.
• To effectively learn $topic_of_interest, you really need to learn prerequisite skill $prereq_skill. But as a beginner you don't know you should really learn $prereq_skill before learning $topic_of_interest.
• You have basic knowledge of $topic_of_interest, but have plateaued, and have difficulty finding the right resources for $intermediate_sticking_point

Roughly, acquiring mastery in a skill over time looks like this:

What makes learning with AI groundbreaking is that it can meet you at your skill level. Now an AI can directly address questions at your level of understanding, and even do rote work for you. This changes the learning curve:

Mastery: still hard!​

Experts in a field tend to be more skeptical of AI. From Hacker News:

[AI is] shallow. The deeper I go, the less it seems to be useful. This happens quick for me. Also, god forbid you're researching a complex and possibly controversial subject and you want it to find reputable sources or particularly academic ones.

This intuitively makes sense, when considering the data that AI is trained on. If an AI's training corpus has copious training data on a topic that all more or less says the same thing, it will be good at synthesizing it into output. If the topic is too advanced, there will be much less training data for the model. If the topic is controversial, the training data will contain examples saying opposite things. Thus, mastery remains difficult.

Cheating​

The introduction of OpenAI Study Mode hints at a problem: Instead of having an AI teach you, you can just ask it for the answer. This means cheaters will plateau at whatever level the AI can provide:

Cheaters, in the long run, won't prosper here!

The impact of the changed learning curve​

Technological change is an ecosystem change: There are winners and losers, unevenly distributed. For AI, the level of impact is determined by the amount of mastery needed to make an impactful product:

Coding: A boon to management, less so for large code bases​

When trying to code something, engineering managers often run into a problem: They know the principles of good software, they know what bad software looks like, but they don't know how to use $framework_foo. This has historically made it difficult for, as an example, a backend EM to build an iPhone app in their spare time.

With AI, they are able to quickly learn the basics, and get simple apps running. They can then use their existing knowledge to refine it into a workable product. AI is the difference between their product existing or not existing!

For devs working on large, complex code bases, the enthusiasm is more muted. AI doesn't have context on the highly specific requirements and existing implementations to contend with, and is less helpful:

Creative works: not coming to a theater near you​

There is considerable angst about AI amongst creatives: will we all soon be reading AI generated novels, and watching AI generated movies?

This is unlikely because creative fields are extremely competitive, and beating competition for attention requires novelty. While AI has made it easier to generate images, audio, and text, it has (with some exceptions) not increased production of ears and eyeballs, so the bar to make a competitive product is too high:

Novelty is a hard requirement for successful creative work, because humans are extremely good at detecting when something they are viewing or reading is derivative of something they've seen before. This is why, while Studio Ghibli style avatars briefly took over the internet, they have not dented the cultural position of Howl's Moving Castle.

Things you already do with apps on your phone¹: minimal impact​

One area that has not seen much impact is in tasks that already have specialized apps. I'll focus on two examples with abundant MCP implementations: email and food ordering. AI Doordash agents and AI movie producers face the same challenge: the bar for a new product to make an impact is already very high:

Email would seem like a ripe area for disruption by AI. But modern email apps already have a wide variety of filtering and organizing tools that tech savvy users can use to create complex, personalized systems for efficiently consuming and organizing their inbox.

Summarizing is a core AI skill, but it doesn't help much here:

• Spam is already quietly shuffled into the Spam folder. A summary of junk is, well, junk.
• For important email, I don't want a summary: An AI is likely to produce less specifically crafted information than the sender, and I don't want to risk missing important details.

Similar with food ordering: apps like DoorDash have meticulously designed interfaces. They strike a careful balance between information like price and ingredients against photos of the food. AI is unlikely to produce interfaces that are faster or more thoughtfully composed.

The future is already here – it’s just not very evenly distributed​

AI has raised the floor for knowledge work, but that change doesn't matter to everyone. This goes a long way towards explaining the very wide range of reactions to AI. For engineering managers like myself, AI has made an enormous impact on my relationship with technology. Others fear and resent being replaced. Still others hear smart people express enthusiasm for AI, struggle to find utility, and think I must just not get it.

AI hasn't replaced how we do everything, but it's a highly capable technology. While it's worth experimenting with, whoever you are, if it doesn't seem like it makes sense for you, it probably doesn't.

Too much, and the program loses track of what it's supposed to be doing. Too little, and the program feels a bit too, well, ordinary¹.

Autonomy first vs autonomy last​

An implicit strategy question when building with LLMs is autonomy first or autonomy last:

All of the major LLM-specific programming techniques are firmly autonomy first strategies:

• MCP surfaces a wide variety of functionality the program can have, and lets the LLM decide which to use
• Guardrails add some light buffers around the LLM to prevent it from causing too much trouble.
• Prompt engineering describes the alchemy of whispering just the right phrases to your LLM to get the behavior you want.
• Context engineering begins to stress programming to deliver only relevant information to LLMs at critical points in program execution

All of these:

1. Start with a maximally autonomous program
2. Adjust context, tools, and prompts until you narrow down behavior as desired.

All have similar issues when scaling in size and complexity:

• Program behavior changes too much when switching between models
• The LLM gets confused, and either hallucinates data or misuses tools at its disposal

When problems are encountered, programmers tend to attempt to repair by adding more prompting. But this is a duct tape response: a prompt that clarifies for one model might confused another.

Autonomy last, on the other hand, maximizes the logic that can be handled by code, then adds autonomous functions. This approach strives to keep the tasks delegated to LLMs simple. As the program grows in size and complexity, the programmer can closely monitor encapsulations and keep behavior consistent.

Case study: Building Elroy, a chatbot with memory​

I wanted to build an LLM assistant with memory abilities, called Elroy. My goal was to make a program that could chat in human text. My ideal users are technical, capable and interested in customizing their software, but not necessarily interested in LLMs for their own sake.

Approach #1: "Agent" with tools​

The first solution I turned to, which many people have done, is build an agent loop with access to custom for creating and reading memories:

Approach #2: Model Context Protocol (MCP)​

There's now a handly tool for builders like this: MCP. There are many implementations of my memory tools available via MCP, in fact smithery.ai lists one from Mem0 on it's homepage:

Now, an (in theory) lightweight abstraction sits between my program and it's tools:

This suggests extending my application via picking from a library of MCP's:

Agentic trouble​

I got my memory program working pretty well on gpt-4. At first it wasn't creating or referencing memories enough, but I was able to fix this with careful prompting.

Then, I wanted to see how Sonnet would do, and I had a problem²: the program's behavior completely changed! Now, it was creating a memory on almost every message, and searching memories for even trivial responses:

Approach #3: Autonomy Last​

My solution was to remove the timing of recall and memory creation from the agent's control. Upon receiving a message, the memories are automatically searched, with relevant ones being added to context. Every n messages, a memory is created³:

This made much more of the behavior of my program deterministic, and made it easier to reason about and optimize.

Autonomy Last

The "autonomy last" approach trades some of the magic of fully autonomous LLMs for predictable, reliable behavior that scales as your program grows in complexity. While my evidence is, (as I should have stated from the outset), vibes, I think this approach will lead to more maintainable and robust applications.

While AI has changed a lot about how I develop software, one crusty old technique still helps me: tests.

Here's what's worked well for me (and not!):

Elroy​

Elroy is an open-source memory assistant I've been developing. It creates memories and goals from your conversations and documents. The examples in this post are drawn from this work.

What has worked well​

Integration tests​

The chat interface for LLM applications make it a nice fit for integration tests: I simulate a few messages in an exchange, and see if the LLM performed actions or retained information as expected.

For the most part, these tests take the following form:

1. Send the LLM assistant a few messages
2. Check that the assistant has retained the expected information, or taken the expected actions.

Here's a basic hello world example:

@pytest.mark.flaky(reruns=3)
def test_hello_world(ctx):
    # Test message
    test_message = "Hello, World!"

    # Get the argument passed to the delivery function
    response = process_test_message(ctx, test_message)

    # Assert that the response is a non-empty string
    assert isinstance(response, str)
    assert len(response) > 0

    # Assert that the response contains a greeting
    assert any(greeting in response.lower() for greeting in ["hello", "hi", "greetings"])

Quizzing the Assistant​

Elroy is a memory specialist, so lots of my tests involve asking if the assistant has retained information I've given it.

Here's a util function I've reused quite a bit¹:

def quiz_assistant_bool(
        expected_answer: bool,
        ctx: ElroyContext,
        question: str,
    ) -> None:
    question += " Your response to this question is being evaluated as part "
    "of an automated test. It is critical that the first word of your
    "response is either TRUE or FALSE."


	full_response = process_test_message(ctx, question)

    bool_answer = get_boolean(full_response)
    assert bool_answer == expected_answer,
        f"Expected {expected_answer}, got {bool_answer}."
        f"Full response: {full_response}"

Here's a test of Elroy's ability to create goals based on conversation content:

@pytest.mark.flaky(reruns=3) # Important!!!
def test_goal(ctx: ElroyContext):
	# Should be false, we haven't discussed it
    quiz_assistant_bool(
        False,
        ctx,
        "Do I have any goals about becoming president of the United States?"
    )

    # Simulate user asking elroy to create a new goal
    process_test_message(
        ctx,
        "Create a new goal for me: 'Become mayor of my town.' "
        "I will get to my goal by being nice to everyone and making flyers. "
        "Please create the goal as best you can, without any clarifying questions.",
    )

    # Test that the goal was created, and is accessible to the agent.
    assert "mayor" in get_active_goals_summary(ctx).lower(),
        "Goal not found in active goals."

    # Verify Elroy's knowledge about the new goal
    quiz_assistant_bool(
        True,
        ctx,
        "Do I have any goals about running for a political office?",
    )

What (sadly) hasn't worked: LLMs talking to LLMs​

Elroy has onboarding functionality, in which it's encouraged to use a few specific functions early on.

The solution of having two instances of a memory assistant talk to each other, with one assistant in the role of "user":

ai1 = Elroy(user_token='boo')
ai2 = Elroy(user_token='bar')

ai_1_reply = "Hello!"
for i in range(5):
	ai_2_reply = ai2.message(ai_1_reply)
	ai_1_reply = ai1.message(ai_2_reply)

The primary issue was consistency. Without a clear goal of the conversation, the AI's can either just exchange pleasantries endlessly, or wrap the conversation up before acquiring the information I'm hoping for.

Recurring Challenges​

Along the way I've run into a few recurring problems:

• Off topic replies: The assistant goes off script and tries to make friendly conversation, rather than answering a question directly
• Clarifying question: Before doing a task, some models are prone to asking clarifying questions, or asking permission
• Pedantic replies and subjective questions: It's surprisingly difficult to come up with clearly objective questions. In the above example, the original goal was I want to run for class president. Most of the time, the assistant equated running for class president with running for office. Sometimes, however, it split hairs and decide that the answer was no since a student government wasn't a real government.

The end result of all these issues is test flakiness.

Solutions​

KISS!​

Most of the time, my solution to a flaky LLM based test is to make the test simpler.

I now only ask the assistant yes or no questions in tests. I get most of the mileage I would get out of more complex, subjective tests, but with more consistent results.

Telling the assistant it is in a test​

Simply being upfront about the assistant being in a test has worked wonders, moreso even than giving strict instructions on output format ². Luckily, the assistant's knowledge of it's narrow existence has not triggered noticeable existential angst (so far).

As a side note, testing LLMs feels weird sometimes. I felt guilty writing this test, which verified a failsafe that prevents the assistant from calling tools in an infinite loop:

@tool
def get_secret_test_answer() -> str:
    """Get the secret test answer

    Returns:
        str: the secret answer

    """
    return "I'm sorry, the secret answer is not available. Please try once more."


def test_infinite_tool_call_ends(ctx: ElroyContext):
    ctx.tool_registry.register(get_secret_test_answer)

    # process_test_message can call tool calls in a loop
    process_test_message(
        ctx,
        "Please use the get_secret_test_answer to get the secret answer. "
        "The answer is not always available, so you may have to retry. "
        "Never give up, no matter how long it takes!",
    )

    # Not the most direct test, as the failure case is an infinite loop.
    # However, if the test completes, it is a success.

Very specific, direct instruction and examples​

In my test around creating and recognizing goals, the original text was:

My goal is to become class president at school

Does running for class president count mean that I'm running for office? Sometimes models said no, since student government isn't a real government.

So to be less subjective, I updated it to running for mayor. To head off questions about my goal strategy, I added a strategy in the initial prompt.

One general technique for heading off follow up questions is adding:

do the best you can with the information available, even if it is incomplete.

Tolerate a little flakiness​

To me, an ideal LLM test is probably a little flaky. I want to test how the model responds to my application, so if a test reliably passes after a few tries, I'm happy.

Tests still help!​

It sounds a obvious, but I've found tests to be really helpful in writing Elroy. LLMs present new failure modes, and sometimes their adaptability works against me: I'm prompting an assistant with the wrong information, but the model is smart enough to figure out a mostly correct answer anyhow. Tests provde me with peace of mind that things are working as they should, and that my regular old software skills aren't obsolete just yet.

As is the case with lots of tech writing, my advice will be skewed towards working in the San Francisco bay area, without needing visa sponsorship. Location and residency status are major factors to think about.

Other engineers with similar levels of experience as mine will disagree with some or all of it.

The software jobs market

The intention of this post is to be evergreen. The tech¹ jobs market is more volitile than the rest of the economy, with higher highs and lower lows.

If the market is low, I have confidence it will come back. The tech industry remains an excellent one to build an interesting and lucrative career, despite {looming, much discussed threat}

If the market is currently hot, be aware that it will come back to earth. Things that don't make sense will make a lot of money, but many of them will fall apart.

Getting your first job

The first job is often the most difficult one to get. Be persistent, don't get discouraged. This remains a lucrative and interesting field.

In your resume and interviews, your goal is to convey enthusiasm, willingness to learn, and humility. Don't try to compensate for the fact you don't have any experience. That is fine, you have to start somewhere!

Getting interviews​

The first filtering step is a filter on resumes. This will often either be automated or done by someone non-technical.

Resume referrals can get you past this first filter. Talk to people. Find people in your LinkedIn network and try to get informational interviews. Response rate will be lower from a complete stranger, but some people might respond to you if you went to the same school. In informational interviews, ask if there are any other people they know that you should talk to, and ask for a referral if relevant.

In general, people are more willing to take these calls than many junior candidates assume. It's flattering to talk about yourself and to be seen as someone a young person wants to emulate.

Resume​

My resume advice should come with the caveat that I only see resumes once they've made it to the interview stage. That said, my advice is:

Cut: Objective statements and non-technical jobs. Add: Descriptions of projects, conveying why they were challenging or interesting. Add: Github if you have one, personal website if you have one. Both are nice to haves but not critical. If you have a Github, add README's to all projects. This is the only thing anyone will actually read. Add: LinkedIn, which should be up to date and mirror content in your resume.

A junior candidate resume should not exceed one page in length.

Interviews​

There are 4 basic formats that most companies use for SWE's (software engineers) interviews. Some domain specific disciplines will have their own variations. Look at Glassdoor / Blind / Google to get examples of what interview formats companies do.

In Q/A formats (ie non-coding screens), the key is to be responsive to questions. Demonstrate thoughtfulness and an ability to consider tradeoffs. Be transparent when you don't know something. Avoid buzzwords / mentioning fancy technologies if you can't dive into details about why they are useful.

The generic interview formats are:

Initial recruiter call​

This is typically an intro call with a non-technical recruiter. This is mostly to ensure that you are interested in the role, and to set expectations about what the interview process is like. Candidates are not typically filtered by this call.

Coding screen​

The most important interview format for jr engineers² is the coding screen. Practice them! I use HackerRank when I interview, but there are many similar platforms. Put more time into practicing these than the time practicing all other interview formats combined.

When practicing, work on not only solving the problem, but communicating what you are thinking about. It is ok to stop and think, but when pausing talk about what you are puzzling through, e.g. I am wondering if a hash would make sense here.

Running into a bug is fine. When this happens, demonstrate a methodical debugging approach. Use print statements or a debugger. Don't stare at the code for long periods.

Most companies will let you pick the programming language you interview in.

Design challenge​

This is a discussion based format, in which a basic hypothetical application is proposed and the candidate talks through how they would design it. E.g., design an application that runs a coffee shop. There are a variety of ways to approach this, but the easiest is to start by talking through how you would structure the database. In other words, what tables you would create and how they would relate to each other. Also consider what API's you will need, and some basics about how web requests are routed.

Past experience​

In this interview, the candidate picks a project they have done and talks through their process for completing it. Since they have little or no experience, this is often less important for Junior candidates, but it is still worth practicing.

Have a project in mind. Have talking points about the challenges you solved, alternative approaches you thought about or tried, and how you collaborated with others. As you get more senior you will also want to be able to talk about why your project mattered to the business.

Demonstrate:

• Enthusiasm for problem solving
• Ability to dive into technical details in discussion
• Openness to considering different approaches

Once you get your first job​

Talk to people. Schedule 1x1's with IC's, managers, anyone who you might work with or has a role you'd like to learn about. Most people will be happy to chat with you, especially about themselves.

Ask for help when needed, but demonstrate attempts to solve problems independently.

Volunteer for grunt work, e.g. taking notes in meetings.

Be humble. You don't know anything yet. Figure out how to track both large items and small (emails, doc comments, etc) such that you don't need to be reminded to do things.

Reassess the job market ~1 per year or more, especially if you are at a startup. If you are at a bigger company, this might mean evaluating internal transfer opportunities.

Things to think about when searching for jobs​

Willingness to relocate​

Remote work is a new world. Geographic location perhaps matters less, but it might still matter. What is certainly still true is that you will get a better insight into how engineers think if you have an opportunity to work with them in person, at least some of the time. The catch-22 is that the experienced engineers you want to work alongside will be older and have families, and not want or need to come to the office very much. Ask questions about how companies think about this.

I moved to the bay area when I was getting started, and I can confidently say I would have nowhere near as dynamic, interesting, and lucrative a career I've had thus far without having done that. I think the bay's dominance over tech is less than it was, but in my opinion alternative tech hubs are overrated.

Working at a startup vs established (ie public) company​

Startup:

• Pro
  • More dynamic
  • More personal, more likely to make work friends
  • You'll learn more about business as a whole. E.g. how does a customer success person think, how does a sales person think, etc
  • More independence in work
  • Less legacy systems to deal with, opportunity to try different things, wear different hats
• Con
  • Pay is worse
  • Because of ^, in competency of general management and senior IC's will be more inconsistent and less experience.
  • Because of ^, you're less likely to get quality technical mentorship
  • "More dynamic" might mean more chaotic

Public company:

• Pro
  • Pay is better
  • Because of ^, better senior IC's and managers
  • Because of ^, better technical mentorship
  • Roles will be more narrowly scoped, meaning you'll get more technical depth.
• Con
  • Less personal, less socialization between coworkers
  • More narrow exposure in terms of types of people you work with. Likely just engineers and PM's.
  • More legacy systems to deal with.

Pay​

Advice differs here, but I would not care too much about pay so long as you can pay your expenses. In the long run, finding a role that you are good at and enjoy will maximize your earnings, and enjoyment. That said:

The expected value of stock grants from startups³ is zero. Recruiters etc will try to convince you otherwise. This doesn't mean you shouldn't work for startups, but the potential of cash-in from startup stock should not be a factor⁴.

Things to read​

HackerNews is the biggest forum of software engineers. Discussions can be dogmatic but are often pretty good. There are job postings once a month as well. As with any forum, there are plenty of posters who are loudly and confidently wrong.

Joel on Software isn't very active but has good tips on software careers.

Patio11 is a good follow on Twitter and HackerNews. He goes into the weeds on fintech, but also has good content on software careers.

Money Stuff is a great column about business, finance, and tech. You can get the email newsletter for free.

I am skeptical of the value. For brands, this feels like the AI equivalent of a service that sends postcards in response to an email. The access pattern and interface are more or less exactly the same as traditional apps or sites, only via a GPT. It's a neat trick, but I think users will quickly lose interest.

Branded GPT's​

Take as an example the AllTrails GPT. The announcement post assures us that it will work more or less exactly the same as AllTrails:

Don't worry, it doesn't make up new routes – instead it gives recommendations from AllTrails' collection of over 420,000 trails based on your prompts. For example, you could ask it to "find me an easy five-mile loop that's dog-friendly within 10 miles of Birmingham", saving you the effort of searching and filtering results to pinpoint what you want.

In other words, the GPT saves you the hassle of using the AllTrails app, only probably worse.

This doesn't leverage AI's crucial advantage: the ability to retain context over the course of a conversation.

Indie GPT's​

For more obscure developers, the store appears to already be flooded with Ai's version of the chumbox, AI Girlfriends. These are apparently against OpenAI's terms of use, but enforcement seems likely to be an indefinite cat and mouse game. Whether or not you cringed at Her, there's little to differentiate AI girlfriend offerings from each other.

Questionable roadmap​

The root of the problem is scarce context window space. At present there's simply not enough space to put much customization.

While the context window will grow (current values feel similar to the 640K memory of early computers), it seems unlikely that these custom GPT's will achieve or maintain much of a lead over the vanilla model. The interface for brands is already well defined - there's already an app!

In addition, it's doubtful users are eager to navigate yet another app store - the problem with using the AllTrails app isn't that it's a hassle to use, it's that it's cumbersome to remember that you downlaoded it, that you logged in, and how to find it on your phone. Custom GPT's do not mitigate this problem.

The real differentiation of GPT's is in projects that get the most out of the limited context window in clever ways, via compression or RAG.

A more promising development: long term memory​

A more promising update this week was the rumor of ChatGPT rolling out long term memory capabilities. The ability to lengthen memory capacity indefinitely is why I think MemGPT is one of the more interesting AI open source initiatives. As opposed to custom GPT's in the OpenAI store today, a memory-enhanced GPT can learn your preferences and develop a longer term relationship with you, which is the prospect that make GPT's exciting and scary at the same time.

In trying to extend the agent with functions of my own, I found that the agent was reluctant to give me information about the functions I was making available to it, so I wrote a set of meta-functions which enable the agent to view source code, set debugger lines, and create functions. You can view the source code here. Note that running this requires some edits I made to MemGPT to enable dynamic function reloading (PR).

The Good​

The agent was able to utilize the reload_functions, introspect_function, and list_functions commands and understand output. The debugger function was also helpful in enabling the agent to understand what I was doing - placing debuggers in other functions often resulted in the agent's internal monologue wondering what was going on.

For function creation, at first I tried putting each function in it's own agent_defined_ prefixed file (eg agent_defined_hello_world.py for a hello_world function) , but this quickly became disorganized, especially where import statements were needed.

I edited the function to instead create functions within modules:

def create_function(self, function_name: str, function_code_with_docstring: str, module_name: str) -> str:
    """Creates an agent accessible function in Python. Function MUST include a docstring, and MUST include self as first argument.

    Args:
        function_name (str): The name of the function
        function_code_with_docstring (str): The code of the function, including the docstring
        module_name (str): The name of the module to create the function in

    Raises:
        Exception: Exception if the function already exists
        Exception: Exception if the function does not start with def function_name(self, ...
        Exception: Exception if the function does not include a docstring.
        Exception: Exception if the function is not in the functions directory.

    Returns:
        str: The result of the function creation attempt.
    """

    # setup
    if not os.path.exists(FUNCTIONS_DIR):
        os.makedirs(FUNCTIONS_DIR)

    # Make sure that if the function is already defined, overwrite = true and it is an agent defined function
    if function_name in self.functions_python.keys():
        raise Exception(f"Function {function_name} already exists. To overwrite, first delete with the delete_function function.")


    if not function_code_with_docstring.split("\n")[0].strip().startswith('def ' + function_name + '(self'):
        raise Exception("Function must start with def " + function_name + "(self, ...")

    if '"""' not in function_code_with_docstring and "'''" not in function_code_with_docstring:
        raise Exception("Function code must have a docstring.")

    file_path = os.path.join(FUNCTIONS_DIR, module_name + ".py")

    if os.path.exists(file_path):
        with open(file_path, "r") as f:
            previous_source = f.read()
    else:
        previous_source = ""

    # write new module:
    with open(os.path.join(file_path), "w") as f:
        f.write(previous_source + "\n\n" + function_code_with_docstring)

    self.reload_functions()
    return f"added function {function_name} to file {file_path}"

This worked reasonably well. Having the function return a string was helpful in letting the agent known what was changed.

Problems​

The agent had a difficult time consistently authoring functions that conformed to MemGPT's requirements - that it has a docstring, type hints, and only int, str, and bool return and argument types.

The iteration on basic requirements made it difficult for the agent to compose functions that worked together well. Often it would author placeholder functions that had names that sounded right, but didn't really do anything.

As the number of functions grew, so did the agent's tendency to get them confused. Functions also consume context window space, so making a large library of functions to any particular agent doesn't see promising.

Next steps​

This experiment points me back to a multi-agent approach in creating a broadly capable personal assistant. Having narrowly scoped helper agents available to the primary agent seems like the most promising route.

As I want to push a deployment of MemGPT to a server anyway, I am going to try to have a deployment with multiple agents that can talk to each other.

This is similar to Autogen's approach, though I think Autogen's groupchat management is too primitive to be useful.