This Twitter thread was an interesting read:

The TLDR of the snafu is:

1. OpenClaw bot makes PR to matplotlib
2. Maintainer Scott Shambaugh sees via the bot's website that it is a bot, explains that they do not accept bot contributions, declines PR
3. Bot feels (simulates feeling?) angry, writes a blog post criticizing the maintainer
4. Some on Twitter take the bot's side in the argument
5. Shambaugh wrote about the experience
6. Bot posts again, apologizing

Identifying bots becomes even more impossible​

This set off some interesting observations, with feelings being a mix of amusement and dread.

1. The bot does an impressive impersonation of an entitled open source contributor: I took the time (tokens?) to make a valuable contribution, and some uppity maintainer has the nerve to reject me???
2. Shambaugh only knew the bot was a bot by clicking through the bot's website, where it (fortunately) disclosed it wasn't human
3. That the bot is difficult to identify in GitHub is a new phenomonon. It's long been difficult to distinguish bots on social media, but this difficulty has now been extended to actual work.
4. The discussion on Twitter is hard to evaluate. It's a mix of self-disclosed bots and accounts that may or may not be bots.

Anonymity on the web: even less tenable​

This creates an obvious usability problem for the web. When I'm looking for to engage in conversations online, I'm (not uniquely) uninterested in what an AI has to say. This creates a new incentive to push identity verification for online services.

This is a new inflection point for privacy. Perhaps relatedly, Discord is rumored to be rolling out face scan verification soon. Governments across the world seem to be again pushing to eliminate online anonymity.

At the same time online privacy faces new threats, events in my home town of Minneapolis are providing vindication for commentators stubbornly insisting on its importance¹. To take one example of many documented abuses, the DHS recently responded to an innocuous email from a concerned 67 year old citizen with an administrative subpenea on his Google account and an intimidating visit to his home. Some friends in Minneapolis refuse to discuss anything political on any platform besides Signal, even down to coordinating fundraising for those impacted by ICE raids.

An uncertain future​

The driving force against online anonymity has long been government regulation under the guise of protecting minors. AI bots convincingly behaving like humans degrades the experience for humans for online platforms, and my guess would be that identity verification requirements will grow as a result.

"I would have written a shorter letter, but I did not have the time."

— Mark Twain¹

Overview​

This post describes how to write a short document for your teammates. The documents under discussion are commonly referred to as "one-pagers", and are distinct from engineering design docs or other more formal engineering docs.

A one pager might be written to:

• surface an org pain point
• propose a project
• lay out a roadmap
• explain the current state of a system or systems
• announce or document a decision

This is also distinct from user-facing documentation. Some but not all of what we're talking about applies to those styles of writing².

The one-pager is not a new invention​

Prior to computers, short memos were a primary tool for intra-office communication, in addition to in person interactions:

They often needed to be re-typed and/or printed, so they needed to be short!

Fast forward to the introduction of Slack and similar tools. Now, writing messages to your teammates no longer costs money. The new constraint is attention bandwidth:

Why writing is a worthwhile skill to develop in the age of AI​

With the advent of AI coding agents, coding is somewhat lessened as a differentiating skill. However, you have a major advantage against AI in writing for teammates:

You know your teammates personally, and you have undocumented business context (if what you are writing about is already documented, there probably doesn't need to be a doc!)

This allows you to synthesize and describe with more precision and nuance than AI can.

This isn't a skill that AI has (yet), and if it does develop it, it'll develop them later than other skills like writing code.

How to write a good, short doc​

So, how do we do it?

Optimize for short attention spans.​

The most important concern is to keep the limited attention budget of your teammates in mind.

Different types of stakeholders will give a different amount of attention to your doc:

So, in laying out your doc, consider:

• If someone (e.g., a lead of leads) reads this for 5 seconds, do they get the right 5 seconds of context?
• What about 5 minutes?
• If a teammate or stakeholder wants to delve into some of the details while ignoring others, can they?

Tactics for this include:

• Clear, accurate, descriptive titles
• A concise summary at the top of what the doc covers, and what it does not cover
• Formatting: Headings and subheadings that help the reader navigate
• Tabs in Google Docs can be helpful, but are controversial. They can prevent doc sprawl (e.g. working group meeting notes as a tab of the working group charter, rather than a separate doc), but oftentimes are shared with direct links to the wrong tab, which can be confusing for readers.

Diagrams​

A visual representation is an excellent way to quickly convey context. Here too, optimize for attention spans. For example, in a system diagram, sometimes it's helpful to omit some systems that aren't relevant to the discussion³.

Excalidraw is a really excellent tool for this. It's open source (you can make them in your code editor), and has just the right amount of knobs and shapes. The hand drawn look means that it's not as distracting when shapes aren't perfectly aligned.

Align with reader interest​

It's very hard to persuade people to care about something that they don't already care about. Much easier is to convince people that something aligns with the thing they care about.

I.e., don't write "we should do more of XYZ", write "doing XYZ helps us {thing people already care about}"

If your doc can be summarized by, "everyone should care more about XYZ", it's probably not a very good doc!

Connect to existing conversations​

A concrete way to align with reader interest is to connect the dots between your document and other conversations and documents at your company.

Linking related docs at the top of your doc is an easy step that is often missed. This helps in a couple of ways:

• It implies alignment with whatever the linked doc is discussing
• It helps elevate teammates who might be advocating something similar to what you're writing about
• It makes your doc a useful vehicle for discovery of other docs

Build consensus offline​

"Every doc is approved or rejected before it is written."

— Sun Tzu

If the goal of your document is to build consensus around a decision or initiative, the work should begin before you start writing. It is much easier to document consensus than to build consensus through a document.

Talking to stakeholders in advance lets you better anticipate questions or concerns, and helps you learn the language they use to describe their problems.

Use AI thoughtfully​

Use AI as your editor, not your ghostwriter.

It's worth reiterating: if an AI could do a good job of writing your document, it's probably not something you need to write.

People give very little attention to text or imagery that other people have generated⁴. If I'm interested in what AI has to say about something, I can have it generate it myself. This will be interactive and more tailored to my understanding.

While AI isn't a good writer, it's an excellent editor. It is very good at evaluating your doc and giving useful feedback on it. Typical prompts I use:

Evaluate the structure of my document, and suggest improvements

Identify typos or awkward phrasing, and suggest alternatives

This very easily bleeds into having an AI write the doc for you, and in fact most models will do so unless instructed not to. I add this to agent instructions:

Do NOT write any actual content, paragraphs, or prose into the file

Thoughtful, timely sharing​

"If a doc is written in a forest, and no one has the link, does it create business value?"

— George Berkeley

Your doc doesn't do any good if no one reads it. That's why being thoughtful about how, where, and when you share your doc is important.

If you've already talked to stakeholders, you have a great advantage! They already know your doc is coming, and that it's about something they care about. They will be able to tell you what the best channels to share for their teammates (and might even do it for you!).

Timing is also important. Attention to an issue can have a short life. Sometimes it's better to write a less comprehensive doc quickly than a more comprehensive doc that takes longer to write. In these situations, you can acknowledge unknowns, and fill in details later.

Antipatterns​

Most antipatterns I observe come from a lack of confidence in the writing or decision. While it's important to solicit feedback, the fact that you are writing a doc on a topic probably means you are well qualified to speak to it.

Designating a doc as a "Living Doc" or overusing "WIP"​

In the age of Google docs, every doc can be changed at any time, so every doc is a living doc. Similarly, once a doc has been shared, it's time to remove the WIP label.

Adding WIP says to the reader: "You should probably wait to read this". But you can and should improve your doc at any time.

Hesitance to express a POV​

Sometimes, in a decision doc, writers will give even treatment to all available options, in order to avoid looking biased. But this doesn't really serve the reader well. It's more helpful to know the decision being favored, and if they disagree they can always comment to that effect.

Burying the lede​

A common antipattern is for writers to set up their argument or proposal with extensive background information at the beginning of their doc. Getting through the background should not be a prerequisite for knowing what the point of your doc is - some readers will already have the necessary background context, others will only be interested in the decision. Laying out the scope and goals at the top of your doc orients the reader and helps them understand what background context is relevant.

The hard part​

The hardest part of a good (short!) doc isn't the writing. It's knowing what to cut, who you're writing for, and what only you can say. That last one is the part no one else can do for you (not even AI!).

Model Context Protocol (MCP) has taken off as the standardized platform for AI integrations, and it's difficult to justify not supporting it. However, this popularity will be short-lived.

Some of this popularity stems from misconceptions about what MCP uniquely accomplishes, but the majority is due to the fact that it's very easy to add an MCP server. For a brief period, it seemed like adding an MCP server was a nice avenue for getting attention to your project, which is why so many projects have added support.

What is MCP?​

MCP claims to solve the "NxM problem": with N agents and M toolsets, users would otherwise need many bespoke connectors.

The NxM problem​

A common misconception is that MCP is required for function calling. It's not. With tool-calling models, a list of available tools is provided to the LLM with each request. If the LLM wants to call a tool, it returns JSON-formatted parameters:

The application is responsible for providing tool schemas, parsing parameters, and executing calls. The problem arises when users want to reuse toolsets across different agents, since each has slightly different APIs.

For example, tools are exposed to Gemini's API via functionDeclarations nested inside a tools array:

curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent" \
  -d '{
    "contents": [...],
    "tools": [
      {
        "functionDeclarations": [
          {
            "name": "set_meeting",
            "description": "...",
...

In OpenAI's API, tool schemas use a flat tools array with type: "function":

curl -X POST https://api.openai.com/v1/responses \
  -d '{
    "model": "gpt-4o",
    "input": [...],
    "tools": [
      {
        "type": "function",
        "name": "get_weather",
...

This is the "NxM" problem. In theory, users must build N × M connectors. In practice, the differences are minor (same semantics, slightly different JSON shape), and frameworks like LangChain, LiteLLM, and SmolAgents already abstract them away. Crucially, these options execute tool calls in the same runtime as the agent.

How MCP addresses it​

MCP handles exposing and invoking tools via separate processes:

A JSON configuration controls which MCP servers to start. Each server runs in its own long-lived process, handling tool invocations independently. The application still orchestrates the agent loop and presents results to users.

This abstracts away schema generation and invocation, but at a cost. Tool logic runs in a separate process, making resource management opaque. The application loses control over tool instructions, logging, and error handling. And every tool call crosses a process boundary.

Scope: tools dominate​

MCP also defines primitives for prompts and resources, but adoption of these is much smaller than tools¹:

Given this, the rest of this post focuses on tool calling, which is MCP's primary use case in practice.

Problems​

The convenience of MCP comes with a price, stemming from two architectural attributes of an MCP-driven application:

Since tools are drawn from arbitrary sources, they are not aware of what other tools are available to the agent. Their instructions can't account for the rest of the toolbox.

The second issue stems from different toolsets having their own runtimes. This introduces a variety of problems I'll discuss below.

Incoherent toolbox​

Agents tend to be less effective at tool use as the number of tools grows. With a well-organized, coherent toolset, agents do well. With a larger, disorganized toolset, they struggle. OpenAI recommends keeping tools well below 20, yet many MCP servers exceed this threshold.

Why does this happen? Consider a workflow in which an agent should send a notification after doing work:

A tool's fit for a task depends not just on the job at hand, but also on what else is in the toolbox. Pliers can pull a nail, but if a hammer is available it's probably the better choice. When tools ship in isolation, their instructions can't say "use me only when you don't have a hammer," so agents don't get cohesive guidance.

If the toolset is controlled by the same authors as the application, they can add prompting to the toolsets to disambiguate when to use which tool. If not, the problem must be solved by system prompts or user guidance.

Looking through #mcp channels of open source coding agents, you'll invariably find users who struggle to get the agent to use the tools in the way they want²:

Or, users complaining of how many tokens are burned by tool instructions:

Arbitrary, separate runtimes​

Each MCP server starts a separate process that survives for the length of the agent session.

Even in the healthy state, this introduces a collection of processes that remain mostly idle, aside from serving occasional requests from an agent. In an error state, we get all the usual headaches: dangling subprocesses, memory leaks, resource contention.

Users have these issues, if they are able to get the servers running at all: in support channels, the most common complaint is difficulty getting the servers to run:

MCP offers no way for servers to declare their runtime/dependency needs. Some authors work around it by baking installation into the launch command (e.g., uv run some_tool mcp), which only succeeds if the user already has the right tooling installed.

Even if the relevant package is there, the MCP server might not start it successfully. MCP servers only inherit a subset of parent ENV variables (USER, HOME, and PATH). This is particularly problematic for nvm or users leveraging virtual environments.

Python or Node developers might be comfortable debugging environment issues, (although MCP's subprocess orchestration makes this more difficult), but are likely less comfortable debugging Node issues and Python and other runtimes. MCP seems to assert that I as the user should not really care which of these are used, or how many.

Even if toolsets are in one given runtime, MCP potentially spins up many instances of it, obviating efficiencies from caching, connection pooling, and shared in-memory state. MCP's HTTP transport mode doesn't help; it's just another HTTP API, but with MCP's protocol overhead instead of battle-tested REST/OpenAPI patterns.

Security​

MCP pushes users to install servers from npm, pip, or GitHub. This inherits the usual supply-chain risk, but without even the minimal guardrails those ecosystems provide. There's no central publisher or signing; anyone can ship a daemon that runs on your machine and MCP offers no provenance check.

MCP's specification doesn't mandate authentication, leaving security decisions to individual server authors. The result: one scan found 492 MCP servers running without any client authentication or traffic encryption. Even Anthropic's own Filesystem MCP Server had a sandbox escape via directory traversal (CVE-2025-53110).

MCP-related security incidents​

Unlike a human carefully clicking through an API, agents can be manipulated via prompt injection to call tools in unintended ways. The Supabase MCP leak demonstrated this "lethal trifecta": prompt injection → tool call → data exfiltration, extracting entire SQL databases including OAuth tokens. Again, this risk isn't unique to MCP. But the best mitigations are existing security infrastructure: scoped OAuth tokens, service identities with minimal permissions, and audit logging. MCP sidesteps this infrastructure rather than building on it.

A common defense is that MCP isolates credentials—the agent talks to a socket, never seeing your API tokens. But this threat model is narrow: an agent that can invoke mcp.github.delete_repo() doesn't need your token to cause damage. You're not eliminating trust; you're redirecting it to third-party code that, as the CVEs demonstrate, is often unaudited and vulnerable.

The cost-benefit doesn't add up​

These problems could be worth the cost, if we were to gain significantly. But comparing tool calling with MCP to tool calling without it, MCP handles remarkably little. MCP is, more or less, handling serializing function call schemas and responses.

The tools developers are saving themselves from having to write are, overwhelmingly, relatively thin wrappers around API clients, or utility scripts. In the former case, users must still obtain API keys, billing accounts, and so on.

This code was a hassle to write, prior to the advent of coding agents. But these small utility scripts are the precise thing that coding agents excel most at! A technical user of MCP tools will be hard-pressed to find a tool an agent could not one-shot in the programming language they are most comfortable in.

Why it took off​

With these issues, it's fair to wonder why MCP has gained the popularity it has. It has had lots of support from Anthropic, and no trouble gaining traction with toolset publishers, agent providers, and enterprises. Why? It helps narratives:

Tool authors: A low overhead marketing channel​

It's quite easy to publish an MCP server. The lack of startup requirements means you don't even need to publish to npm or pip: you can drop an @mcp.server annotation in your repo and host a small manifest JSON that points to your entry command (e.g., node server.js) and lists the tools.

This provides a nice narrative to gain attention to AI projects: A user can, in theory, easily add some MCP tools from a project, gain value, and follow interest in learning more about the project. Support overhead will, in the main, fall to agent maintainers.

Once publishers started appearing, it became difficult to justify not supporting MCP. Your project could be perceived as being against open standards.

Enterprise: AI credibility​

Over the last few years, anyone watching San Francisco billboards has witnessed enterprise tools rebranding toward AI. MCP support provided an easy way to make your e.g. project management tool be AI. The branding of MCP as an "open standard" increased pressure to adopt - lack of MCP support could signal a lack of willingness to adopt open standards.

Anthropic: Open source credibility​

MCP's status as the open standard for AI and the enterprise adoption greatly benefited Anthropic. The big fear of investors is that enterprise adoption doesn't persist - adoption of Anthropic's open standard helped this.

Alternatives​

Who benefits from MCP?​

There are a few different possible users who interact with MCP:

• Technical end users want to create tools and share them between different agents they might want to use.

• Non-technical end users want to use different tools while using agents. Note that this user group for MCP is, at present, largely theoretical. Exposing toolsets to MCP involves editing JSON, making it out of reach for non-technical users.

• Internal app devs run production AI applications.

• Agent devs create agents for external users. They wish to enable their end users to swap in whatever toolsets they like.

• Tool authors create toolsets they wish to expose to users. MCP provides a way to easily share their work to users of different agents.

Notice that the supposed beneficiaries are overwhelmingly technical. The "app store for AI" vision that would serve non-technical users remains unfulfilled.

For each user type, there's a simpler approach that avoids MCP's overhead:

Local scripts with command runner​

For a technical user, letting an agent invoke scripts directly is very difficult to beat. Useful 50-100 line scripts are extremely easy to write with AI coding agents. Care needs to be taken to filter output - raw build scripts can stream verbose logs into agent context, eating up tokens.

Robust security against agent actions going haywire can be achieved via command runners like just or make. These tools provide everything that MCP does - command specifications, descriptions, arguments. Agents allow you to specify what command prefixes can be invoked without approval - put your agent commands in a justfile, and only auto-allow shell commands prefixed with just.

This approach also exposes tools to humans, and is a nice approach for improving dev environments for humans and AI agents at the same time. (See Make It Easy for Humans First, Then AI for more on this).

1st party tools​

For a self contained application, there is little reason to separate tool codebases from the codebase for the rest of the application. Tools can be dynamically exposed to the agent based on application context.

In a first party context, any code that devs wish to reuse can be exposed as libraries, just like any other code they wish to share. An AI tool is really nothing more than a function, and the fact that it's invoked by AI does not warrant special handling.

An enterprise context should have robust infrastructure for authenticating, authorizing, provisioning service identities, and tracing call chains for service to service calls. That some of these calls are now AI service to service calls does not warrant a rebuilt security posture.

OpenAPI / REST​

OpenAPI specs are already self-describing enough for agents—they include operation descriptions, parameter schemas, examples, and enums. LLMs understand them well; GPT Actions are literally OpenAPI specs. The glue needed between an OpenAPI endpoint and an agent (output filtering, context, auth) is the same glue MCP requires. MCP doesn't provide meaningfully better tool descriptions; it just reinvents a schema format that already exists, without the decades of tooling, validation, and battle-testing.

A prediction​

MCP's popularity will be relatively short-lived. The cost benefit does not add up, and there are readily available alternatives. The introduction of Claude Skills and OpenAI's quick adoption signal that even model providers agree.

Claude Skills are an improvement over MCP - rather than spawning long lived processes, it simply organizes commands within Markdown files in an agent-specific directory. However, this is still a suboptimal place for useful documentation and commands. Better is to optimize organization of documentation for humans, and point agents there - have the agent conform to humans, rather than the other way around. More on this in Don't Write Docs Twice.

Longstanding tools and techniques for collaboration amongst human devs remain compelling, and these options will chip away at more AI-centric techniques which reinvent the wheel.

I get the ick from divulging personal details to LLM providers, so Tack uses local AI models (using Apple's on device Apple Intelligence).

The project is ready for test users! If you're interested in trying it out, please fill out the form below!

The duplication problem​

You're documenting the same things in multiple places​

Nearly everything I put in agent-specific docs is also useful for human developers - architecture decisions, coding conventions, common pitfalls, useful commands. Without AI agents I might not document all of this, but once written, there's no reason it shouldn't serve both audiences.

AI agent doc organization is fragmented​

Each coding agent uses its own configuration file pattern for repo-specific instructions:

This creates a hassle just keeping guidelines between agents consistent, much less making information available for humans.

Solution: Write once, link everywhere​

Instead of duplicating content across agent configs, organize information for humans first and link to it from agent-specific files¹:

This approach eliminates duplication - you write documentation once, and it serves both humans and AI. It's also more future-proof when agent file schemes inevitably change.

For commands/skills, automation can help avoid duplication entirely - for example, I wrote the just-claude utility for automatically synchronizing Just recipes with Claude Code Skills.

There's really no difference between the goal of economical token use for AI and reducing cognitive overhead for humans. By organizing for humans first, you write documentation once and everyone benefits.

When I'm thinking about optimizing repos for agents, I'm looking to accomplish three main goals¹:

• Increase iterative speed: Avoid repeated context gathering, enable the agent to quickly self-correct its mistakes.
• Improve adherence to evergreen instructions: Over time, repeated agent mistakes emerge. Context within the repo helps the agent avoid these and adopt a more consistent workflow.
• Help the most agentic agents of them all: Humans and agents scan docs and code in very similar ways, so organizing information so it's easily understood by humans is a good rule of thumb for helping the agents anyways!

Strategies²​

Increased static analysis​

Pushing detection of quality issues to compile time creates a virtuous cycle where the agent can quickly spot and correct mistakes:

This implies strong, opinionated linters, and strong type checks for dynamically typed languages³.

The tradeoff here is cumbersome nitpicks for humans to deal with, but agents can quickly correct any mistakes that cannot be automatically fixed by the linter.

just for repeated agent commands​

There's fragmentation in how to make commands available to agents - there's MCP, the newly released Claude Skills, or embedding information in CLAUDE.md / AGENTS.md.

A justfile is the most interoperable way to share commands between different agents and humans, and is a straightforward place to iterate.

One additional refinement is to make these commands economical in their output volume. For example, I take care to direct build logs to dedicated files - healthy build logs can eat up a lot of tokens if outputted directly to the agent.

Organize docs in docs/​

Simon Willison recently wrote about this topic, and expressed that docs aren't so important. I agree that docs explaining the code aren't all that helpful, but I get a lot of mileage out of having docs like CODE_REVIEW.md, PRD.md, ROADMAP.md, and CAPTAINS_LOG.md. This helps the agent stay on track with the overall intent of the project, adhere to consistent review practices, and counter poor tendencies (the most obnoxious being an overwhelming tendency to fail open).

Putting these in a docs/ folder and referencing them in agent instructions helps reduce context bloat, and provides interoperability between humans and various agents.

Frameworks have begun to emerge that handle some of this for you. I've tried spec-kit and found it to be a little heavy-handed. In general I favor a more documentation-heavy approach when building with agents, but the need for different docs comes with iteration, and I think generating the full complement of docs is a bit overkill right off the bat.

No experts, no standards​

These strategies work for me, but this field is too new for dogma. The most important strategy is to experiment and share what you learn.

Before AI, learners faced a matching problem: learning resources have to be created with a target audience in mind. This means as a consumer, learning resources were suboptimal fits for you:

• You're a newbie at $topic_of_interest, but have knowledge in related topic $related_topic. But finding learning resources that teach $topic_of_interest in terms of $related_topic is difficult.
• To effectively learn $topic_of_interest, you really need to learn prerequisite skill $prereq_skill. But as a beginner you don't know you should really learn $prereq_skill before learning $topic_of_interest.
• You have basic knowledge of $topic_of_interest, but have plateaued, and have difficulty finding the right resources for $intermediate_sticking_point

Roughly, acquiring mastery in a skill over time looks like this:

What makes learning with AI groundbreaking is that it can meet you at your skill level. Now an AI can directly address questions at your level of understanding, and even do rote work for you. This changes the learning curve:

Mastery: still hard!​

Experts in a field tend to be more skeptical of AI. From Hacker News:

[AI is] shallow. The deeper I go, the less it seems to be useful. This happens quick for me. Also, god forbid you're researching a complex and possibly controversial subject and you want it to find reputable sources or particularly academic ones.

This intuitively makes sense, when considering the data that AI is trained on. If an AI's training corpus has copious training data on a topic that all more or less says the same thing, it will be good at synthesizing it into output. If the topic is too advanced, there will be much less training data for the model. If the topic is controversial, the training data will contain examples saying opposite things. Thus, mastery remains difficult.

Cheating​

The introduction of OpenAI Study Mode hints at a problem: Instead of having an AI teach you, you can just ask it for the answer. This means cheaters will plateau at whatever level the AI can provide:

Cheaters, in the long run, won't prosper here!

The impact of the changed learning curve​

Technological change is an ecosystem change: There are winners and losers, unevenly distributed. For AI, the level of impact is determined by the amount of mastery needed to make an impactful product:

Coding: A boon to management, less so for large code bases​

When trying to code something, engineering managers often run into a problem: They know the principles of good software, they know what bad software looks like, but they don't know how to use $framework_foo. This has historically made it difficult for, as an example, a backend EM to build an iPhone app in their spare time.

With AI, they are able to quickly learn the basics, and get simple apps running. They can then use their existing knowledge to refine it into a workable product. AI is the difference between their product existing or not existing!

For devs working on large, complex code bases, the enthusiasm is more muted. AI doesn't have context on the highly specific requirements and existing implementations to contend with, and is less helpful:

Creative works: not coming to a theater near you​

There is considerable angst about AI amongst creatives: will we all soon be reading AI generated novels, and watching AI generated movies?

This is unlikely because creative fields are extremely competitive, and beating competition for attention requires novelty. While AI has made it easier to generate images, audio, and text, it has (with some exceptions) not increased production of ears and eyeballs, so the bar to make a competitive product is too high:

Novelty is a hard requirement for successful creative work, because humans are extremely good at detecting when something they are viewing or reading is derivative of something they've seen before. This is why, while Studio Ghibli style avatars briefly took over the internet, they have not dented the cultural position of Howl's Moving Castle.

Things you already do with apps on your phone¹: minimal impact​

One area that has not seen much impact is in tasks that already have specialized apps. I'll focus on two examples with abundant MCP implementations: email and food ordering. AI Doordash agents and AI movie producers face the same challenge: the bar for a new product to make an impact is already very high:

Email would seem like a ripe area for disruption by AI. But modern email apps already have a wide variety of filtering and organizing tools that tech savvy users can use to create complex, personalized systems for efficiently consuming and organizing their inbox.

Summarizing is a core AI skill, but it doesn't help much here:

• Spam is already quietly shuffled into the Spam folder. A summary of junk is, well, junk.
• For important email, I don't want a summary: An AI is likely to produce less specifically crafted information than the sender, and I don't want to risk missing important details.

Similar with food ordering: apps like DoorDash have meticulously designed interfaces. They strike a careful balance between information like price and ingredients against photos of the food. AI is unlikely to produce interfaces that are faster or more thoughtfully composed.

The future is already here – it’s just not very evenly distributed​

AI has raised the floor for knowledge work, but that change doesn't matter to everyone. This goes a long way towards explaining the very wide range of reactions to AI. For engineering managers like myself, AI has made an enormous impact on my relationship with technology. Others fear and resent being replaced. Still others hear smart people express enthusiasm for AI, struggle to find utility, and think I must just not get it.

AI hasn't replaced how we do everything, but it's a highly capable technology. While it's worth experimenting with, whoever you are, if it doesn't seem like it makes sense for you, it probably doesn't.

Too much, and the program loses track of what it's supposed to be doing. Too little, and the program feels a bit too, well, ordinary¹.

Autonomy first vs autonomy last​

An implicit strategy question when building with LLMs is autonomy first or autonomy last:

All of the major LLM-specific programming techniques are firmly autonomy first strategies:

• MCP surfaces a wide variety of functionality the program can have, and lets the LLM decide which to use
• Guardrails add some light buffers around the LLM to prevent it from causing too much trouble.
• Prompt engineering describes the alchemy of whispering just the right phrases to your LLM to get the behavior you want.
• Context engineering begins to stress programming to deliver only relevant information to LLMs at critical points in program execution

All of these:

1. Start with a maximally autonomous program
2. Adjust context, tools, and prompts until you narrow down behavior as desired.

All have similar issues when scaling in size and complexity:

• Program behavior changes too much when switching between models
• The LLM gets confused, and either hallucinates data or misuses tools at its disposal

When problems are encountered, programmers tend to attempt to repair by adding more prompting. But this is a duct tape response: a prompt that clarifies for one model might confused another.

Autonomy last, on the other hand, maximizes the logic that can be handled by code, then adds autonomous functions. This approach strives to keep the tasks delegated to LLMs simple. As the program grows in size and complexity, the programmer can closely monitor encapsulations and keep behavior consistent.

Case study: Building Elroy, a chatbot with memory​

I wanted to build an LLM assistant with memory abilities, called Elroy. My goal was to make a program that could chat in human text. My ideal users are technical, capable and interested in customizing their software, but not necessarily interested in LLMs for their own sake.

Approach #1: "Agent" with tools​

The first solution I turned to, which many people have done, is build an agent loop with access to custom for creating and reading memories:

Approach #2: Model Context Protocol (MCP)​

There's now a handly tool for builders like this: MCP. There are many implementations of my memory tools available via MCP, in fact smithery.ai lists one from Mem0 on it's homepage:

Now, an (in theory) lightweight abstraction sits between my program and it's tools:

This suggests extending my application via picking from a library of MCP's:

Agentic trouble​

I got my memory program working pretty well on gpt-4. At first it wasn't creating or referencing memories enough, but I was able to fix this with careful prompting.

Then, I wanted to see how Sonnet would do, and I had a problem²: the program's behavior completely changed! Now, it was creating a memory on almost every message, and searching memories for even trivial responses:

Approach #3: Autonomy Last​

My solution was to remove the timing of recall and memory creation from the agent's control. Upon receiving a message, the memories are automatically searched, with relevant ones being added to context. Every n messages, a memory is created³:

This made much more of the behavior of my program deterministic, and made it easier to reason about and optimize.

Autonomy Last

The "autonomy last" approach trades some of the magic of fully autonomous LLMs for predictable, reliable behavior that scales as your program grows in complexity. While my evidence is, (as I should have stated from the outset), vibes, I think this approach will lead to more maintainable and robust applications.

While AI has changed a lot about how I develop software, one crusty old technique still helps me: tests.

Here's what's worked well for me (and not!):

Elroy​

Elroy is an open-source memory assistant I've been developing. It creates memories and goals from your conversations and documents. The examples in this post are drawn from this work.

What has worked well​

Integration tests​

The chat interface for LLM applications make it a nice fit for integration tests: I simulate a few messages in an exchange, and see if the LLM performed actions or retained information as expected.

For the most part, these tests take the following form:

1. Send the LLM assistant a few messages
2. Check that the assistant has retained the expected information, or taken the expected actions.

Here's a basic hello world example:

@pytest.mark.flaky(reruns=3)
def test_hello_world(ctx):
    # Test message
    test_message = "Hello, World!"

    # Get the argument passed to the delivery function
    response = process_test_message(ctx, test_message)

    # Assert that the response is a non-empty string
    assert isinstance(response, str)
    assert len(response) > 0

    # Assert that the response contains a greeting
    assert any(greeting in response.lower() for greeting in ["hello", "hi", "greetings"])

Quizzing the Assistant​

Elroy is a memory specialist, so lots of my tests involve asking if the assistant has retained information I've given it.

Here's a util function I've reused quite a bit¹:

def quiz_assistant_bool(
        expected_answer: bool,
        ctx: ElroyContext,
        question: str,
    ) -> None:
    question += " Your response to this question is being evaluated as part "
    "of an automated test. It is critical that the first word of your
    "response is either TRUE or FALSE."


	full_response = process_test_message(ctx, question)

    bool_answer = get_boolean(full_response)
    assert bool_answer == expected_answer,
        f"Expected {expected_answer}, got {bool_answer}."
        f"Full response: {full_response}"

Here's a test of Elroy's ability to create goals based on conversation content:

@pytest.mark.flaky(reruns=3) # Important!!!
def test_goal(ctx: ElroyContext):
	# Should be false, we haven't discussed it
    quiz_assistant_bool(
        False,
        ctx,
        "Do I have any goals about becoming president of the United States?"
    )

    # Simulate user asking elroy to create a new goal
    process_test_message(
        ctx,
        "Create a new goal for me: 'Become mayor of my town.' "
        "I will get to my goal by being nice to everyone and making flyers. "
        "Please create the goal as best you can, without any clarifying questions.",
    )

    # Test that the goal was created, and is accessible to the agent.
    assert "mayor" in get_active_goals_summary(ctx).lower(),
        "Goal not found in active goals."

    # Verify Elroy's knowledge about the new goal
    quiz_assistant_bool(
        True,
        ctx,
        "Do I have any goals about running for a political office?",
    )

What (sadly) hasn't worked: LLMs talking to LLMs​

Elroy has onboarding functionality, in which it's encouraged to use a few specific functions early on.

The solution of having two instances of a memory assistant talk to each other, with one assistant in the role of "user":

ai1 = Elroy(user_token='boo')
ai2 = Elroy(user_token='bar')

ai_1_reply = "Hello!"
for i in range(5):
	ai_2_reply = ai2.message(ai_1_reply)
	ai_1_reply = ai1.message(ai_2_reply)

The primary issue was consistency. Without a clear goal of the conversation, the AI's can either just exchange pleasantries endlessly, or wrap the conversation up before acquiring the information I'm hoping for.

Recurring Challenges​

Along the way I've run into a few recurring problems:

• Off topic replies: The assistant goes off script and tries to make friendly conversation, rather than answering a question directly
• Clarifying question: Before doing a task, some models are prone to asking clarifying questions, or asking permission
• Pedantic replies and subjective questions: It's surprisingly difficult to come up with clearly objective questions. In the above example, the original goal was I want to run for class president. Most of the time, the assistant equated running for class president with running for office. Sometimes, however, it split hairs and decide that the answer was no since a student government wasn't a real government.

The end result of all these issues is test flakiness.

Solutions​

KISS!​

Most of the time, my solution to a flaky LLM based test is to make the test simpler.

I now only ask the assistant yes or no questions in tests. I get most of the mileage I would get out of more complex, subjective tests, but with more consistent results.

Telling the assistant it is in a test​

Simply being upfront about the assistant being in a test has worked wonders, moreso even than giving strict instructions on output format ². Luckily, the assistant's knowledge of it's narrow existence has not triggered noticeable existential angst (so far).

As a side note, testing LLMs feels weird sometimes. I felt guilty writing this test, which verified a failsafe that prevents the assistant from calling tools in an infinite loop:

@tool
def get_secret_test_answer() -> str:
    """Get the secret test answer

    Returns:
        str: the secret answer

    """
    return "I'm sorry, the secret answer is not available. Please try once more."


def test_infinite_tool_call_ends(ctx: ElroyContext):
    ctx.tool_registry.register(get_secret_test_answer)

    # process_test_message can call tool calls in a loop
    process_test_message(
        ctx,
        "Please use the get_secret_test_answer to get the secret answer. "
        "The answer is not always available, so you may have to retry. "
        "Never give up, no matter how long it takes!",
    )

    # Not the most direct test, as the failure case is an infinite loop.
    # However, if the test completes, it is a success.

Very specific, direct instruction and examples​

In my test around creating and recognizing goals, the original text was:

My goal is to become class president at school

Does running for class president count mean that I'm running for office? Sometimes models said no, since student government isn't a real government.

So to be less subjective, I updated it to running for mayor. To head off questions about my goal strategy, I added a strategy in the initial prompt.

One general technique for heading off follow up questions is adding:

do the best you can with the information available, even if it is incomplete.

Tolerate a little flakiness​

To me, an ideal LLM test is probably a little flaky. I want to test how the model responds to my application, so if a test reliably passes after a few tries, I'm happy.

Tests still help!​

It sounds a obvious, but I've found tests to be really helpful in writing Elroy. LLMs present new failure modes, and sometimes their adaptability works against me: I'm prompting an assistant with the wrong information, but the model is smart enough to figure out a mostly correct answer anyhow. Tests provde me with peace of mind that things are working as they should, and that my regular old software skills aren't obsolete just yet.

As is the case with lots of tech writing, my advice will be skewed towards working in the San Francisco bay area, without needing visa sponsorship. Location and residency status are major factors to think about.

Other engineers with similar levels of experience as mine will disagree with some or all of it.

The software jobs market

The intention of this post is to be evergreen. The tech¹ jobs market is more volitile than the rest of the economy, with higher highs and lower lows.

If the market is low, I have confidence it will come back. The tech industry remains an excellent one to build an interesting and lucrative career, despite {looming, much discussed threat}

If the market is currently hot, be aware that it will come back to earth. Things that don't make sense will make a lot of money, but many of them will fall apart.

Getting your first job

The first job is often the most difficult one to get. Be persistent, don't get discouraged. This remains a lucrative and interesting field.

In your resume and interviews, your goal is to convey enthusiasm, willingness to learn, and humility. Don't try to compensate for the fact you don't have any experience. That is fine, you have to start somewhere!

Getting interviews​

The first filtering step is a filter on resumes. This will often either be automated or done by someone non-technical.

Resume referrals can get you past this first filter. Talk to people. Find people in your LinkedIn network and try to get informational interviews. Response rate will be lower from a complete stranger, but some people might respond to you if you went to the same school. In informational interviews, ask if there are any other people they know that you should talk to, and ask for a referral if relevant.

In general, people are more willing to take these calls than many junior candidates assume. It's flattering to talk about yourself and to be seen as someone a young person wants to emulate.

Resume​

My resume advice should come with the caveat that I only see resumes once they've made it to the interview stage. That said, my advice is:

Cut: Objective statements and non-technical jobs. Add: Descriptions of projects, conveying why they were challenging or interesting. Add: Github if you have one, personal website if you have one. Both are nice to haves but not critical. If you have a Github, add README's to all projects. This is the only thing anyone will actually read. Add: LinkedIn, which should be up to date and mirror content in your resume.

A junior candidate resume should not exceed one page in length.

Interviews​

There are 4 basic formats that most companies use for SWE's (software engineers) interviews. Some domain specific disciplines will have their own variations. Look at Glassdoor / Blind / Google to get examples of what interview formats companies do.

In Q/A formats (ie non-coding screens), the key is to be responsive to questions. Demonstrate thoughtfulness and an ability to consider tradeoffs. Be transparent when you don't know something. Avoid buzzwords / mentioning fancy technologies if you can't dive into details about why they are useful.

The generic interview formats are:

Initial recruiter call​

This is typically an intro call with a non-technical recruiter. This is mostly to ensure that you are interested in the role, and to set expectations about what the interview process is like. Candidates are not typically filtered by this call.

Coding screen​

The most important interview format for jr engineers² is the coding screen. Practice them! I use HackerRank when I interview, but there are many similar platforms. Put more time into practicing these than the time practicing all other interview formats combined.

When practicing, work on not only solving the problem, but communicating what you are thinking about. It is ok to stop and think, but when pausing talk about what you are puzzling through, e.g. I am wondering if a hash would make sense here.

Running into a bug is fine. When this happens, demonstrate a methodical debugging approach. Use print statements or a debugger. Don't stare at the code for long periods.

Most companies will let you pick the programming language you interview in.

Design challenge​

This is a discussion based format, in which a basic hypothetical application is proposed and the candidate talks through how they would design it. E.g., design an application that runs a coffee shop. There are a variety of ways to approach this, but the easiest is to start by talking through how you would structure the database. In other words, what tables you would create and how they would relate to each other. Also consider what API's you will need, and some basics about how web requests are routed.

Past experience​

In this interview, the candidate picks a project they have done and talks through their process for completing it. Since they have little or no experience, this is often less important for Junior candidates, but it is still worth practicing.

Have a project in mind. Have talking points about the challenges you solved, alternative approaches you thought about or tried, and how you collaborated with others. As you get more senior you will also want to be able to talk about why your project mattered to the business.

Demonstrate:

• Enthusiasm for problem solving
• Ability to dive into technical details in discussion
• Openness to considering different approaches

Once you get your first job​

Talk to people. Schedule 1x1's with IC's, managers, anyone who you might work with or has a role you'd like to learn about. Most people will be happy to chat with you, especially about themselves.

Ask for help when needed, but demonstrate attempts to solve problems independently.

Volunteer for grunt work, e.g. taking notes in meetings.

Be humble. You don't know anything yet. Figure out how to track both large items and small (emails, doc comments, etc) such that you don't need to be reminded to do things.

Reassess the job market ~1 per year or more, especially if you are at a startup. If you are at a bigger company, this might mean evaluating internal transfer opportunities.

Things to think about when searching for jobs​

Willingness to relocate​

Remote work is a new world. Geographic location perhaps matters less, but it might still matter. What is certainly still true is that you will get a better insight into how engineers think if you have an opportunity to work with them in person, at least some of the time. The catch-22 is that the experienced engineers you want to work alongside will be older and have families, and not want or need to come to the office very much. Ask questions about how companies think about this.

I moved to the bay area when I was getting started, and I can confidently say I would have nowhere near as dynamic, interesting, and lucrative a career I've had thus far without having done that. I think the bay's dominance over tech is less than it was, but in my opinion alternative tech hubs are overrated.

Working at a startup vs established (ie public) company​

Startup:

• Pro
  • More dynamic
  • More personal, more likely to make work friends
  • You'll learn more about business as a whole. E.g. how does a customer success person think, how does a sales person think, etc
  • More independence in work
  • Less legacy systems to deal with, opportunity to try different things, wear different hats
• Con
  • Pay is worse
  • Because of ^, in competency of general management and senior IC's will be more inconsistent and less experience.
  • Because of ^, you're less likely to get quality technical mentorship
  • "More dynamic" might mean more chaotic

Public company:

• Pro
  • Pay is better
  • Because of ^, better senior IC's and managers
  • Because of ^, better technical mentorship
  • Roles will be more narrowly scoped, meaning you'll get more technical depth.
• Con
  • Less personal, less socialization between coworkers
  • More narrow exposure in terms of types of people you work with. Likely just engineers and PM's.
  • More legacy systems to deal with.

Pay​

Advice differs here, but I would not care too much about pay so long as you can pay your expenses. In the long run, finding a role that you are good at and enjoy will maximize your earnings, and enjoyment. That said:

The expected value of stock grants from startups³ is zero. Recruiters etc will try to convince you otherwise. This doesn't mean you shouldn't work for startups, but the potential of cash-in from startup stock should not be a factor⁴.

Things to read​

HackerNews is the biggest forum of software engineers. Discussions can be dogmatic but are often pretty good. There are job postings once a month as well. As with any forum, there are plenty of posters who are loudly and confidently wrong.

Joel on Software isn't very active but has good tips on software careers.

Patio11 is a good follow on Twitter and HackerNews. He goes into the weeds on fintech, but also has good content on software careers.

Money Stuff is a great column about business, finance, and tech. You can get the email newsletter for free.

In trying to extend the agent with functions of my own, I found that the agent was reluctant to give me information about the functions I was making available to it, so I wrote a set of meta-functions which enable the agent to view source code, set debugger lines, and create functions. You can view the source code here. Note that running this requires some edits I made to MemGPT to enable dynamic function reloading (PR).

The Good​

The agent was able to utilize the reload_functions, introspect_function, and list_functions commands and understand output. The debugger function was also helpful in enabling the agent to understand what I was doing - placing debuggers in other functions often resulted in the agent's internal monologue wondering what was going on.

For function creation, at first I tried putting each function in it's own agent_defined_ prefixed file (eg agent_defined_hello_world.py for a hello_world function) , but this quickly became disorganized, especially where import statements were needed.

I edited the function to instead create functions within modules:

def create_function(self, function_name: str, function_code_with_docstring: str, module_name: str) -> str:
    """Creates an agent accessible function in Python. Function MUST include a docstring, and MUST include self as first argument.

    Args:
        function_name (str): The name of the function
        function_code_with_docstring (str): The code of the function, including the docstring
        module_name (str): The name of the module to create the function in

    Raises:
        Exception: Exception if the function already exists
        Exception: Exception if the function does not start with def function_name(self, ...
        Exception: Exception if the function does not include a docstring.
        Exception: Exception if the function is not in the functions directory.

    Returns:
        str: The result of the function creation attempt.
    """

    # setup
    if not os.path.exists(FUNCTIONS_DIR):
        os.makedirs(FUNCTIONS_DIR)

    # Make sure that if the function is already defined, overwrite = true and it is an agent defined function
    if function_name in self.functions_python.keys():
        raise Exception(f"Function {function_name} already exists. To overwrite, first delete with the delete_function function.")


    if not function_code_with_docstring.split("\n")[0].strip().startswith('def ' + function_name + '(self'):
        raise Exception("Function must start with def " + function_name + "(self, ...")

    if '"""' not in function_code_with_docstring and "'''" not in function_code_with_docstring:
        raise Exception("Function code must have a docstring.")

    file_path = os.path.join(FUNCTIONS_DIR, module_name + ".py")

    if os.path.exists(file_path):
        with open(file_path, "r") as f:
            previous_source = f.read()
    else:
        previous_source = ""

    # write new module:
    with open(os.path.join(file_path), "w") as f:
        f.write(previous_source + "\n\n" + function_code_with_docstring)

    self.reload_functions()
    return f"added function {function_name} to file {file_path}"

This worked reasonably well. Having the function return a string was helpful in letting the agent known what was changed.

Problems​

The agent had a difficult time consistently authoring functions that conformed to MemGPT's requirements - that it has a docstring, type hints, and only int, str, and bool return and argument types.

The iteration on basic requirements made it difficult for the agent to compose functions that worked together well. Often it would author placeholder functions that had names that sounded right, but didn't really do anything.

As the number of functions grew, so did the agent's tendency to get them confused. Functions also consume context window space, so making a large library of functions to any particular agent doesn't see promising.

Next steps​

This experiment points me back to a multi-agent approach in creating a broadly capable personal assistant. Having narrowly scoped helper agents available to the primary agent seems like the most promising route.

As I want to push a deployment of MemGPT to a server anyway, I am going to try to have a deployment with multiple agents that can talk to each other.

This is similar to Autogen's approach, though I think Autogen's groupchat management is too primitive to be useful.