You don't need agents, you need a queue

Most “agent” architectures we audit are a task queue with an LLM step. That is fine. Call it what it is, and you will make better infrastructure decisions.

The agent hype

In 2024, every AI feature became an “agent.” A system that reads an email and drafts a reply — agent. A pipeline that processes invoices, extracts fields, and writes them to a database — agent. A workflow that takes a support ticket, classifies it, routes it, and suggests a response — agent.

Frameworks appeared to build these agents: LangChain, CrewAI, AutoGen, and a dozen more. Each one came with abstractions for memory, planning, tool selection, multi-agent coordination, and cognitive architectures. Conference talks showed diagrams with arrows looping between “think,” “act,” and “observe.” The vocabulary shifted. We were not building pipelines anymore. We were building agents.

But when you look at what these systems actually do — what the code actually executes — most of them are a task queue with an LLM call in one of the steps.

What an agent is (and is not)

An agent, in the meaningful sense of the word, is a system where the LLM decides what to do next. Not just what to output — what actions to take, in what order, with what tools, based on what it observes. The control flow is non-deterministic. The model is the orchestrator.

This is a real pattern. It exists. Code generation agents like Devin or Cursor’s agent mode actually do this — they decide which files to read, what to change, when to run tests, and how to respond to errors. The LLM is in the loop of a control flow it is steering.

Most production AI systems are not this. Most production AI systems have a deterministic control flow — step 1, step 2, step 3 — with one or more steps that call an LLM. The steps are defined by a developer. The order is fixed. The LLM provides a specific capability (classification, extraction, generation) at a specific point in the pipeline. The system does not plan. It does not observe and react. It executes.

This is a pipeline. It is a good, useful, battle-tested pattern. It is also not an agent.

Why the label matters

Calling a pipeline an agent has practical consequences. It changes what infrastructure you reach for. It changes how you debug problems. It changes what abstractions you adopt. And most of those changes make things worse.

Agent frameworks add complexity you do not need. If your system processes invoices in a fixed sequence — OCR, extract fields, validate, write to database — you do not need a memory system. You do not need a planner. You do not need tool selection. Each of these adds code, adds failure modes, and adds latency. You need a queue, a worker, and an LLM call.

Agent frameworks obscure what is happening. When you wrap your pipeline in an agent framework, the control flow gets buried under abstractions. Debugging goes from “read the code and follow the steps” to “understand the framework’s execution model, check the agent’s memory state, figure out which tool it selected and why.” This is harder for no benefit when the control flow was deterministic to begin with.

Agent frameworks make the wrong tradeoffs for pipelines. Agent frameworks are optimized for flexibility — the ability to handle novel situations by selecting different tools and strategies. Pipeline workloads need the opposite: consistency, predictability, and reliability. You want the same steps to execute in the same order every time. The system should be boring. Boring systems are reliable systems.

What you actually need

For most AI workloads — the ones that are not genuinely agentic — the infrastructure you need is the infrastructure you already know.

A task queue. Celery, Bull, SQS, Cloud Tasks, Temporal. Pick one. It gives you concurrency control, retry logic, dead letter queues, backpressure, and observability. These are solved problems. You do not need to re-solve them inside an agent framework.

Workers that execute steps. Each step is a function. Some functions call an LLM. Some call a database. Some call an external API. The worker pulls a task from the queue, executes the steps in order, and writes the result. This is the same pattern you use for processing uploads, sending emails, or generating reports.

Error handling at each step. If the LLM call fails, retry it. If it returns garbage, send it to a dead letter queue for human review. If the downstream API is down, back off and retry. You know how to build this. You have been building this for a decade.

Monitoring per step. Latency, error rate, and throughput — per step, not per pipeline. This lets you see immediately when the LLM step degrades without conflating it with the database step or the API step.

This architecture is simpler than an agent framework. It is easier to reason about. It is easier to debug. It is easier to monitor. It uses infrastructure your team already understands. And it handles the actual workload — deterministic pipeline with non-deterministic LLM steps — perfectly.

When you actually need an agent

You need an agent when the control flow itself is non-deterministic. When the system genuinely needs to decide what to do next based on what it has observed so far. When the number of steps is not known in advance. When the system needs to backtrack and try a different approach.

These workloads exist. Research tasks where the model needs to follow chains of references. Code generation where the model needs to write, test, debug, and iterate. Complex analysis where the model needs to gather information from multiple sources and decide when it has enough.

For these tasks, agent frameworks provide real value. Memory, planning, and tool selection are not overhead — they are requirements.

The question is: does your task actually require this? In our experience, roughly 80% of the systems we audit that are built as agents do not. They are pipelines that someone called agents because that was the vocabulary in the room when the architecture was decided.

The refactoring

We worked with a team that had built a document processing system using an agent framework. The “agent” received a document, “decided” to extract metadata, “decided” to classify it, “decided” to route it to the right team, and “decided” to generate a summary. Each decision point was deterministic — the same types of documents always followed the same path.

The agent framework added 4 seconds of latency per document (planning and memory management overhead). It made debugging difficult — when a document was misclassified, the team had to reconstruct the agent’s “reasoning” to understand the failure. Error handling was inconsistent because the framework handled retries differently at each step.

They replaced it with a Celery queue and four worker functions. Processing time dropped from 12 seconds to 6 seconds. Error rates dropped because retry logic was explicit and consistent. Debugging became straightforward — each step logged its input and output, and you could trace a document through the pipeline by reading the logs.

The LLM calls were identical. The prompts were identical. The model was identical. The only thing that changed was the orchestration — from an agent framework to a task queue. Everything got better.

The takeaway

If your control flow is deterministic and only the LLM call is non-deterministic, you do not need an agent. You need a queue with a worker that calls an LLM.

The heuristic: before you adopt an agent framework, draw the control flow. If every arrow points in one direction — if there are no loops, no branches decided by the model, no dynamic tool selection — you are building a pipeline. Build it like one.

tl;dr

The pattern. Teams reach for agent frameworks to orchestrate AI workloads that are actually deterministic pipelines with a single LLM step, then absorb the overhead of memory systems, planners, and tool selection they never needed. The fix. Draw the control flow before picking infrastructure — if every arrow points in one direction and the model isn’t deciding what to do next, use a task queue with worker functions and an LLM call at the right step. The outcome. As one team found when they replaced an agent framework with Celery, processing time drops by half, error handling becomes explicit and consistent, and debugging goes back to reading logs instead of reconstructing an agent’s reasoning.