Langchain Agents vs Langgraph

When I started working on DataLine, I used Langchain to do some basic large language model calls. It was still new, and the API was way more complicated than it is now (which is still not simple), but that's natural given how newly powerful and accessible large language models have become.

Over time, I've moved over to manual LLM calls and loops and strayed away from Langchain and Llamaindex and other libraries as I was waiting for their APIs to mature and stabilize a little.

As Langchain "agents" emerged, I started to see the limitations of my design which was haphazardly put together to get something working. Having the LLM do the planning was very useful, and giving it access to "tools" that I write was a very powerful design. Simply providing an LLM with a toolkit was something that could be done very easily, and at the same time easily matches complex implementations in performance since it could create it's own plan on how to use the tools and when to stop iterating on a solution.

It's as if it designed the software is designing a novel algorithm based on the user's input every time a request is made.

But using agents in production is a little harder. Why? Because they're so free and hard to control. Because the algorithm is so dynamic.

Most of the time consumers of software expect constraints in place, whether it is for security reasons, compliance reasons, our limited brains, etc. As I'm building DataLine, the main constraint I think of frequently is data security. As you put your data in, you might not want to expose it to the LLM. You might just want to share the absolute minimum necessary to build structured queries on top of your data and then execute and fetch the results only locally. Controlling an agent in this way is hard.

To do this using agents, we can't really modify or limit the "flow" of the agent easily since that is totally free-form. The easiest way is to set constraints on the toolkit the agent has access to. Maybe we can design a safer toolkit that ensures data security for example. But with this, it becomes harder to conditionally expose the unsafe functionality. I don't always care about data security. Maybe I'm working with publicly available data and want higher quality results now, maybe my public company data is suddenly made private. We have to stay flexible.

Enter Langgraph. Langgraph allows us to build an agent with a more controlled flow. We have conditional edges in the flow graph, we can stop before some steps and allow human intervention easily, and we can add in loops in the flow to still reap the benefits of agent-based software. It's generally a lot more flexible as it's basically exposing the foundations upon which the typical "agent" is built.

One of my favorite things about Langgraph is controlling the state which gets passed throughout the graph as it executes. This allows us to really control and see what's going on throughout the flow. I would bet that most production implementations would require this for decent UX. As we control the state and see it update as the software executes, we can communicate these state updates to the user. For example, in DataLine we communicate which tables were selected for the SQL query generation based on the user input. This can give the user a very quick way to check if the results will make sense or not. If not, the user can stop the generation and help correct the generation step. And this is just one use-case out of many more in this young open source project that's used by no one yet. Imagine how many of these interventions you'd have in a widely-shipped production application.

In short, agents are great for prototyping. You'll really get to see your toolkit in action, and make sure that it's good enough to take to the next step. But once you want ship software and start catering to your users' needs, you'll probably want a lot more control to impose a lot of constraints. Then you're better off moving to Langgraph. And the nice thing is that you'll get to re-use all the code you wrote for your basic agents prototype as the toolkits can stay exactly the same!

If you're interested in an advanced implementation using Langgraph (as of this moment), check out DataLine.

You can find me on Twitter every day @iamramiawar. For more content like this, be sure to subscribe below!