Start Right with Deepchecks: Agent Evaluation Out-of-the-Box

A Quick Guide to Deepchecks Agent Evaluation

Evaluating LLM-based applications is anything but easy, and the challenges mount as the app’s complexity increases. Gauging the quality of bare LLMs is a non-trivial task. Add to it a RAG component, and you’ll need to think about groundedness and ranking quality on top of your usual text generation metrics. Now, consider multi-step agentic workflows. There is so much that can go wrong: planning efficiency, tool coverage, or inappropriate tool usage. And naturally, due to agents’ non-deterministic nature, these failures will only happen on some occasions, making debugging difficult. This leads to developers shipping agents without knowing their blind spots. This article explores how to start agent evaluation the right way: using Deepchecks to get quick and actionable metrics out of the box.

We will work with a simple LangGraph agent designed for travel planning. The agent’s job is to compose a complete travel plan for a requested city, covering flights, accommodation, attractions, weather, restaurants, transportation, and budget. In its system prompt, we specifically ask it to ensure it addresses all these aspects of the trip.

Travel Planning Agent design diagram. Image by the author.

We give the agent access to some dummy tools dedicated to different areas of research, such as a get_weather function returning hardcoded weather per city. You can check out the full code on GitHub.

Running the agent locally reveals what seems to be a sound travel plan.

Excerpt from our agent’s execution output. Image by the author.

At first glance, this agent seems to work fine. But does it actually deliver on its promises? Let’s find out.

In order to capture traces from LangGraph interactions and send them to Deepchecks for evaluation, we need to set up the instrumentation and name our agent correctly for Deepchecks to pick it up.

Deepchecks instrumentation is just a few lines of code needed to call register_dc_exporter, passing the API key. Notice that no changes to the agent logic are necessary.

Instrumenting LangGraph with Deepchecks takes just a few lines of code. Image by the author.

As far as the naming is concerned, we just need to make sure that the name argument passed to LangGraph’s create_agent function starts or ends with “agent_graph”.

Additionally, in order to get even richer agent-level evals from Deepchecks, it is a best practice to encapsulate the agent as a subgraph (as opposed to implementing the agent inline inside the parent graph). Check out this article’s full source code or the Deepchecks docs for more details and examples.

With this setup, we can now run our agent and view results. Let’s do that next.

We have run the agent five times, once for each of five different cities. We can see this reflected in the five sessions appearing in Deepcheck dashboard’s landing page. It’s immediately clear from the summary chart on the right-hand side that something’s wrong: all five sessions have been evaluated negatively.

Deepchecks dashboard’s landing page reveals issues in all agent sessions.

Let’s dive deeper to figure out what happened. We will focus on the Agent interaction type, since its 0% score makes it the most likely to reveal bugs in agent design.

Plan Efficiency

The first metric under the agent tab is plan efficiency. It captures how well the agent’s execution aligns with its initial plan.

Plan Efficiency scores 4.2 out of 5, where higher is better. Screenshot from the Deepchecks app by the author.

It scores 4.2 out of 5 for us on average, where higher is better. This is not too bad, but let’s take a more detailed look at the metric’s distribution.

Low plan efficiency hints towards hallucinated outputs. Screenshot from the Deepchecks app by the author.

There are many fives, but there’s also an interaction scoring only a two. If we single it out, we receive the following insight from Deepchecks:

“Flight and hotel details (names, schedules, prices) are presented without tool evidence; these are likely fabricated. Weather, attractions, and restaurants are supported by tool outputs. Budget estimates for flights/hotels lack evidence. User receives a plausible plan, but key facts are unsupported.”

It seems that the agent delivers partially hallucinated information. Does it decide not to use the tools, or are they somehow not available to it? Let’s dig deeper and focus on the tool coverage next.

Tool Coverage

Tool coverage measures how well the tools address the overall goal. As you can see from the previous dashboard screenshots, it only scores 2 out of 5 on average, where higher is better. A quick look at the metric’s distribution shows that all of the sessions only score a two – there’s definitely some systemic error going on with our agentic workflow.

Low tool coverage across the board indicates issues with tool accessibility. Screenshot from the Deepchecks app by the author.

To analyze it in greater detail, we can use the “Analyze Property Failures” feature. It produces the following report on our agent’s tool coverage:

Tool Coverage failure analysis reveals a consistent lack of certain tool usage. Screenshot from the Deepchecks app by the author.

The analysis reveals that the agent consistently neglects some aspects of the trip planning task, focusing solely on weather, attractions, and restaurants, as though it did not have access to the remaining tools it should use to provide flights, hotels, and so on. Let’s inspect how we provide the agent with tool access.

Sure enough, we did not give the agent all the tools it needs to fulfill its task! Note that in spite of this, the Tool interaction type got a 100% score (you can see it in the first screenshot above, showing the dashboard’s landing page). This is because the tools that were called, were called correctly, and their responses fulfilled the intended purpose. What went wrong was that some tools were unavailable.

The process of diagnosing issues with our agent can be measured in minutes, thanks to the insights from the Deepchecks dashboards. We started by noticing negative overall evaluations for every single session, immediately signalling that something’s fishy.

Analyzing an agent session with a low plan efficiency score surfaced a lack of tool support for some of the output, indicating hallucinations. This pointed us towards the agent either choosing not to use the tools correctly or not being able to use them.

A look at tool coverage, very low across the board, was our next step. The metric’s failure analysis suggested that the agent ignored certain aspects of the trip planning process, as though it did not have access to the right tools for the job.

All of this led us to discover that we, in fact, only provided our agent with some of the tools it needs. The solution to this can be two-fold. We could either equip our agent with the missing tools, including flight search, hotel booking, and so on. Or, perhaps, we don’t actually want the agent to do those things for us. Maybe the food, weather, and attraction suggestions are all we are after. In this case, we should adapt the prompt and remove the requests for flights and hotels to align the agent’s task with its capabilities.

The point of this short demonstration was to prove that agent evaluation doesn’t have to be a black box. Deepchecks gives us specific, actionable metrics fast. What normally would have taken lots of manual debugging involving running the agent repeatedly and inspecting outputs, took just a few clicks in the Deepchecks dashboard. We went from “the agent seems fine” to “we forgot to wire up half the tools” in a few minutes.

Most notably, though, all it took was a few lines of instrumentation setup code. Without any extra configuration, we got visibility into plan efficiency, tool coverage, and other agent-specific metrics, allowing us to set up agent evaluation in the right way from the get-go. In this case, we only encountered a tool access issue, but Deepchecks would also have surfaced other failure modes if they had occurred.

To take the evaluation pipeline further up a notch, Deepchecks offers features to add custom properties tailored to specific use cases, configure auto-annotation pipelines, or map spans to different interaction types. However, even without these more involved configurations, what we get out of the box is often enough to diagnose issues in agentic applications and fix them.