AI Observability Lessons From the Database

By Corey Beck, VP of Service Delivery at DataStrike

‍

For many years, database monitoring was a low priority in IT budgets. Companies started funding it once the database became too important to the business to risk an outage, and after that it stayed a standard expense. AI observability is going through the same shift, only much faster.

‍

Much of the work behind AI observability will be familiar to anyone who has managed databases or cloud platforms. Teams check a system's actual output against what was expected, and a dedicated team member is responsible for any gap. The main change with AI is what's being monitored, since an agent operates very differently from a server or a database.

‍

How agents differ from traditional systems

‍

A language model on its own generates responses to prompts. An agent built on top of the model can take actions, using tokens and compute and connecting to other systems as it completes a task. Because an agent acts without a person approving each step, monitoring it becomes a governance responsibility as well as a technical one. Most observability tools in use today were built to report whether servers and applications are healthy, and that information says little about whether an agent made a good decision.

‍

Incident response for AI

‍

Many of the costliest AI incidents are likely to happen while infrastructure monitoring shows no problems. Servers can be running normally and the database responding while an agent spends hundreds of thousands of dollars or sends messages to the wrong customers. The problem may first be reported by someone outside of IT.

‍

At the data tier, many likely AI problems are routine operational issues. An agent can write to a table faster than the application was designed to handle, and an integer key column can run out of available values within an hour. The agent is working as intended, but the volume exceeds what the system was built for, and the application stops accepting new writes. Standard incident runbooks rarely cover this scenario.

‍

For that reason, AI systems need their own incident-response plan. AI failures can range from excessive resource use, as in this example, to a model working around the limits placed on it. Most organizations have not yet built a plan that covers both.

‍

Monitoring usage and cost

‍

The two most important things to monitor are usage, meaning the tokens and compute an agent consumes, and the quality of its output. Once a system is in production, teams should also track model drift, availability, cost per request, and whether guardrails are working.

‍

Usage is often where organizations find unexpected costs. AI spending can increase tenfold from one month to the next when no one is tracking consumption in real time. Early enterprise AI projects focused on whether the technology would work. Now finance teams are asking who is accountable for the cost.

‍

The return on AI observability can be measured in the same terms. Unmonitored AI usage is often expensive and difficult to forecast, and observability helps organizations avoid those costs and the risks that come with them. Without monitoring, the costs often go unnoticed until an invoice arrives, or an incident review begins.

‍

AI cost management and observability are becoming connected. Both rely on the same token and compute data. As companies start budgeting for AI the way they budget for other infrastructure, both functions end up working from the same numbers, even if they use different tools.

‍

Agent ownership and identity

‍

Agent ownership is a less obvious risk because an employee may set up an agent to automate part of their work and then leave the company. If there is no handoff, the agent can keep running and using resources, and the remaining staff may not know if it exists or whether it is still needed.

‍

Agent identity is a related issue, since agents usually need their own logins and permissions to do their work, and many companies have not yet set policies for managing them. Some security teams will not create accounts for non-human users at all. That approach limits risk, but it also makes it hard to use agents at scale.

‍

Responsibility for AI observability usually falls into two groups. Day-to-day governance fits best with infrastructure and operations staff, who can confirm that an agent uses approved data sources and stays within its limits. Security teams should handle outside attacks and the management of agent credentials. Some cases fall between the two, such as an agent that drifts from its intended behavior without any malicious cause, and one of the groups needs to be responsible for catching it.

‍

Reviewing agent decisions

‍

One of the most useful capabilities in an AI observability tool is the ability to review a decision after it happens. Database administrators have relied on transaction logs for decades because they show what changed and in what order. Agents need a similar record of the information they had and the reasons for the actions they took. Usage tracking and guardrail controls are still important. Tools that can also explain why an outcome occurred give IT teams the information they need to investigate problems.

‍

Many current tools use one AI model to evaluate another. Each additional layer of review raises the question of who checks it, and the industry has not settled on an answer yet.

‍

Choosing tools and standards

‍

Few companies are running AI observability in production today. Most are still evaluating vendors and reviewing product demos. A few early adopters are moving ahead, while other organizations are waiting until the tools and practices are more established.

‍

Dedicated AI evaluation platforms are available and effective. They are designed for companies that already have platform engineers and machine learning specialists on staff. Most mid-market IT teams do not have those specialists, so buying a platform does not answer who will operate it or who is responsible when results are wrong. For these teams, a managed service that handles monitoring and evaluation on an ongoing basis may be easier to sustain than building and staffing a platform internally.

‍

OpenTelemetry is likely to remain the main standard for collecting telemetry and connecting traces across LLM and agent workloads. Newer specifications mostly add to it, and much of the development over the next few years will probably focus on the semantic layer, where the industry still needs to agree on what AI telemetry should capture and how to measure agent behavior against business results.

‍

Planning for AI observability

‍

Regulation and company policies are struggling to keep pace with AI development, and the gap is getting wider. As agents become more autonomous, IT teams should expect new incidents like behavior that monitoring tools were not designed to detect.

‍

AI observability should measure whether agents are making accurate decisions and delivering business results, in addition to tracking infrastructure metrics. Teams should put this monitoring in place before an incident occurs.

‍

For most mid-market organizations, the data tier is a practical place to start. Agents read from and write to databases as they work, so a modernized, well-monitored data environment gives organizations early visibility into how much their agents use and whether their output is accurate. DataStrike has managed this layer for clients for 28 years and recommends it as the starting point. To learn how DataStrike can help manage your data infrastructure, contact our experts.  

‍

Frequently Asked Questions (FAQs)

‍

What is AI observability?

‍

AI observability is the ongoing monitoring of AI models and agents after they are deployed. It measures what an agent consumes and whether its output is correct, and it gives organizations a way to keep agents within approved limits and assign clear responsibility when performance drops.

‍

How is AI observability different from traditional observability?

‍

Traditional observability tells IT teams whether infrastructure and applications are up and performing. AI observability covers agents that act on their own, so it adds visibility into the quality of each decision alongside the health of the systems underneath.

‍

What metrics should AI observability track?

‍

The core signals are resource usage, including tokens and compute, and the accuracy of the output. Production teams should also watch for changes in model behavior over time, service availability, cost per request and guardrail effectiveness.

‍

Who should own AI observability in an organization?

‍

IT operations and infrastructure groups are well suited to everyday governance work, such as verifying data sources and enforcing guardrails. Security teams are better placed to handle external threats and the credentials agents use. Agents that drift from their intended behavior without malicious intent fall between the two, and both groups share responsibility for catching them.

‍

Can mid-market companies adopt AI observability without a dedicated platform team?

‍

Yes, provided they plan for who will run it. Specialized AI evaluation platforms are effective, though they're designed for companies with in-house platform engineers and ML specialists. Organizations without that staff may find a managed service that handles monitoring and evaluation on an ongoing basis easier to sustain.

‍

Will OpenTelemetry remain the standard for AI observability?

‍

OpenTelemetry is expected to stay the primary framework for gathering telemetry from LLM and agent workloads and correlating traces across them. Other emerging specifications mostly build on it, and DataStrike expects the biggest advances over the next couple of years to come in the semantic layer, where the industry still needs to define what AI telemetry should capture.