BraintrustUnclaimed

Braintrust is an observability and evaluation platform for AI agents. It records traces of agents running in production, including prompts, responses and tool calls, so teams can search them, track latency, cost and quality, and see what actually happened.

Evals let teams define what good output looks like before they ship. Teams can run experiments against datasets, compare prompts and models side by side, and score outputs with LLMs, code or human reviewers. Production traces can be turned into eval datasets, so real failures become regression tests.

The Discover feature applies analysis to traces to find recurring behaviour and patterns. Loop is an AI agent that generates prompts, scorers and datasets to help optimise evals. An MCP server connects a coding agent in an IDE to logs and prompts, and SDKs are available for Python, TypeScript, Go, Ruby and C#.

Braintrust is aimed at engineering teams and product managers building AI agents, from first release to enterprise scale. The site says it is SOC 2 Type II certified and GDPR and HIPAA compliant, with SSO and role-based access control.

The Starter plan is free and includes $10 of model credits, 1 GB of processed data, 10,000 scores and 14-day retention, with no credit card required. The Pro plan costs $249 per month, and Enterprise pricing is custom and includes on-prem or hosted deployment. Qualifying startups can get six to twelve months of Pro free.