Skip to main content
Atla isn’t just for monitoring production traces and shipping improvements faster. It’s also used to help teams iterate so they can launch in days, not weeks.

Development Workflow Steps

1

Create custom metrics

Navigate to your Metrics screen and create custom metrics that matter to your domain. While we automatically flag error types and failure patterns, you can add domain-specific metrics like tool_call_efficiency.
2

Configure metadata for test runs

Set up metadata to track different test configurations. For example, you might run three versions of a prompt that are concise, balanced, and verbose:
from atla_insights import configure, set_metadata

# Configure global metadata for the experiment
configure(
    token="<YOUR_TOKEN>",
    metadata={"experiment": "prompt_comparison"}
)

# Then set dynamic metadata for each test run
set_metadata({"prompt": "v1"})  # concise version
set_metadata({"prompt": "v2"})  # balanced version  
set_metadata({"prompt": "v3"})  # verbose version
3

Run your tests

Execute your test suite with the configured metadata and custom metrics enabled.
4

Compare results

Navigate to the Compare screen to analyze the relative error rates and performance on your custom metrics across different configurations.
5

Deep dive into issues

Click “View” on any column to deep dive into the specific step-level errors in the traces to understand deeper issues.
6

Continue iterating

Continue to iterate on different dimensions. Use the “experiment” metadata tag to track performance of different experiments, such as testing different architectures and setups.

Example Metadata Configuration

from atla_insights import configure, set_metadata

# Configure with global experiment metadata
configure(
    token="<YOUR_TOKEN>",
    metadata={"experiment": "model_comparison"}
)

# Set dynamic metadata during runtime
set_metadata({"prompt": "v1", "model": "gpt-4"})
set_metadata({"prompt": "v2", "model": "claude-3"})
This approach helps you systematically test and improve your AI applications before they reach production.
I