Custom Metrics

If our default metrics don’t meet your needs, you can create your own custom metrics. You can create custom metrics in the Atla Alignment Platform.

After creating and registering a custom metric, it immediately becomes available for use in both our API and Python SDK.

Simply pass your custom metric name in the metric_name field.

from atla import Atla
client = Atla()

evaluation = client.evaluation.create(
    model_id="atla-selene",
    model_input="What is the capital of France?",
    model_output="Paris",
    metric_name="my_custom_metric", # replace with your custom metric name
).result.evaluation

print(evaluation)