If our default metrics don’t meet your needs, you can create your own custom metrics. You can create custom metrics in the Atla Alignment Platform.

After creating and registering a custom metric, it immediately becomes available for use in both our API and Python SDK.

Simply pass your custom metric name in the metric_name field.

 Custom metrics can currently only be created through the Alignment Platform. Only you will be able to access your created metrics via your API key.

from atla import Atla
client = Atla()

evaluation = client.evaluation.create(
    model_id="atla-selene",
    model_input="What is the capital of France?",
    model_output="Paris",
    metric_name="my_custom_metric", # replace with your custom metric name
).result.evaluation

print(evaluation)