If our default metrics don’t meet your needs, you can create your own custom metrics. You can create custom metrics in the Atla Alignment Platform.

After creating and registering a custom metric, it immediately becomes available for use in both our API and Python SDK. Simply pass your custom metric name in the metric_name field.

Custom metrics can currently only be created through the Alignment Platform.

The Atla Alignment Platform is not available just yet. If you are keen to get early access, reach out to us at support@atla-ai.com. In the meantime, we recommend using custom evaluation criteria via the evaluation_criteria field.