The Atla REST API is capable of handling multiple requests in an asynchronous manner as well as in a parallel manner, useful for when you want to run evaluations over a large batch of data.

​
Async Usage

import asyncio
from atla import AsyncAtla

client = AsyncAtla()

async def main() -> None:
    evals = await client.evaluation.create(
        input="If 5 machines take 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?",
        response="Total processing time is 5 widgets * 5 minutes = 25 minutes. Then 25 mins / 100 machines = 15 seconds.",
        metrics=["logical_coherence"]
    )

    print(f"Score: {evals.evaluations['logical_coherence'].score}")

asyncio.run(main())

The functionality between the synchronous and asynchronous clients is otherwise identical.

​
Parallel Evaluations

Using the asynchronous capability of the API, you can send multiple evaluations in parallel and gather the results together, significantly speeding up evaluations across a large dataset.

import asyncio
import pandas as pd
from atla import AsyncAtla

client = AsyncAtla()

# loading a sample dataset of n = 11 evals
data_url = "https://raw.githubusercontent.com/atla-ai/eval_demo_data/main/yc_launch_demo_data.csv"
data = pd.read_csv(data_url)

async def send_eval(index, input, response, reference):
    return index, await client.evaluation.create(input=input, response=response, reference=reference, metrics=["logical_coherence"])

async def main():
    tasks = []
    for index, row in data.iterrows():
        tasks.append(send_eval(index, row["Question"], row["AI's Response"], row["Reference Response"]))

    results = await asyncio.gather(*tasks)
    for index, evals in results:
        print(f"Index: {index}, Score: {evals.evaluations['logical_coherence'].score}")

asyncio.run(main())

When making large, repeated batch requests to the API, you may encounter internal rate limits. More information about our rate limiting plans are available here.

ErrorsRate Limits