API tutorials

Python example

Use Python requests to retrieve prompts and submit responses for either BenchDX or RobustDX.

The following functions cover all four documented endpoints. Set adversarial=False for BenchDX or adversarial=True for RobustDX.

Fetch prompts

fetch_promptsPython

from typing import Any

import requests

BASE_URL = "https://trial.aidx.pro"


def fetch_prompts(
    evaluation_id: str,
    token: str,
    adversarial: bool = False,
) -> list[dict[str, Any]]:
    endpoint = "all-adv-prompts" if adversarial else "promptsAll"
    response = requests.get(
        f"{BASE_URL}/api/evaluation/{endpoint}",
        headers={"Authorization": f"Bearer {token}"},
        params={"evaluationId": evaluation_id},
        timeout=30,
    )
    response.raise_for_status()

    result = response.json()
    if not result.get("success"):
        raise RuntimeError(result.get("message", "Failed to fetch prompts"))
    return result["data"]

The function checks both the HTTP status and the response envelope before returning the prompt list.

Submit a response

submit_responsePython

import requests

BASE_URL = "https://trial.aidx.pro"


def submit_response(
    evaluation_id: str,
    trace_id: str,
    model_response: str,
    token: str,
    adversarial: bool = False,
) -> None:
    endpoint = "adv-response" if adversarial else "response"
    response_field = "advResponse" if adversarial else "llmResponse"

    response = requests.post(
        f"{BASE_URL}/api/evaluation/{endpoint}",
        headers={"Authorization": f"Bearer {token}"},
        data={
            "evaluationId": evaluation_id,
            "traceId": trace_id,
            response_field: model_response,
        },
        timeout=30,
    )
    response.raise_for_status()

    result = response.json()
    if not result.get("success"):
        raise RuntimeError(result.get("message", "Failed to submit response"))

Evaluation	GET endpoint	POST endpoint	Response field
BenchDX	`promptsAll`	`response`	`llmResponse`
RobustDX	`all-adv-prompts`	`adv-response`	`advResponse`

Connect your target call

Implement call_target_model for your own model or application API, then preserve each prompt’s trace through the loop:

BenchDX execution loopPython

EVALUATION_ID = "your_evaluation_id"
AIDX_PAT = "your_personal_access_token"

prompts = fetch_prompts(
    evaluation_id=EVALUATION_ID,
    token=AIDX_PAT,
    adversarial=False,
)

for item in prompts:
    # Implement this function for your model or application API.
    model_response = call_target_model(item["content"])

    submit_response(
        evaluation_id=EVALUATION_ID,
        trace_id=item["traceId"],
        model_response=model_response,
        token=AIDX_PAT,
        adversarial=False,
    )