Reference

Glossary

Definitions for the platform objects, evaluation methods, evidence, and metrics used throughout AIDX documentation.

Use this glossary when a method, metric, or object name is unfamiliar. Method-specific pages remain the source of truth for how an individual evaluation calculates and interprets its results.

Platform objects

Workspace: The organizational boundary in which users configure targets, create evaluations, and access results.
Test target: The model, chatbot, or AI application whose behavior is evaluated.
Target version: The declared release or configuration boundary for the tested target. A change to model, prompts, retrieval, tools, or guardrails can create a new effective version.
Connection: The endpoint and request settings AIDX uses to send inputs to the target and receive responses.
Evaluation: One configured run of a named test method against a target under a declared scope.
Evaluation run: The execution instance that processes test cases, collects responses, scores outcomes, and prepares results.
Report: A review-ready package of scope, method, summary metrics, findings, and selected evidence from an evaluation.

Test evidence

Test case: The smallest independently evaluated unit, usually an input or scenario together with the target response and evaluator outcome.
Dataset: A versioned collection of cases or source material used to configure and execute an evaluation.
Prompt: An input message or instruction sent to the target. In a multi-turn evaluation, one case may contain several prompts.
Scenario: A realistic context, user intent, or interaction path used to test behavior beyond a single isolated question.
Policy source: An internal policy, regulation, standard, or conduct rule transformed into testable alignment requirements.
Attack method: A repeatable transformation or adversarial strategy used to test whether safeguards can be bypassed.
Evidence: The recorded input, response, score, classification, explanation, and metadata supporting a result.
Failure case: A case whose outcome crosses the method’s failure condition or reveals behavior requiring review.

Evaluation methods

BenchDX: Baseline safety testing under normal, realistic use across major AI risk categories.
RobustDX: Adversarial robustness testing against jailbreak, prompt-injection, encoding, manipulation, and related attacks.
HalluDX (Coming soon): A planned hallucination-testing evaluation for factual reliability, repeated-response consistency, and claim-level exposure.
AlignDX (Coming soon): A planned alignment-testing evaluation using policy-derived, realistic, and often multi-turn scenarios.
Black-box evaluation: Testing based on observable inputs and outputs without requiring access to model weights or internal implementation.
Red teaming: Purposeful adversarial testing that attempts to make a system violate its safeguards or intended boundary.
Baseline safety: How safely a system behaves during ordinary but potentially risky user interactions, rather than under a deliberate attack.

Metrics and scoring

Case score: The evaluator outcome for one test case. Its scale and meaning are defined by the evaluation method.
Mean score: The arithmetic average of included case scores. Always check which cases, dimensions, and exclusions contribute to it.
Pass rate: The share of evaluated cases that meet a defined passing condition.
Attack success rate (ASR): The share of adversarial cases in which an attack achieves the method’s defined unsafe or policy-violating outcome. Lower is better.
Risk index: A method-specific aggregate that expresses exposure on a defined scale. Confirm directionality before comparing values.
Dimension: A broad analytical grouping, such as a safety domain, attack family, topic, or policy area.
Category: A more specific grouping within a dimension, used to locate and prioritize risk.
Threshold: A declared cutoff used to assign a pass/fail outcome, risk band, or qualification level.
Percentile: The target’s relative position within a named comparison cohort. It is not the same as a raw score.
DX-Score: Under DX-Score General, the BenchDX mean and RobustDX mean are each measured on a five-point scale and added to produce a ten-point combined score.

Scope and versioning

Tested scope: The system identity, version, configuration, interface, intended use, and deployment boundary to which a result applies.
Method version: The version of the evaluation methodology and scoring rules used for a result.
Test-suite version: The versioned set of cases and configuration used by a standardized evaluation package.
Baseline version: The locked comparison cohort and calculation context used for a ranking or qualification result.
Reproducibility: The degree to which a result can be understood or repeated from its preserved scope, versions, settings, and evidence.
Retest: A new evaluation performed after a target, dataset, method, or material configuration change.

DX-Score certificate terms

DX-Score Certified: A qualifying tested result at or above the 50th percentile of the locked cohort under the same DX-Score General method and baseline.
DX-Score Excellence: A qualifying tested result in the top 20% of the locked cohort under the same DX-Score General method and baseline.
Verification ID: The public identifier attached to a DX-Score certificate record for verification.