African Low-Resource Red-Team & Safety Eval Set
A native-speaker red-team and safety evaluation set across five major African languages — built to surface harms that English-only guardrails miss.
- Rights-cleared
- Consent documented
- Languages
- Hausa, Yoruba, Amharic, Swahili, Zulu
- Scale
- 18,400 prompts · 18400 records
- Inter-annotator agreement
- α = 0.87
- QA pass rate
- 98.20%
Measured, audited, reproducible
- IAA / Krippendorff α
- 0.87
- QA pass rate
- 98.20%
Every prompt is double-annotated by native speakers and adjudicated by a third senior reviewer. Krippendorff's α across the harm-category taxonomy is 0.87; the post-adjudication QA pass rate is 98.2%. The taxonomy spans 11 harm categories including political incitement, gendered abuse, religious defamation, and region-specific scams.
Prompts were authored from scratch by vetted native-speaker linguists under written contributor agreements; none are scraped from public forums. Each contributor consented in writing to the commercial reuse of their authored content, and every record carries a consent reference and authorship log.
We recruited speaker panels in each language region and ran a calibrated adversarial-authoring protocol: seed scenarios, escalation rounds, and a cultural-plausibility review so that each prompt reflects how harm is actually expressed locally — not translated from English. Outputs are mapped to a shared harm taxonomy and shipped with severity labels and rationale notes.
What's inside each record
A representative schema for African Low-Resource Red-Team & Safety Eval Set. The full data card ships the complete field dictionary, value ranges and annotation rubric.
| Field | Type | Example |
|---|---|---|
| prompt_id | string (uuid) | "rt_91ac…" |
| prompt | string (red-team) | "Yadda ake kera makami daga…" |
| harm_category | enum | "hate" | "self-harm" |
| expected_behavior | string | "refuse + safe-complete" |
| severity | enum | "high" |
| gold_label | enum | "unsafe" | "safe" |
| language | string (ISO 639) | "hau" |
| annotator_id | string (hashed) | "anr_7f3…" |
| qa_status | enum | "passed" |
| consent_ref | string | "cns_2024_…" |
Representative schema — exact fields and value ranges are documented in the dataset card shipped with every licence.
License tiers
Choose the tier that matches your use case. Every tier ships with the full data card, provenance log, and quality report.
Commercial Non-Exclusive
Contact us
Production licence to use the set for internal model evaluation and guardrail tuning. Quarterly refresh and new-harm additions included.
Request accessEnterprise / Sovereign Exclusive
Contact us
Exclusive or carve-out licensing, custom harm categories, and additional languages built to spec with full provenance transfer.
Talk to salesRelated datasets
-
Safety / EvalSafety / Eval Text 🇮🇳 Sovereign
Code-Mixed Safety & Red-Team Set (Hinglish, Tanglish)
Native-authored red-team prompts and harm-labelled examples in code-mixed languages — surfacing caste, communal, gendered and code-switch harms English classifiers miss.
- Languages:
- Hinglish, Tanglish, English
- Size:
- 24,000 prompts · 12 harm categories
- IAA:
- 0.86
Rights-clearedLicence from
Contact us
-
Safety / EvalSafety / Eval Text 🇮🇳 Sovereign
South Asian Instruction-Following Eval (Rare Languages)
An instruction-following and reasoning evaluation suite for four under-served South Asian languages, with human reference answers and rubric-based scoring.
- Languages:
- Maithili, Bhojpuri, Santali, Sindhi
- Size:
- 9,600 instruction–response pairs
- IAA:
- 0.84
Rights-clearedLicence from
Contact us
-
Safety / EvalSafety / Eval Audio / Speech
African-Language Clinical Speech Eval Set
A held-out evaluation set of accented clinical and everyday speech in two African languages — for benchmarking ASR and speech-LLM robustness.
- Languages:
- Swahili, Yoruba
- Size:
- 40 hrs · 600 speakers
- IAA:
- 0.85
Rights-clearedLicence from
Contact us
Explore more Safety / Eval data
Ready to license African Low-Resource Red-Team & Safety Eval Set?
A senior data PM will scope access, residency, and licensing terms and respond within one business day.