fix(evaluations): correct fallback rounds + friendlier error messages
Some checks failed
Aegis CI / lint-and-test (push) Has been cancelled
Some checks failed
Aegis CI / lint-and-test (push) Has been cancelled
- Fallback names now use hyphens matching live API (carbanak-fin7, wizard-spider-sandworm) - Add APT3 (R1) and Enterprise 2025/er7 (R7) to fallback - verified from live API - Remove OilRig (R6) from fallback - CrowdStrike did not participate in Round 6 - Orange fallback banner only shows when NO rounds are available at all - Soft gray note when rounds are loaded but API had transient error - Check-new and import errors: detect 502/Cloudflare messages and show user-friendly text instead of raw Cloudflare HTML error messages Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -64,10 +64,20 @@ _HEADERS = {
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fallback: hardcoded public CrowdStrike ENTERPRISE rounds
|
||||
# Used when evals.mitre.org API is unreachable (Cloudflare, outage, etc.)
|
||||
# These are well-known, publicly documented rounds — safe to hardcode.
|
||||
# Used when evals.mitre.org API is unreachable (Cloudflare 502, outage, etc.)
|
||||
#
|
||||
# Names use the EXACT slugs the live API returns (hyphens, not underscores).
|
||||
# Verified from live API response on 2025-06-05.
|
||||
# CrowdStrike did NOT participate in Round 6 (OilRig) — not included.
|
||||
# ---------------------------------------------------------------------------
|
||||
_FALLBACK_ROUNDS: list[dict[str, Any]] = [
|
||||
{
|
||||
"name": "apt3",
|
||||
"display_name": "APT3",
|
||||
"eval_round": 1,
|
||||
"domain": "ENTERPRISE",
|
||||
"status": "PUBLIC",
|
||||
},
|
||||
{
|
||||
"name": "apt29",
|
||||
"display_name": "APT29",
|
||||
@@ -76,14 +86,14 @@ _FALLBACK_ROUNDS: list[dict[str, Any]] = [
|
||||
"status": "PUBLIC",
|
||||
},
|
||||
{
|
||||
"name": "carbanak_fin7",
|
||||
"name": "carbanak-fin7",
|
||||
"display_name": "Carbanak+FIN7",
|
||||
"eval_round": 3,
|
||||
"domain": "ENTERPRISE",
|
||||
"status": "PUBLIC",
|
||||
},
|
||||
{
|
||||
"name": "wizard_spider_sandworm",
|
||||
"name": "wizard-spider-sandworm",
|
||||
"display_name": "Wizard Spider + Sandworm",
|
||||
"eval_round": 4,
|
||||
"domain": "ENTERPRISE",
|
||||
@@ -97,9 +107,9 @@ _FALLBACK_ROUNDS: list[dict[str, Any]] = [
|
||||
"status": "PUBLIC",
|
||||
},
|
||||
{
|
||||
"name": "oilrig",
|
||||
"display_name": "OilRig",
|
||||
"eval_round": 6,
|
||||
"name": "er7",
|
||||
"display_name": "Enterprise 2025",
|
||||
"eval_round": 7,
|
||||
"domain": "ENTERPRISE",
|
||||
"status": "PUBLIC",
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user