feat(phase-33): final polish V3 - navigation, performance, and documentation (T-238 to T-240)
This commit is contained in:
98
docs/API.md
98
docs/API.md
@@ -604,3 +604,101 @@ Common HTTP status codes:
|
||||
- `404` - Not Found (resource doesn't exist)
|
||||
- `409` - Conflict (duplicate resource)
|
||||
- `500` - Internal Server Error
|
||||
|
||||
---
|
||||
|
||||
## V3 Endpoints
|
||||
|
||||
### Campaigns
|
||||
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/campaigns` | Authenticated | List campaigns (filters: status, type, search; pagination: offset, limit) |
|
||||
| POST | `/api/v1/campaigns` | Authenticated | Create campaign |
|
||||
| GET | `/api/v1/campaigns/{id}` | Authenticated | Campaign detail with tests |
|
||||
| PATCH | `/api/v1/campaigns/{id}` | Creator, Admin | Update campaign |
|
||||
| DELETE | `/api/v1/campaigns/{id}` | Creator, Admin | Delete campaign |
|
||||
| POST | `/api/v1/campaigns/{id}/tests` | Authenticated | Add test to campaign |
|
||||
| DELETE | `/api/v1/campaigns/{id}/tests/{test_id}` | Authenticated | Remove test from campaign |
|
||||
| PATCH | `/api/v1/campaigns/{id}/schedule` | Authenticated | Set recurring schedule |
|
||||
| GET | `/api/v1/campaigns/{id}/history` | Authenticated | Execution history for recurring campaigns |
|
||||
|
||||
### Threat Actors
|
||||
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/threat-actors` | Authenticated | List threat actors (filters: country, motivation, search) |
|
||||
| GET | `/api/v1/threat-actors/{id}` | Authenticated | Detail with technique mappings |
|
||||
|
||||
### Detection Rules
|
||||
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/detection-rules` | Authenticated | List rules (filters: source, mitre_technique_id, severity, search) |
|
||||
| GET | `/api/v1/detection-rules/{id}` | Authenticated | Rule detail |
|
||||
|
||||
### D3FEND (Defensive Techniques)
|
||||
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/d3fend` | Authenticated | List defensive techniques |
|
||||
| GET | `/api/v1/d3fend/{id}` | Authenticated | Detail with ATT&CK mappings |
|
||||
|
||||
### Compliance
|
||||
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/compliance/frameworks` | Authenticated | List compliance frameworks |
|
||||
| GET | `/api/v1/compliance/{framework_id}/controls` | Authenticated | List controls with coverage status |
|
||||
| GET | `/api/v1/compliance/{framework_id}/summary` | Authenticated | Coverage summary (total, covered, gaps) |
|
||||
| GET | `/api/v1/compliance/{framework_id}/gaps` | Authenticated | Gap analysis — uncovered controls |
|
||||
|
||||
### Scores
|
||||
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/scores/technique/{mitre_id}` | Authenticated | Technique score with detailed breakdown |
|
||||
| GET | `/api/v1/scores/tactic/{tactic}` | Authenticated | Average tactic score |
|
||||
| GET | `/api/v1/scores/threat-actor/{id}` | Authenticated | Coverage score against threat actor |
|
||||
| GET | `/api/v1/scores/organization` | Authenticated | Overall organization score (cached 5 min) |
|
||||
| GET | `/api/v1/scores/history` | Authenticated | Weekly score history (period: 30d, 90d, 1y) |
|
||||
| GET | `/api/v1/scores/config` | Admin | Current scoring weights |
|
||||
| PATCH | `/api/v1/scores/config` | Admin | Update scoring weights |
|
||||
|
||||
### Operational Metrics
|
||||
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/metrics/operational` | Authenticated | All KPIs (MTTD, MTTR, efficacy, etc.) — cached 5 min |
|
||||
| GET | `/api/v1/metrics/operational/trend` | Authenticated | Weekly trend (period: 30d, 90d, 1y) |
|
||||
| GET | `/api/v1/metrics/operational/by-team` | Authenticated | Red vs Blue team breakdown |
|
||||
|
||||
### Heatmap
|
||||
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/heatmap` | Authenticated | Full ATT&CK Navigator-style heatmap data |
|
||||
|
||||
### Coverage Snapshots
|
||||
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/snapshots` | Authenticated | List snapshots (pagination: offset, limit) |
|
||||
| POST | `/api/v1/snapshots` | Authenticated | Create new snapshot |
|
||||
| GET | `/api/v1/snapshots/{id}` | Authenticated | Snapshot detail with technique states |
|
||||
| DELETE | `/api/v1/snapshots/{id}` | Admin | Delete snapshot |
|
||||
| GET | `/api/v1/snapshots/compare` | Authenticated | Compare two snapshots (query: a, b) |
|
||||
|
||||
### Re-testing
|
||||
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/tests/{id}/retest-chain` | Authenticated | Full retest chain for a test |
|
||||
|
||||
### Data Sources
|
||||
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/data-sources` | Admin | List all data sources |
|
||||
| PATCH | `/api/v1/data-sources/{id}` | Admin | Update source config |
|
||||
| POST | `/api/v1/data-sources/{id}/sync` | Admin | Trigger manual sync |
|
||||
|
||||
220
docs/ARCHITECTURE.md
Normal file
220
docs/ARCHITECTURE.md
Normal file
@@ -0,0 +1,220 @@
|
||||
# Aegis — Architecture
|
||||
|
||||
## High-Level Overview
|
||||
|
||||
```
|
||||
┌────────────────────┐ ┌─────────────────────┐
|
||||
│ React Frontend │──────▶│ FastAPI Backend │
|
||||
│ (Vite / TS / TW) │ REST │ (Python 3.11) │
|
||||
└────────────────────┘ └──────┬──────┬────────┘
|
||||
│ │
|
||||
┌─────────┘ └─────────┐
|
||||
▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ PostgreSQL │ │ MinIO │
|
||||
│ (Data Store) │ │ (Object Storage) │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
- **Frontend** — React 19 + TypeScript + Tailwind CSS v4 + TanStack Query
|
||||
- **Backend** — FastAPI with SQLAlchemy ORM + Alembic migrations
|
||||
- **Database** — PostgreSQL 15 with UUID primary keys and JSONB columns
|
||||
- **Object Storage** — MinIO (S3-compatible) for evidence files
|
||||
- **Scheduler** — APScheduler (in-process) for background jobs
|
||||
|
||||
---
|
||||
|
||||
## Database Schema
|
||||
|
||||
### Core Tables
|
||||
|
||||
| Table | Description |
|
||||
|-------|-------------|
|
||||
| `users` | User accounts with role-based access (admin, red_tech, blue_tech, red_lead, blue_lead, viewer) |
|
||||
| `techniques` | MITRE ATT&CK techniques with coverage status, tactic, platforms (JSONB) |
|
||||
| `tests` | Security tests with full Red/Blue workflow fields, dual validation, remediation, and retest chain |
|
||||
| `test_templates` | Predefined test catalog from Atomic Red Team, Sigma, CALDERA, LOLBAS, custom |
|
||||
| `evidences` | Evidence files separated by team (red/blue) with SHA256 integrity verification |
|
||||
|
||||
### Detection & Defense
|
||||
|
||||
| Table | Description |
|
||||
|-------|-------------|
|
||||
| `detection_rules` | Imported detection rules (Sigma, Elastic, custom) linked to ATT&CK techniques |
|
||||
| `test_detection_results` | Per-test detection rule evaluation results (triggered / not triggered) |
|
||||
| `test_template_detection_rules` | Template ↔ detection rule associations |
|
||||
| `defensive_techniques` | MITRE D3FEND defensive techniques |
|
||||
| `defensive_technique_mappings` | ATT&CK technique ↔ D3FEND defensive technique mappings |
|
||||
|
||||
### Campaigns & Scheduling
|
||||
|
||||
| Table | Description |
|
||||
|-------|-------------|
|
||||
| `campaigns` | Test campaign groupings with scheduling (recurring, weekly/monthly/quarterly) |
|
||||
| `campaign_tests` | Ordered test assignments within campaigns with dependency support |
|
||||
|
||||
### Intelligence & Actors
|
||||
|
||||
| Table | Description |
|
||||
|-------|-------------|
|
||||
| `threat_actors` | MITRE CTI intrusion sets with aliases, country, motivation, JSONB targets |
|
||||
| `threat_actor_techniques` | Threat actor ↔ ATT&CK technique mappings |
|
||||
| `intel_items` | Threat intelligence items from RSS feeds |
|
||||
|
||||
### Compliance
|
||||
|
||||
| Table | Description |
|
||||
|-------|-------------|
|
||||
| `compliance_frameworks` | Compliance frameworks (e.g., NIST 800-53) |
|
||||
| `compliance_controls` | Individual controls within a framework |
|
||||
| `compliance_control_mappings` | Control ↔ ATT&CK technique mappings |
|
||||
|
||||
### Operational
|
||||
|
||||
| Table | Description |
|
||||
|-------|-------------|
|
||||
| `coverage_snapshots` | Point-in-time coverage status captures with aggregate metrics |
|
||||
| `snapshot_technique_states` | Normalized per-technique state within a snapshot |
|
||||
| `audit_logs` | System-wide audit trail with JSONB details |
|
||||
| `notifications` | In-app notifications with read status |
|
||||
| `data_sources` | External data source configuration and sync status |
|
||||
|
||||
### Key Relationships
|
||||
|
||||
```
|
||||
Technique ──1:N── Test ──1:N── Evidence
|
||||
│ │
|
||||
│ ├── TestDetectionResult ──N:1── DetectionRule
|
||||
│ └── CampaignTest ──N:1── Campaign
|
||||
│
|
||||
├── ThreatActorTechnique ──N:1── ThreatActor
|
||||
├── DefensiveTechniqueMapping ──N:1── DefensiveTechnique
|
||||
├── ComplianceControlMapping ──N:1── ComplianceControl ──N:1── ComplianceFramework
|
||||
└── SnapshotTechniqueState ──N:1── CoverageSnapshot
|
||||
|
||||
Test ──retest_of──▶ Test (self-referential retest chain)
|
||||
Campaign ──parent_campaign_id──▶ Campaign (recurring execution history)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Backend Architecture
|
||||
|
||||
### Layered Structure
|
||||
|
||||
```
|
||||
routers/ ← HTTP endpoints (input validation, auth, response shaping)
|
||||
↓
|
||||
services/ ← Business logic (state machines, calculations, imports)
|
||||
↓
|
||||
models/ ← SQLAlchemy ORM models
|
||||
↓
|
||||
database.py ← Engine + session management (lazy initialization)
|
||||
```
|
||||
|
||||
### Services
|
||||
|
||||
| Service | Responsibility |
|
||||
|---------|---------------|
|
||||
| `test_workflow_service` | Test state machine (draft → validated/rejected) with dual validation |
|
||||
| `scoring_service` | 0–100 scoring for techniques, tactics, actors, organization |
|
||||
| `score_cache` | In-memory TTL cache (5 min) for expensive score/metric calculations |
|
||||
| `operational_metrics_service` | MTTD, MTTR, detection efficacy, alert fidelity, coverage velocity |
|
||||
| `snapshot_service` | Coverage snapshot creation, temporal comparison, cleanup |
|
||||
| `campaign_service` | Campaign CRUD, progress tracking, circular dependency prevention |
|
||||
| `campaign_scheduler_service` | Recurring campaign execution (clone + schedule next run) |
|
||||
| `status_service` | Technique status recalculation from test results |
|
||||
| `notification_service` | In-app notification CRUD and state-change alerts |
|
||||
| `audit_service` | Immutable audit trail logging |
|
||||
| `mitre_sync_service` | MITRE ATT&CK sync via TAXII 2.0 / GitHub fallback |
|
||||
| `atomic_import_service` | Atomic Red Team template import from GitHub |
|
||||
| `sigma_import_service` | SigmaHQ detection rule import |
|
||||
| `elastic_import_service` | Elastic detection rule import (TOML) |
|
||||
| `caldera_import_service` | CALDERA ability import |
|
||||
| `lolbas_import_service` | LOLBAS/GTFOBins template import |
|
||||
| `d3fend_import_service` | MITRE D3FEND defensive technique import |
|
||||
| `threat_actor_import_service` | MITRE CTI threat actor import (STIX) |
|
||||
| `compliance_import_service` | NIST 800-53 ↔ ATT&CK mapping import |
|
||||
| `intel_service` | RSS-based threat intelligence scanning |
|
||||
|
||||
### Scheduled Jobs (APScheduler)
|
||||
|
||||
| Job | Schedule | Description |
|
||||
|-----|----------|-------------|
|
||||
| MITRE Sync | Every 24h | Sync ATT&CK techniques from TAXII/GitHub |
|
||||
| Intel Scan | Every 7 days | Scan RSS feeds for threat intelligence |
|
||||
| Notification Cleanup | Every 24h | Remove old read notifications |
|
||||
| Weekly Snapshot | Sundays 00:00 | Create coverage snapshot + cleanup old ones |
|
||||
| Recurring Campaigns | Every 24h | Check and execute due recurring campaigns |
|
||||
|
||||
---
|
||||
|
||||
## Test Lifecycle (State Machine)
|
||||
|
||||
```
|
||||
┌──────┐ ┌──────────────┐ ┌─────────────────┐ ┌───────────┐
|
||||
│ DRAFT│───▶│RED_EXECUTING │───▶│ BLUE_EVALUATING │───▶│ IN_REVIEW │
|
||||
└──────┘ └──────────────┘ └─────────────────┘ └─────┬─────┘
|
||||
│
|
||||
┌───────────────────┤
|
||||
▼ ▼
|
||||
┌──────────┐ ┌──────────┐
|
||||
│ REJECTED │ │VALIDATED │
|
||||
└────┬─────┘ └──────────┘
|
||||
│ │
|
||||
└──▶ Back to DRAFT ├──▶ Remediation
|
||||
└──▶ Auto Re-test
|
||||
```
|
||||
|
||||
**Dual Validation in IN_REVIEW:**
|
||||
- Red Lead votes approve/reject
|
||||
- Blue Lead votes approve/reject
|
||||
- Both approve → VALIDATED
|
||||
- Either rejects → REJECTED
|
||||
- One votes, other pending → stays IN_REVIEW
|
||||
|
||||
**Auto Re-testing:** When remediation is completed on a validated test, the system automatically creates a follow-up retest (up to `MAX_RETEST_COUNT` = 3).
|
||||
|
||||
---
|
||||
|
||||
## Frontend Architecture
|
||||
|
||||
### Key Technologies
|
||||
|
||||
- **React 19** with TypeScript
|
||||
- **Vite 7** for bundling
|
||||
- **Tailwind CSS v4** for styling
|
||||
- **TanStack Query** for server state management
|
||||
- **TanStack Virtual** for table virtualization
|
||||
- **React Router v7** for routing
|
||||
- **Recharts** for charts and visualizations
|
||||
- **Lucide React** for icons
|
||||
|
||||
### Page Lazy Loading
|
||||
|
||||
All pages except `LoginPage` and `DashboardPage` are lazy-loaded via `React.lazy()` with `<Suspense>` fallbacks for optimal initial bundle size.
|
||||
|
||||
### Role-Based Navigation
|
||||
|
||||
The sidebar dynamically filters navigation items based on the current user's role:
|
||||
|
||||
| Section | Visible to |
|
||||
|---------|-----------|
|
||||
| Dashboard | All roles |
|
||||
| Executive Dashboard | admin, red_lead, blue_lead |
|
||||
| ATT&CK Matrix | All roles |
|
||||
| Tests (sub-menu) | All roles |
|
||||
| Campaigns | All roles |
|
||||
| Threat Actors | All roles |
|
||||
| Compliance | All roles |
|
||||
| Comparison | admin, red_lead, blue_lead |
|
||||
| Reports | All roles |
|
||||
| System (admin section) | admin only |
|
||||
|
||||
### Performance Optimizations
|
||||
|
||||
- **React.memo** on `HeatmapCell` (renders 3000+ times in full matrix)
|
||||
- **useMemo** / **useCallback** for expensive calculations in memoized components
|
||||
- **useDebounce** hook for search inputs (300ms delay)
|
||||
- **TanStack Virtual** for large table virtualization (test templates, detection rules, audit logs)
|
||||
- **Lazy loading** for all non-critical page bundles
|
||||
281
docs/DATA_SOURCES.md
Normal file
281
docs/DATA_SOURCES.md
Normal file
@@ -0,0 +1,281 @@
|
||||
# Aegis — Data Sources
|
||||
|
||||
Aegis imports security data from multiple external sources to populate its test catalog, detection rules, defensive techniques, threat actors, and compliance mappings. This document describes each source, its format, and how to manage imports.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
| Source | Type | Format | Destination |
|
||||
|--------|------|--------|-------------|
|
||||
| MITRE ATT&CK | Techniques | STIX 2.0 (TAXII / GitHub) | `techniques` |
|
||||
| Atomic Red Team | Test Templates | YAML | `test_templates` |
|
||||
| SigmaHQ | Detection Rules | YAML | `detection_rules` |
|
||||
| Elastic Detection Rules | Detection Rules | TOML | `detection_rules` |
|
||||
| CALDERA | Test Templates | YAML (multi-doc) | `test_templates` |
|
||||
| LOLBAS | Test Templates | YAML | `test_templates` |
|
||||
| MITRE D3FEND | Defensive Techniques | JSON-LD | `defensive_techniques` + mappings |
|
||||
| MITRE CTI | Threat Actors | STIX 2.0 (JSON) | `threat_actors` + technique mappings |
|
||||
| NIST 800-53 → ATT&CK | Compliance Mappings | STIX 2.0 (JSON) | `compliance_*` tables |
|
||||
|
||||
---
|
||||
|
||||
## MITRE ATT&CK (Techniques)
|
||||
|
||||
**Repository:** https://github.com/mitre/cti (Enterprise ATT&CK)
|
||||
**Protocol:** TAXII 2.0 with GitHub JSON fallback
|
||||
**Format:** STIX 2.0 bundles
|
||||
**Service:** `mitre_sync_service.py`
|
||||
**Schedule:** Automatic every 24 hours via APScheduler
|
||||
|
||||
**Extracted fields:**
|
||||
- `mitre_id` — External ID (e.g., T1059, T1059.001)
|
||||
- `name` — Technique name
|
||||
- `description` — Full description
|
||||
- `tactic` — ATT&CK tactic (execution, persistence, etc.)
|
||||
- `platforms` — Target platforms (windows, linux, macos, etc.)
|
||||
- `is_subtechnique` — Whether it's a sub-technique
|
||||
- `url` — Link to MITRE page
|
||||
|
||||
**Manual trigger:**
|
||||
```bash
|
||||
# Via API
|
||||
curl -X POST http://localhost:8000/api/v1/system/sync-mitre \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
|
||||
# Via container
|
||||
docker exec aegis-backend python -c "from app.services.mitre_sync_service import sync_mitre_attack; sync_mitre_attack()"
|
||||
```
|
||||
|
||||
**Volume:** ~700 techniques (Enterprise ATT&CK v16)
|
||||
|
||||
**Troubleshooting:**
|
||||
- If TAXII fails (timeout/rate limit), the service automatically falls back to the GitHub JSON bundle
|
||||
- Check `data_sources` table for `last_sync_at` and `last_sync_stats`
|
||||
|
||||
---
|
||||
|
||||
## Atomic Red Team (Test Templates)
|
||||
|
||||
**Repository:** https://github.com/redcanaryco/atomic-red-team
|
||||
**Format:** YAML (one file per technique under `atomics/T*/T*.yaml`)
|
||||
**Service:** `atomic_import_service.py`
|
||||
|
||||
**Extracted fields:**
|
||||
- `name` — Test name
|
||||
- `description` — What the test does
|
||||
- `mitre_technique_id` — ATT&CK technique ID
|
||||
- `attack_commands` — Commands to execute
|
||||
- `expected_detection` — What should be detected
|
||||
- `platform` — Target OS
|
||||
- `severity` — Derived from technique context
|
||||
- `cleanup_commands` — Cleanup procedures
|
||||
|
||||
**Import:**
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/api/v1/system/import-atomic-red-team \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
```
|
||||
|
||||
**Volume:** ~3,500 test templates
|
||||
**Frequency:** Monthly or after ATT&CK version updates
|
||||
|
||||
---
|
||||
|
||||
## SigmaHQ (Detection Rules)
|
||||
|
||||
**Repository:** https://github.com/SigmaHQ/sigma
|
||||
**Format:** YAML (Sigma rule format)
|
||||
**Service:** `sigma_import_service.py`
|
||||
|
||||
**Extracted fields:**
|
||||
- `name` — Rule title
|
||||
- `description` — Rule description
|
||||
- `query` — Sigma detection logic
|
||||
- `mitre_technique_id` — Extracted from `tags: attack.tXXXX`
|
||||
- `severity` — From `level` field (low, medium, high, critical)
|
||||
- `platforms` — From `logsource.product`
|
||||
- `source` = `"sigma"`
|
||||
|
||||
**Example Sigma rule tags:**
|
||||
```yaml
|
||||
tags:
|
||||
- attack.execution
|
||||
- attack.t1059.001
|
||||
- attack.defense_evasion
|
||||
- attack.t1562.001
|
||||
```
|
||||
|
||||
**Volume:** ~3,000 detection rules
|
||||
**Frequency:** Monthly recommended
|
||||
|
||||
**Troubleshooting:**
|
||||
- Rules without MITRE technique tags in `attack.tXXXX` format are skipped
|
||||
- Duplicate detection is by `name` + `source` + `mitre_technique_id`
|
||||
|
||||
---
|
||||
|
||||
## Elastic Detection Rules
|
||||
|
||||
**Repository:** https://github.com/elastic/detection-rules
|
||||
**Format:** TOML (one file per rule under `rules/`)
|
||||
**Service:** `elastic_import_service.py`
|
||||
|
||||
**Extracted fields:**
|
||||
- `name` — Rule name from `[rule]`
|
||||
- `description` — Rule description
|
||||
- `query` — KQL/EQL query
|
||||
- `mitre_technique_id` — From `[[rule.threat]]` entries
|
||||
- `severity` — From `rule.severity`
|
||||
- `rule_type` — eql, query, threshold, etc.
|
||||
- `source` = `"elastic"`
|
||||
|
||||
**TOML structure:**
|
||||
```toml
|
||||
[rule]
|
||||
name = "Scheduled Task Created via Schtasks"
|
||||
severity = "medium"
|
||||
type = "eql"
|
||||
|
||||
[[rule.threat]]
|
||||
framework = "MITRE ATT&CK"
|
||||
[[rule.threat.technique]]
|
||||
id = "T1053"
|
||||
name = "Scheduled Task/Job"
|
||||
[[rule.threat.technique.subtechnique]]
|
||||
id = "T1053.005"
|
||||
```
|
||||
|
||||
**Volume:** ~1,200 detection rules
|
||||
**Frequency:** Quarterly recommended
|
||||
|
||||
---
|
||||
|
||||
## CALDERA (Test Templates)
|
||||
|
||||
**Repository:** https://github.com/mitre/caldera
|
||||
**Format:** YAML (multi-document, abilities under `data/abilities/`)
|
||||
**Service:** `caldera_import_service.py`
|
||||
|
||||
**Extracted fields:**
|
||||
- `name` — Ability name
|
||||
- `description` — What the ability does
|
||||
- `mitre_technique_id` — From `technique.attack_id`
|
||||
- `tactic` — ATT&CK tactic
|
||||
- `platforms` — Target platforms
|
||||
- `attack_commands` — Commands per platform/executor
|
||||
|
||||
**Volume:** ~500 abilities
|
||||
**Frequency:** Quarterly recommended
|
||||
|
||||
---
|
||||
|
||||
## LOLBAS (Test Templates)
|
||||
|
||||
**Repository:** https://github.com/LOLBAS-Project/LOLBAS
|
||||
**Format:** YAML (one file per binary under `yml/OSBinaries/`, `yml/OtherMSBinaries/`, etc.)
|
||||
**Service:** `lolbas_import_service.py`
|
||||
|
||||
**Extracted fields:**
|
||||
- `name` — Binary name (e.g., Mshta.exe)
|
||||
- `mitre_technique_id` — From `Commands[].MitreID`
|
||||
- `attack_commands` — From `Commands[].Command`
|
||||
- `description` — From `Commands[].Description`
|
||||
- `usecase` — From `Commands[].Usecase`
|
||||
|
||||
**Volume:** ~200 living-off-the-land binaries with ~500 commands
|
||||
**Frequency:** Quarterly recommended
|
||||
|
||||
---
|
||||
|
||||
## MITRE D3FEND (Defensive Techniques)
|
||||
|
||||
**Repository:** https://d3fend.mitre.org/
|
||||
**Format:** JSON-LD (REST API at `https://d3fend.mitre.org/api/`)
|
||||
**Service:** `d3fend_import_service.py`
|
||||
|
||||
**Extracted fields:**
|
||||
- `d3fend_id` — D3FEND identifier (e.g., D3-AL, D3-NI)
|
||||
- `name` — Defensive technique name
|
||||
- `description` — Definition or comment
|
||||
- `tactic` — Defensive tactic (Detect, Isolate, Deceive, Evict, Harden)
|
||||
- ATT&CK ↔ D3FEND mappings stored in `defensive_technique_mappings`
|
||||
|
||||
**Volume:** ~200 defensive techniques
|
||||
**Frequency:** Annually (D3FEND updates are infrequent)
|
||||
|
||||
---
|
||||
|
||||
## MITRE CTI — Threat Actors
|
||||
|
||||
**Repository:** https://github.com/mitre/cti (enterprise-attack)
|
||||
**Format:** STIX 2.0 JSON bundles (`intrusion-set`, `relationship`, `attack-pattern`)
|
||||
**Service:** `threat_actor_import_service.py`
|
||||
|
||||
**Extracted fields:**
|
||||
- `name` — Actor name (e.g., APT28)
|
||||
- `mitre_id` — MITRE group ID (e.g., G0007)
|
||||
- `aliases` — Known aliases (JSONB array)
|
||||
- `description` — Full description
|
||||
- `country` — Attribution (when available)
|
||||
- `motivation` — Espionage, financial, etc.
|
||||
- `target_sectors` / `target_regions` — JSONB arrays
|
||||
- Technique mappings via `relationship` objects
|
||||
|
||||
**Volume:** ~140 threat actors with ~2,000 technique mappings
|
||||
**Frequency:** Quarterly recommended
|
||||
|
||||
---
|
||||
|
||||
## NIST 800-53 → ATT&CK (Compliance)
|
||||
|
||||
**Repository:** https://github.com/center-for-threat-informed-defense/attack-control-framework-mappings
|
||||
**Format:** STIX 2.0 JSON bundles
|
||||
**Service:** `compliance_import_service.py`
|
||||
|
||||
**Extracted fields:**
|
||||
- `ComplianceFramework` — Framework name and version
|
||||
- `ComplianceControl` — Control ID (e.g., AC-2), title, category
|
||||
- `ComplianceControlMapping` — Control ↔ ATT&CK technique associations
|
||||
|
||||
**Volume:** ~1,000 controls with ~5,000 mappings
|
||||
**Frequency:** Annually (mappings are versioned with the framework)
|
||||
|
||||
---
|
||||
|
||||
## Managing Data Sources
|
||||
|
||||
### Admin UI
|
||||
|
||||
Navigate to **System → Data Sources** in the Aegis frontend to:
|
||||
- View all configured data sources and their sync status
|
||||
- Trigger manual imports
|
||||
- Enable/disable individual sources
|
||||
- View import statistics (imported, updated, errors)
|
||||
|
||||
### API Endpoints
|
||||
|
||||
```bash
|
||||
# List data sources
|
||||
GET /api/v1/data-sources
|
||||
|
||||
# Trigger import for a specific source
|
||||
POST /api/v1/data-sources/{id}/sync
|
||||
|
||||
# Enable/disable a source
|
||||
PATCH /api/v1/data-sources/{id}
|
||||
```
|
||||
|
||||
### Recommended Update Schedule
|
||||
|
||||
| Source | Frequency | Reason |
|
||||
|--------|-----------|--------|
|
||||
| MITRE ATT&CK | Automatic (24h) | Core framework, frequent updates |
|
||||
| Atomic Red Team | Monthly | Active community contributions |
|
||||
| SigmaHQ | Monthly | Active community contributions |
|
||||
| Elastic Rules | Quarterly | Major version-aligned releases |
|
||||
| CALDERA | Quarterly | Less frequent updates |
|
||||
| LOLBAS | Quarterly | Less frequent updates |
|
||||
| D3FEND | Annually | Infrequent updates |
|
||||
| CTI Actors | Quarterly | New groups and campaigns |
|
||||
| NIST 800-53 | Annually | Framework revision cycles |
|
||||
285
docs/SCORING.md
Normal file
285
docs/SCORING.md
Normal file
@@ -0,0 +1,285 @@
|
||||
# Aegis — Scoring System
|
||||
|
||||
Aegis uses a granular 0–100 scoring system to measure security coverage at multiple levels: individual techniques, tactics, threat actors, and the overall organization.
|
||||
|
||||
---
|
||||
|
||||
## Technique Score (0–100)
|
||||
|
||||
Each ATT&CK technique receives a composite score based on five weighted components:
|
||||
|
||||
| Component | Default Weight | Description |
|
||||
|-----------|---------------|-------------|
|
||||
| Tests Validated | 40% | Ratio of detected tests to total validated tests |
|
||||
| Detection Rules | 20% | Number of active detection rules linked to the technique |
|
||||
| D3FEND Coverage | 15% | Number of D3FEND defensive techniques mapped |
|
||||
| Freshness | 15% | How recent the latest validated test is |
|
||||
| Platform Diversity | 10% | Coverage across different platforms (Windows, Linux, macOS) |
|
||||
|
||||
### Tests Validated Component
|
||||
|
||||
```
|
||||
score = (detected_tests / total_validated_tests) × weight
|
||||
```
|
||||
|
||||
- Only tests in `validated` state are counted
|
||||
- `detected` means `detection_result = "detected"`
|
||||
- Example: 2 detected out of 3 validated → `2/3 × 40 = 26.7`
|
||||
|
||||
### Detection Rules Component
|
||||
|
||||
```
|
||||
score = min(active_rules / 3, 1.0) × weight
|
||||
```
|
||||
|
||||
- Counts active detection rules linked to the technique's `mitre_id`
|
||||
- 3+ rules gives full marks (capped at 1.0)
|
||||
- Example: 2 active rules → `2/3 × 20 = 13.3`
|
||||
|
||||
### D3FEND Coverage Component
|
||||
|
||||
```
|
||||
score = min(d3fend_mappings / 2, 1.0) × weight
|
||||
```
|
||||
|
||||
- Counts D3FEND defensive technique mappings
|
||||
- 2+ mappings gives full marks
|
||||
- Example: 1 mapping → `1/2 × 15 = 7.5`
|
||||
|
||||
### Freshness Component
|
||||
|
||||
```
|
||||
days = (now - newest_validated_test.red_validated_at).days
|
||||
score = max(0, 1.0 - days / 180) × weight
|
||||
```
|
||||
|
||||
- 0 days old = full freshness score
|
||||
- 180+ days old = 0 (completely stale)
|
||||
- Linear decay between 0 and 180 days
|
||||
- Example: test is 60 days old → `(1 - 60/180) × 15 = 10.0`
|
||||
|
||||
### Platform Diversity Component
|
||||
|
||||
```
|
||||
platforms_covered = unique platforms across validated tests
|
||||
score = min(platforms_covered / 3, 1.0) × weight
|
||||
```
|
||||
|
||||
- Counts unique platforms (windows, linux, macos) from validated tests
|
||||
- 3+ platforms gives full marks
|
||||
- Example: windows + linux → `2/3 × 10 = 6.7`
|
||||
|
||||
### Example Calculation
|
||||
|
||||
A technique with:
|
||||
- 2/3 tests detected, 2 detection rules, 1 D3FEND mapping, 60 days old, 2 platforms
|
||||
|
||||
```
|
||||
Tests: (2/3) × 40 = 26.7
|
||||
Detection: (2/3) × 20 = 13.3
|
||||
D3FEND: (1/2) × 15 = 7.5
|
||||
Freshness: (1 - 60/180) × 15 = 10.0
|
||||
Platform: (2/3) × 10 = 6.7
|
||||
─────
|
||||
Total: 64.2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuring Weights
|
||||
|
||||
Weights are configurable via environment variables or the admin API. They must sum to 100.
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```env
|
||||
SCORING_WEIGHT_TESTS=40
|
||||
SCORING_WEIGHT_DETECTION_RULES=20
|
||||
SCORING_WEIGHT_D3FEND=15
|
||||
SCORING_WEIGHT_FRESHNESS=15
|
||||
SCORING_WEIGHT_PLATFORM_DIVERSITY=10
|
||||
```
|
||||
|
||||
### API Configuration
|
||||
|
||||
```bash
|
||||
# Get current weights
|
||||
GET /api/v1/scores/config
|
||||
|
||||
# Update weights (admin only)
|
||||
PATCH /api/v1/scores/config
|
||||
{
|
||||
"tests": 50,
|
||||
"detection_rules": 20,
|
||||
"d3fend": 10,
|
||||
"freshness": 10,
|
||||
"platform_diversity": 10
|
||||
}
|
||||
```
|
||||
|
||||
Note: Runtime changes do not persist across restarts. Update the `.env` file or environment variables for permanent changes.
|
||||
|
||||
---
|
||||
|
||||
## Tactic Score
|
||||
|
||||
The tactic score is the **average** of all technique scores within that tactic:
|
||||
|
||||
```
|
||||
tactic_score = mean(technique_scores for techniques in tactic)
|
||||
```
|
||||
|
||||
Also provides:
|
||||
- `techniques_total` — number of techniques in the tactic
|
||||
- `techniques_evaluated` — techniques with score > 0
|
||||
- `techniques_by_status` — count by status (validated, partial, not_covered, not_evaluated)
|
||||
|
||||
### API
|
||||
|
||||
```bash
|
||||
GET /api/v1/scores/tactic/execution
|
||||
GET /api/v1/scores/tactic/persistence
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Threat Actor Coverage Score
|
||||
|
||||
Measures how well the organization is covered against a specific threat actor:
|
||||
|
||||
```
|
||||
actor_score = mean(technique_scores for techniques used by actor)
|
||||
```
|
||||
|
||||
Also provides:
|
||||
- `techniques_total` — techniques attributed to the actor
|
||||
- `techniques_covered` — techniques with score > 0
|
||||
- `coverage_percentage` — percentage of techniques covered
|
||||
- `uncovered_techniques` — list of technique IDs with score = 0
|
||||
|
||||
### API
|
||||
|
||||
```bash
|
||||
GET /api/v1/scores/threat-actor/{actor_id}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Organization Score
|
||||
|
||||
The top-level organizational security score is a weighted average of four sub-scores:
|
||||
|
||||
| Sub-score | Weight | Description |
|
||||
|-----------|--------|-------------|
|
||||
| Total Coverage | 40% | Average technique score across all evaluated techniques |
|
||||
| Critical Coverage | 25% | Average score for techniques with high/critical severity templates |
|
||||
| Detection Maturity | 20% | `(triggered_rules / total_active_rules) × 100` |
|
||||
| Response Readiness | 15% | `(remediation_completed / remediation_total) × 100` |
|
||||
|
||||
```
|
||||
org_score = total_coverage × 0.4
|
||||
+ critical_coverage × 0.25
|
||||
+ detection_maturity × 0.2
|
||||
+ response_readiness × 0.15
|
||||
```
|
||||
|
||||
### Caching
|
||||
|
||||
The organization score is cached in-memory for 5 minutes. The cache is automatically invalidated when:
|
||||
- A test is validated (state → `validated`)
|
||||
- Scoring weights are updated via the API
|
||||
|
||||
### API
|
||||
|
||||
```bash
|
||||
GET /api/v1/scores/organization
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Operational Metrics
|
||||
|
||||
In addition to coverage scores, Aegis tracks operational KPIs:
|
||||
|
||||
### Mean Time to Detect (MTTD)
|
||||
|
||||
Time from test execution start (`start_execution` audit entry) to red team submission (`submit_red`).
|
||||
|
||||
```
|
||||
MTTD = mean(submit_red.timestamp - start_execution.timestamp) for all tests
|
||||
```
|
||||
|
||||
### Mean Time to Respond (MTTR)
|
||||
|
||||
Time from blue team evaluation (`blue_validated_at`) to remediation completion (`update_remediation` audit entry).
|
||||
|
||||
```
|
||||
MTTR = mean(update_remediation.timestamp - blue_validated_at) for remediated tests
|
||||
```
|
||||
|
||||
### Detection Efficacy
|
||||
|
||||
```
|
||||
efficacy = (detected_tests / total_validated_tests) × 100
|
||||
```
|
||||
|
||||
### Alert Fidelity
|
||||
|
||||
Ratio of true positive detections to total detection rule evaluations.
|
||||
|
||||
### Coverage Velocity
|
||||
|
||||
Rate at which new techniques are being covered over time (techniques covered per week).
|
||||
|
||||
### Validation Throughput
|
||||
|
||||
Number of tests moving through the pipeline per time period.
|
||||
|
||||
### Rejection Rate
|
||||
|
||||
Percentage of tests rejected during dual validation.
|
||||
|
||||
### API
|
||||
|
||||
```bash
|
||||
# All operational metrics
|
||||
GET /api/v1/metrics/operational
|
||||
|
||||
# Weekly trend data
|
||||
GET /api/v1/metrics/operational/trend?period=90d
|
||||
|
||||
# Breakdown by team
|
||||
GET /api/v1/metrics/operational/by-team
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Score History
|
||||
|
||||
Weekly score snapshots for trend analysis:
|
||||
|
||||
```bash
|
||||
GET /api/v1/scores/history?period=90d
|
||||
# Returns weekly data points with: date, overall_score, total_coverage,
|
||||
# critical_coverage, detection_maturity, response_readiness
|
||||
```
|
||||
|
||||
Periods: `30d`, `90d`, `1y`
|
||||
|
||||
---
|
||||
|
||||
## Coverage Snapshots
|
||||
|
||||
Point-in-time captures of the complete coverage state for historical comparison:
|
||||
|
||||
```bash
|
||||
# Create a snapshot
|
||||
POST /api/v1/snapshots
|
||||
{ "name": "Q1 2026 Baseline" }
|
||||
|
||||
# Compare two snapshots
|
||||
GET /api/v1/snapshots/compare?a={snapshot_id_a}&b={snapshot_id_b}
|
||||
# Returns: score_delta, improved techniques, worsened techniques, unchanged count
|
||||
```
|
||||
|
||||
Automatic weekly snapshots are created every Sunday at 00:00 by the scheduler, with old snapshots cleaned up to keep the last 52 (one year).
|
||||
Reference in New Issue
Block a user