feat(phase-33): final polish V3 - navigation, performance, and documentation (T-238 to T-240)
This commit is contained in:
478
README.md
478
README.md
@@ -1,40 +1,53 @@
|
||||
# Aegis - MITRE ATT&CK Coverage Platform
|
||||
# Aegis — MITRE ATT&CK Coverage Platform
|
||||
|
||||
Aegis is a comprehensive platform for tracking and managing security coverage against the MITRE ATT&CK framework. It enables security teams to document, validate, and visualize their defensive capabilities against known adversary techniques through a structured Red Team / Blue Team validation workflow.
|
||||
|
||||
## Features
|
||||
|
||||
- **MITRE ATT&CK Integration**: Automatic synchronization with the MITRE ATT&CK framework via TAXII (with GitHub fallback), scheduled every 24h
|
||||
- **Red/Blue Validation Workflow**: Structured dual-validation lifecycle for security tests (draft → red_executing → blue_evaluating → in_review → validated/rejected)
|
||||
- **Test Template Catalog**: Import tests from Atomic Red Team, create custom templates, and instantiate real tests from them
|
||||
- **Dual Validation**: Independent approval/rejection by Red Lead and Blue Lead before a test is finalized
|
||||
- **Coverage Tracking**: Track validation status for each technique (validated, partial, not covered, in progress)
|
||||
- **Evidence Storage**: Secure evidence file storage with SHA256 integrity verification, separated by team (red/blue)
|
||||
- **In-App Notifications**: Real-time notification bell with polling, automatic alerts on state changes
|
||||
- **Reports & Export**: Coverage summary, test results, and remediation reports in JSON and CSV formats
|
||||
- **Remediation Tracking**: Step-by-step remediation assignments with status tracking per test
|
||||
- **Role-Based Access Control**: Granular permissions for red team, blue team, and leadership roles
|
||||
- **Intel Monitoring**: Automated scanning for new threat intelligence related to techniques
|
||||
- **Metrics Dashboard**: Pipeline funnel, team activity, validation rates, and recent tests
|
||||
### Core (V1)
|
||||
- **MITRE ATT&CK Integration** — Automatic synchronization via TAXII 2.0 (with GitHub fallback), scheduled every 24h
|
||||
- **Red/Blue Validation Workflow** — Structured dual-validation lifecycle: draft → red_executing → blue_evaluating → in_review → validated/rejected
|
||||
- **Dual Validation** — Independent Red Lead / Blue Lead approval before finalization
|
||||
- **Coverage Tracking** — Per-technique status (validated, partial, not covered, in progress)
|
||||
- **Evidence Storage** — Secure evidence with SHA256 integrity, separated by team (red/blue)
|
||||
- **Role-Based Access Control** — Granular permissions for 6 roles (admin, red_tech, blue_tech, red_lead, blue_lead, viewer)
|
||||
|
||||
### Enhanced (V2)
|
||||
- **Test Template Catalog** — Import from Atomic Red Team, create custom templates, instantiate tests
|
||||
- **In-App Notifications** — Real-time notification bell with polling and automatic state-change alerts
|
||||
- **Reports & Export** — Coverage summary, test results, and remediation reports in JSON and CSV
|
||||
- **Remediation Tracking** — Step-by-step remediation assignments with status tracking
|
||||
- **Metrics Dashboard** — Pipeline funnel, team activity, validation rates
|
||||
|
||||
### Advanced (V3)
|
||||
- **Multi-Source Data Import** — Sigma, Elastic, CALDERA, LOLBAS, D3FEND, MITRE CTI threat actors, compliance mappings
|
||||
- **Detection Rule Tracking** — Import and evaluate Sigma/Elastic detection rules per test
|
||||
- **ATT&CK Heatmap** — Interactive Navigator-style heatmap with layers, filters, and export
|
||||
- **Threat Actor Intelligence** — Track intrusion sets and their technique coverage
|
||||
- **Campaign Management** — Group tests into campaigns with dependencies, scheduling, and recurring execution
|
||||
- **Compliance Mapping** — Map NIST 800-53 controls to ATT&CK techniques with gap analysis
|
||||
- **Granular Scoring** — 0–100 scoring for techniques, tactics, actors, and organization with configurable weights
|
||||
- **Operational Metrics** — MTTD, MTTR, detection efficacy, alert fidelity, coverage velocity
|
||||
- **Executive Dashboard** — High-level KPIs for leadership (leads + admin)
|
||||
- **Temporal Comparison** — Coverage snapshots with historical comparison and trend analysis
|
||||
- **Auto Re-testing** — Automatic retest creation after remediation completion (configurable limit)
|
||||
- **Performance Optimizations** — Score caching, lazy loading, React.memo, database index optimization
|
||||
|
||||
## Red Team / Blue Team Validation Flow
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ TEST LIFECYCLE │
|
||||
│ │
|
||||
│ ┌──────┐ ┌──────────────┐ ┌─────────────────┐ ┌───────────┐ │
|
||||
│ │ DRAFT│───▶│RED_EXECUTING │───▶│ BLUE_EVALUATING │───▶│ IN_REVIEW │ │
|
||||
│ └──────┘ └──────────────┘ └─────────────────┘ └───────────┘ │
|
||||
│ │ │
|
||||
│ ┌────────────────────┤ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ REJECTED │ │VALIDATED │ │
|
||||
│ └──────────┘ └──────────┘ │
|
||||
│ │ │
|
||||
│ └──────▶ Back to DRAFT │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
┌──────┐ ┌──────────────┐ ┌─────────────────┐ ┌───────────┐
|
||||
│ DRAFT│───▶│RED_EXECUTING │───▶│ BLUE_EVALUATING │───▶│ IN_REVIEW │
|
||||
└──────┘ └──────────────┘ └─────────────────┘ └─────┬─────┘
|
||||
│
|
||||
┌───────────────────┤
|
||||
▼ ▼
|
||||
┌──────────┐ ┌──────────┐
|
||||
│ REJECTED │ │VALIDATED │
|
||||
└────┬─────┘ └──────────┘
|
||||
│ │
|
||||
└──▶ Back to DRAFT ├──▶ Remediation
|
||||
└──▶ Auto Re-test
|
||||
```
|
||||
|
||||
### States
|
||||
@@ -66,23 +79,15 @@ Both Red Lead and Blue Lead must independently vote:
|
||||
| `blue_lead` | Blue team lead | Validate/reject the blue side of tests |
|
||||
| `viewer` | Read-only access | View all data |
|
||||
|
||||
## Test Template Catalog
|
||||
|
||||
Tests can be created from predefined templates sourced from:
|
||||
1. **Atomic Red Team** (Red Canary) — imported via the System admin panel
|
||||
2. **Custom templates** — created by admins with suggested procedures and remediation
|
||||
3. **MITRE procedures** — based on MITRE ATT&CK documentation
|
||||
|
||||
Templates include attack procedures, expected detections, suggested tools, severity levels, and suggested remediation steps. When instantiated, these fields are pre-populated into the new test.
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Backend**: FastAPI (Python 3.11)
|
||||
- **Database**: PostgreSQL 15
|
||||
- **Database**: PostgreSQL 15 with UUID primary keys and JSONB columns
|
||||
- **Object Storage**: MinIO (S3-compatible)
|
||||
- **ORM**: SQLAlchemy with Alembic migrations
|
||||
- **Frontend**: React 19 + TypeScript + Vite + Tailwind CSS v4 + TanStack Query
|
||||
- **Scheduler**: APScheduler (MITRE sync, Intel scan, Notification cleanup)
|
||||
- **ORM**: SQLAlchemy with Alembic migrations (18 migration files)
|
||||
- **Frontend**: React 19 + TypeScript + Vite 7 + Tailwind CSS v4 + TanStack Query + TanStack Virtual
|
||||
- **Scheduler**: APScheduler (MITRE sync, Intel scan, Notification cleanup, Snapshots, Recurring campaigns)
|
||||
- **Charts**: Recharts
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -106,21 +111,17 @@ docker-compose up -d
|
||||
|
||||
3. Run database migrations:
|
||||
```bash
|
||||
docker exec -w /app aegis-backend-1 alembic upgrade head
|
||||
docker exec aegis-backend alembic upgrade head
|
||||
```
|
||||
|
||||
4. Seed the admin user:
|
||||
```bash
|
||||
docker exec -w /app aegis-backend-1 python -m app.seed
|
||||
docker exec aegis-backend python -m app.seed
|
||||
```
|
||||
|
||||
5. Start the frontend:
|
||||
```bash
|
||||
cd frontend && npm install && npm run dev
|
||||
```
|
||||
|
||||
6. Verify the installation:
|
||||
5. Access the application:
|
||||
```bash
|
||||
# API health check
|
||||
curl http://localhost:8000/health
|
||||
# Expected: {"status":"ok"}
|
||||
|
||||
@@ -131,13 +132,51 @@ curl http://localhost:8000/health
|
||||
|
||||
JWT-based authentication. Default admin credentials after seeding:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/api/v1/auth/login \
|
||||
-d "username=admin&password=admin123"
|
||||
```
|
||||
Username: admin
|
||||
Password: admin123
|
||||
```
|
||||
|
||||
> **Important:** Change the default `admin123` password and `SECRET_KEY` in production.
|
||||
|
||||
### Importing Data Sources
|
||||
|
||||
After initial setup, populate the platform with data:
|
||||
|
||||
```bash
|
||||
# 1. Sync MITRE ATT&CK techniques
|
||||
curl -X POST http://localhost:8000/api/v1/system/sync-mitre -H "Authorization: Bearer $TOKEN"
|
||||
|
||||
# 2. Import test templates from Atomic Red Team
|
||||
curl -X POST http://localhost:8000/api/v1/system/import-atomic-red-team -H "Authorization: Bearer $TOKEN"
|
||||
|
||||
# 3. Import additional sources via the Data Sources admin page
|
||||
# Navigate to System → Data Sources in the UI
|
||||
```
|
||||
|
||||
See [docs/DATA_SOURCES.md](docs/DATA_SOURCES.md) for detailed instructions on all data sources.
|
||||
|
||||
### Configuring Scoring Weights
|
||||
|
||||
Default scoring weights can be adjusted via environment variables:
|
||||
|
||||
```env
|
||||
SCORING_WEIGHT_TESTS=40
|
||||
SCORING_WEIGHT_DETECTION_RULES=20
|
||||
SCORING_WEIGHT_D3FEND=15
|
||||
SCORING_WEIGHT_FRESHNESS=15
|
||||
SCORING_WEIGHT_PLATFORM_DIVERSITY=10
|
||||
```
|
||||
|
||||
Or at runtime via the admin API — see [docs/SCORING.md](docs/SCORING.md).
|
||||
|
||||
### Configuring Campaigns
|
||||
|
||||
1. Navigate to **Campaigns** in the sidebar
|
||||
2. Create a new campaign (custom or from a threat actor)
|
||||
3. Add tests with dependencies and phases
|
||||
4. Optionally enable **recurring scheduling** (weekly, monthly, quarterly)
|
||||
|
||||
## Services
|
||||
|
||||
| Service | Port | Description |
|
||||
@@ -148,6 +187,28 @@ curl -X POST http://localhost:8000/api/v1/auth/login \
|
||||
| MinIO API | 9000 | S3-compatible object storage |
|
||||
| MinIO Console | 9001 | MinIO web interface |
|
||||
|
||||
## Navigation
|
||||
|
||||
```
|
||||
📊 Dashboard
|
||||
📊 Executive Dashboard (leads + admin)
|
||||
🔲 ATT&CK Matrix (advanced heatmap)
|
||||
🧪 Tests
|
||||
├─ All Tests
|
||||
├─ My Pending Tasks
|
||||
└─ Test Catalog
|
||||
📋 Campaigns
|
||||
👤 Threat Actors
|
||||
📜 Compliance
|
||||
📈 Comparison (leads + admin)
|
||||
📄 Reports
|
||||
⚙️ System (admin only)
|
||||
├─ Data Sources
|
||||
├─ MITRE Sync
|
||||
├─ Users
|
||||
└─ Audit Log
|
||||
```
|
||||
|
||||
## API Documentation
|
||||
|
||||
Interactive API documentation available at:
|
||||
@@ -155,192 +216,33 @@ Interactive API documentation available at:
|
||||
- **Swagger UI**: http://localhost:8000/docs
|
||||
- **ReDoc**: http://localhost:8000/redoc
|
||||
|
||||
## API Endpoints
|
||||
### API Endpoints
|
||||
|
||||
### Auth
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| POST | `/api/v1/auth/login` | Public | Obtain JWT token |
|
||||
| GET | `/api/v1/auth/me` | Authenticated | Current user profile |
|
||||
| Group | Prefix | Key Operations |
|
||||
|-------|--------|---------------|
|
||||
| Auth | `/api/v1/auth` | Login, get current user |
|
||||
| Techniques | `/api/v1/techniques` | CRUD, list with filters, mark reviewed |
|
||||
| Tests | `/api/v1/tests` | Full Red/Blue workflow, remediation, retest chain |
|
||||
| Test Templates | `/api/v1/test-templates` | CRUD, import, stats, toggle active |
|
||||
| Evidence | `/api/v1/tests/{id}/evidence` | Upload evidence, get presigned URLs |
|
||||
| Campaigns | `/api/v1/campaigns` | CRUD, scheduling, history |
|
||||
| Threat Actors | `/api/v1/threat-actors` | CRUD, technique mappings |
|
||||
| Detection Rules | `/api/v1/detection-rules` | List, filter by source/technique |
|
||||
| D3FEND | `/api/v1/d3fend` | Defensive techniques and mappings |
|
||||
| Compliance | `/api/v1/compliance` | Frameworks, controls, gaps |
|
||||
| Scores | `/api/v1/scores` | Technique/tactic/actor/org scores, config |
|
||||
| Operational Metrics | `/api/v1/metrics/operational` | MTTD, MTTR, trends, team breakdown |
|
||||
| Heatmap | `/api/v1/heatmap` | ATT&CK Navigator-style data |
|
||||
| Snapshots | `/api/v1/snapshots` | Create, compare, list snapshots |
|
||||
| Reports | `/api/v1/reports` | Coverage, results, remediation exports |
|
||||
| Notifications | `/api/v1/notifications` | List, read, mark all read |
|
||||
| Metrics | `/api/v1/metrics` | Summary, by-tactic, pipeline, team activity |
|
||||
| System | `/api/v1/system` | MITRE sync, import, scheduler status |
|
||||
| Users | `/api/v1/users` | User CRUD (admin) |
|
||||
| Audit Logs | `/api/v1/audit-logs` | Audit trail (admin) |
|
||||
| Data Sources | `/api/v1/data-sources` | Data source management (admin) |
|
||||
|
||||
### Techniques
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/techniques` | Authenticated | List all (filters: tactic, status, review_required) |
|
||||
| GET | `/api/v1/techniques/{mitre_id}` | Authenticated | Detail with associated tests |
|
||||
| POST | `/api/v1/techniques` | Admin | Create technique |
|
||||
| PATCH | `/api/v1/techniques/{mitre_id}` | Admin | Update technique fields |
|
||||
| PATCH | `/api/v1/techniques/{mitre_id}/review` | Lead, Admin | Mark as reviewed |
|
||||
|
||||
### Tests — Red/Blue Workflow
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/tests` | Authenticated | List with filters (state, technique, platform, creator, pending_validation_side) |
|
||||
| POST | `/api/v1/tests` | Red Tech, Admin | Create test (state=draft) |
|
||||
| POST | `/api/v1/tests/from-template` | Red Tech, Admin | Create from template (pre-populates fields) |
|
||||
| GET | `/api/v1/tests/{id}` | Authenticated | Detail with split red/blue evidences |
|
||||
| PATCH | `/api/v1/tests/{id}` | Creator, Admin | General update (draft/rejected only) |
|
||||
| PATCH | `/api/v1/tests/{id}/red` | Red Tech, Admin | Red Team fields (draft, red_executing) |
|
||||
| PATCH | `/api/v1/tests/{id}/blue` | Blue Tech, Admin | Blue Team fields (blue_evaluating) |
|
||||
| PATCH | `/api/v1/tests/{id}/remediation` | Authenticated | Update remediation fields |
|
||||
| POST | `/api/v1/tests/{id}/start-execution` | Red Tech, Admin | draft → red_executing |
|
||||
| POST | `/api/v1/tests/{id}/submit-red` | Red Tech, Admin | red_executing → blue_evaluating |
|
||||
| POST | `/api/v1/tests/{id}/submit-blue` | Blue Tech, Admin | blue_evaluating → in_review |
|
||||
| POST | `/api/v1/tests/{id}/validate-red` | Red Lead, Admin | Red Lead approves/rejects |
|
||||
| POST | `/api/v1/tests/{id}/validate-blue` | Blue Lead, Admin | Blue Lead approves/rejects |
|
||||
| POST | `/api/v1/tests/{id}/reopen` | Lead, Admin | rejected → draft (clears validation) |
|
||||
| GET | `/api/v1/tests/{id}/timeline` | Authenticated | Audit-log history for this test |
|
||||
|
||||
### Test Templates
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/test-templates` | Authenticated | List templates (filters: source, platform, severity, search, mitre_technique_id) |
|
||||
| POST | `/api/v1/test-templates` | Admin | Create custom template |
|
||||
| GET | `/api/v1/test-templates/stats` | Admin | Catalog statistics |
|
||||
| GET | `/api/v1/test-templates/{id}` | Authenticated | Template detail |
|
||||
| PATCH | `/api/v1/test-templates/{id}` | Admin | Update template |
|
||||
| DELETE | `/api/v1/test-templates/{id}` | Admin | Soft-delete (deactivate) |
|
||||
| POST | `/api/v1/test-templates/{id}/toggle-active` | Admin | Toggle active/inactive |
|
||||
|
||||
### Evidence
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| POST | `/api/v1/tests/{test_id}/evidence` | Authenticated | Upload evidence (team=red/blue) |
|
||||
| GET | `/api/v1/evidence/{id}` | Authenticated | Metadata + presigned download URL |
|
||||
|
||||
### Notifications
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/notifications` | Authenticated | List notifications (paginated, limit=20) |
|
||||
| GET | `/api/v1/notifications/unread-count` | Authenticated | Unread notification count |
|
||||
| PATCH | `/api/v1/notifications/{id}/read` | Authenticated | Mark one as read |
|
||||
| POST | `/api/v1/notifications/read-all` | Authenticated | Mark all as read |
|
||||
|
||||
### Reports
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/reports/coverage-summary` | Authenticated | Full coverage JSON report (filters: tactic, platform) |
|
||||
| GET | `/api/v1/reports/coverage-csv` | Authenticated | CSV export of coverage |
|
||||
| GET | `/api/v1/reports/test-results` | Authenticated | Test results report (filters: state, date_from, date_to) |
|
||||
| GET | `/api/v1/reports/remediation-status` | Authenticated | Remediation status report (filter: status) |
|
||||
|
||||
### Metrics
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/metrics/summary` | Authenticated | Global coverage summary |
|
||||
| GET | `/api/v1/metrics/by-tactic` | Authenticated | Coverage by MITRE tactic |
|
||||
| GET | `/api/v1/metrics/test-pipeline` | Authenticated | Test counts by pipeline state |
|
||||
| GET | `/api/v1/metrics/team-activity` | Authenticated | Red/Blue team activity |
|
||||
| GET | `/api/v1/metrics/validation-rate` | Authenticated | Approval/rejection rates by lead |
|
||||
| GET | `/api/v1/metrics/recent-tests` | Authenticated | Last 10 updated tests |
|
||||
|
||||
### System (Admin)
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| POST | `/api/v1/system/sync-mitre` | Admin | Trigger MITRE ATT&CK sync |
|
||||
| POST | `/api/v1/system/run-intel-scan` | Admin | Trigger threat-intel RSS scan |
|
||||
| POST | `/api/v1/system/import-atomic-red-team` | Admin | Import Atomic Red Team templates |
|
||||
| GET | `/api/v1/system/scheduler-status` | Admin | Background scheduler health |
|
||||
|
||||
### Users (Admin)
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/users` | Admin | List all users |
|
||||
| POST | `/api/v1/users` | Admin | Create new user |
|
||||
| GET | `/api/v1/users/{id}` | Admin | Get user by ID |
|
||||
| PATCH | `/api/v1/users/{id}` | Admin | Update user |
|
||||
|
||||
### Audit Logs (Admin)
|
||||
| Method | Route | Auth | Description |
|
||||
|--------|-------|------|-------------|
|
||||
| GET | `/api/v1/audit-logs` | Admin | List audit logs (filters: action, entity_type, dates) |
|
||||
| GET | `/api/v1/audit-logs/actions` | Admin | List distinct action types |
|
||||
| GET | `/api/v1/audit-logs/entity-types` | Admin | List distinct entity types |
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
Aegis/
|
||||
├── docker-compose.yml
|
||||
├── backend/
|
||||
│ ├── Dockerfile
|
||||
│ ├── requirements.txt
|
||||
│ ├── alembic.ini
|
||||
│ ├── alembic/versions/ # b001–b007 migration files
|
||||
│ └── app/
|
||||
│ ├── main.py # FastAPI app with all routers
|
||||
│ ├── config.py # Settings from environment
|
||||
│ ├── database.py # SQLAlchemy engine + session
|
||||
│ ├── storage.py # MinIO/S3 helpers
|
||||
│ ├── models/
|
||||
│ │ ├── user.py # User with roles
|
||||
│ │ ├── technique.py # MITRE ATT&CK techniques
|
||||
│ │ ├── test.py # Tests with Red/Blue + remediation fields
|
||||
│ │ ├── test_template.py # Template catalog
|
||||
│ │ ├── evidence.py # Evidence files (team-separated)
|
||||
│ │ ├── notification.py # In-app notifications
|
||||
│ │ ├── intel.py # Threat intelligence
|
||||
│ │ ├── audit.py # Audit logging
|
||||
│ │ └── enums.py # Shared enumerations
|
||||
│ ├── schemas/ # Pydantic schemas
|
||||
│ │ ├── test.py # TestCreate/Red/Blue/Validate/Remediation
|
||||
│ │ ├── test_template.py # Template CRUD schemas
|
||||
│ │ ├── notification.py # NotificationOut, UnreadCountOut
|
||||
│ │ └── metrics.py # Pipeline, TeamActivity, ValidationRate
|
||||
│ ├── routers/ # API endpoints
|
||||
│ │ ├── tests.py # Full Red/Blue workflow endpoints
|
||||
│ │ ├── test_templates.py # Template CRUD + import + stats
|
||||
│ │ ├── notifications.py # Notification list/read/mark
|
||||
│ │ ├── reports.py # Coverage/results/remediation reports
|
||||
│ │ ├── metrics.py # V1 + V2 metrics endpoints
|
||||
│ │ └── ... # auth, techniques, evidence, system, users, audit
|
||||
│ ├── services/
|
||||
│ │ ├── test_workflow_service.py # State machine + dual validation
|
||||
│ │ ├── notification_service.py # Create/read/cleanup notifications
|
||||
│ │ ├── status_service.py # Technique status recalculation
|
||||
│ │ └── ... # audit, mitre_sync, intel
|
||||
│ └── jobs/
|
||||
│ └── mitre_sync_job.py # Scheduler: MITRE sync, Intel scan, Notification cleanup
|
||||
├── frontend/src/
|
||||
│ ├── App.tsx # Routes including /reports
|
||||
│ ├── api/ # API clients
|
||||
│ │ ├── notifications.ts # Notification API
|
||||
│ │ ├── reports.ts # Report API
|
||||
│ │ └── ...
|
||||
│ ├── components/
|
||||
│ │ ├── Layout.tsx # Sidebar + header + NotificationBell
|
||||
│ │ ├── Sidebar.tsx # Collapsible nav with admin section
|
||||
│ │ ├── NotificationBell.tsx # Bell icon with badge (polls every 30s)
|
||||
│ │ ├── NotificationDropdown.tsx # Notification list dropdown
|
||||
│ │ ├── ConfirmDialog.tsx # Reusable confirmation modal
|
||||
│ │ ├── Toast.tsx # Toast notification system
|
||||
│ │ └── test-detail/ # Test detail sub-components
|
||||
│ └── pages/
|
||||
│ ├── DashboardPage.tsx # Pipeline funnel, team activity, validation rates
|
||||
│ ├── TestsPage.tsx # Filters, state counters, pending tasks
|
||||
│ ├── TestDetailPage.tsx # Red/Blue tabs, validation, evidence
|
||||
│ ├── TestCatalogPage.tsx # Browse & use templates
|
||||
│ ├── ReportsPage.tsx # Coverage, results, remediation reports
|
||||
│ └── SystemPage.tsx # Template admin, import Atomic Red Team
|
||||
└── backend/tests/ # Test suite
|
||||
├── test_workflow.py # Red/Blue workflow tests
|
||||
├── test_templates_crud.py # Template CRUD tests
|
||||
├── test_metrics_v2.py # V2 metrics tests
|
||||
└── test_integration_v2.py # Full integration E2E tests
|
||||
```
|
||||
|
||||
## Database Schema
|
||||
|
||||
| Table | Description |
|
||||
|-------|-------------|
|
||||
| `users` | User accounts with role-based access |
|
||||
| `techniques` | MITRE ATT&CK techniques with coverage status |
|
||||
| `tests` | Security tests with Red/Blue fields, dual validation, and remediation |
|
||||
| `test_templates` | Predefined test catalog (Atomic Red Team, custom) |
|
||||
| `evidences` | File evidence separated by team (red/blue) |
|
||||
| `notifications` | In-app notifications with read status |
|
||||
| `intel_items` | Threat intelligence items linked to techniques |
|
||||
| `audit_logs` | System-wide audit trail |
|
||||
See [docs/API.md](docs/API.md) for the full endpoint reference.
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -354,30 +256,114 @@ Aegis/
|
||||
| `MINIO_ACCESS_KEY` | `minioadmin` | MinIO access key |
|
||||
| `MINIO_SECRET_KEY` | `minioadmin` | MinIO secret key |
|
||||
| `MINIO_BUCKET` | `evidence` | Evidence bucket |
|
||||
| `MAX_RETEST_COUNT` | `3` | Max automatic retests per original test |
|
||||
| `SCORING_WEIGHT_TESTS` | `40` | Weight for test validation component |
|
||||
| `SCORING_WEIGHT_DETECTION_RULES` | `20` | Weight for detection rules component |
|
||||
| `SCORING_WEIGHT_D3FEND` | `15` | Weight for D3FEND coverage component |
|
||||
| `SCORING_WEIGHT_FRESHNESS` | `15` | Weight for freshness component |
|
||||
| `SCORING_WEIGHT_PLATFORM_DIVERSITY` | `10` | Weight for platform diversity component |
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
Aegis/
|
||||
├── docker-compose.yml
|
||||
├── docker-compose.prod.yml
|
||||
├── docs/
|
||||
│ ├── API.md # Full API endpoint reference
|
||||
│ ├── ARCHITECTURE.md # System architecture and DB schema
|
||||
│ ├── DATA_SOURCES.md # External data source documentation
|
||||
│ └── SCORING.md # Scoring system and metrics
|
||||
├── backend/
|
||||
│ ├── Dockerfile
|
||||
│ ├── requirements.txt
|
||||
│ ├── alembic.ini
|
||||
│ ├── alembic/versions/ # b001–b018 migration files
|
||||
│ ├── pytest.ini
|
||||
│ └── app/
|
||||
│ ├── main.py # FastAPI app with all routers + lifespan
|
||||
│ ├── config.py # Settings from environment
|
||||
│ ├── database.py # SQLAlchemy engine + session (lazy init)
|
||||
│ ├── storage.py # MinIO/S3 helpers
|
||||
│ ├── auth.py # Password hashing + JWT tokens
|
||||
│ ├── models/ # 18 model files (SQLAlchemy ORM)
|
||||
│ ├── schemas/ # Pydantic request/response schemas
|
||||
│ ├── routers/ # 21 API routers
|
||||
│ ├── services/ # 20 business logic services
|
||||
│ ├── dependencies/ # Auth dependencies (get_current_user, require_role)
|
||||
│ └── jobs/
|
||||
│ └── mitre_sync_job.py # APScheduler: 5 background jobs
|
||||
├── frontend/src/
|
||||
│ ├── App.tsx # Routes with lazy loading + role protection
|
||||
│ ├── api/ # 22 API client modules (Axios + TanStack Query)
|
||||
│ ├── components/
|
||||
│ │ ├── Layout.tsx # Sidebar + header + NotificationBell
|
||||
│ │ ├── Sidebar.tsx # Role-aware collapsible navigation
|
||||
│ │ ├── heatmap/ # ATT&CK heatmap (6 components)
|
||||
│ │ ├── compliance/ # Compliance UI (gauge, controls table)
|
||||
│ │ └── test-detail/ # Test detail sub-components
|
||||
│ ├── hooks/
|
||||
│ │ └── useDebounce.ts # Debounce hook for search inputs
|
||||
│ ├── context/
|
||||
│ │ └── AuthContext.tsx # Auth state management
|
||||
│ └── pages/ # 21 page components
|
||||
└── backend/tests/
|
||||
├── conftest.py # SQLite test DB with JSONB/UUID compatibility
|
||||
├── fixtures/ # YAML/TOML/JSON test fixtures
|
||||
├── test_data_sources.py # Data source parsing tests
|
||||
├── test_scoring_and_compliance.py # Scoring + metrics + compliance tests
|
||||
├── test_campaigns_and_snapshots.py # Campaign, snapshot, and retest tests
|
||||
├── test_workflow.py # Red/Blue workflow tests
|
||||
├── test_templates_crud.py # Template CRUD tests
|
||||
├── test_metrics_v2.py # V2 metrics tests
|
||||
└── test_integration_v2.py # Full integration E2E tests
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Running Migrations
|
||||
|
||||
```bash
|
||||
docker exec -w /app aegis-backend-1 alembic upgrade head
|
||||
docker exec -w /app aegis-backend-1 alembic revision --autogenerate -m "description"
|
||||
docker exec -w /app aegis-backend-1 alembic downgrade -1
|
||||
docker exec aegis-backend alembic upgrade head
|
||||
docker exec aegis-backend alembic revision --autogenerate -m "description"
|
||||
docker exec aegis-backend alembic downgrade -1
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run standalone tests (no database required)
|
||||
cd backend && python tests/test_workflow.py
|
||||
cd backend && python tests/test_templates_crud.py
|
||||
cd backend && python tests/test_metrics_v2.py
|
||||
cd backend && python tests/test_integration_v2.py
|
||||
# Run all V3 tests inside the container (recommended)
|
||||
docker exec aegis-backend python -m pytest tests/ -v --tb=short
|
||||
|
||||
# Run with pytest (requires PostgreSQL)
|
||||
docker exec -w /app aegis-backend-1 pytest -v
|
||||
# Run specific test suites
|
||||
docker exec aegis-backend python -m pytest tests/test_data_sources.py -v
|
||||
docker exec aegis-backend python -m pytest tests/test_scoring_and_compliance.py -v
|
||||
docker exec aegis-backend python -m pytest tests/test_campaigns_and_snapshots.py -v
|
||||
|
||||
# Skip integration tests (require network)
|
||||
docker exec aegis-backend python -m pytest tests/ -v -m "not integration"
|
||||
```
|
||||
|
||||
### Generating Reports
|
||||
|
||||
```bash
|
||||
# Coverage summary (JSON)
|
||||
GET /api/v1/reports/coverage-summary
|
||||
|
||||
# Coverage CSV export
|
||||
GET /api/v1/reports/coverage-csv
|
||||
|
||||
# Compliance gap analysis
|
||||
GET /api/v1/compliance/{framework_id}/gaps
|
||||
```
|
||||
|
||||
## Further Documentation
|
||||
|
||||
- **[Architecture](docs/ARCHITECTURE.md)** — Database schema, service layer, state machine diagrams
|
||||
- **[Data Sources](docs/DATA_SOURCES.md)** — All external data sources with import instructions
|
||||
- **[Scoring](docs/SCORING.md)** — Scoring system explained with examples and configuration
|
||||
- **[API Reference](docs/API.md)** — Full endpoint documentation
|
||||
|
||||
## License
|
||||
|
||||
This project is proprietary software. All rights reserved.
|
||||
|
||||
Reference in New Issue
Block a user