384 lines
19 KiB
Markdown
384 lines
19 KiB
Markdown
# Aegis - MITRE ATT&CK Coverage Platform
|
||
|
||
Aegis is a comprehensive platform for tracking and managing security coverage against the MITRE ATT&CK framework. It enables security teams to document, validate, and visualize their defensive capabilities against known adversary techniques through a structured Red Team / Blue Team validation workflow.
|
||
|
||
## Features
|
||
|
||
- **MITRE ATT&CK Integration**: Automatic synchronization with the MITRE ATT&CK framework via TAXII (with GitHub fallback), scheduled every 24h
|
||
- **Red/Blue Validation Workflow**: Structured dual-validation lifecycle for security tests (draft → red_executing → blue_evaluating → in_review → validated/rejected)
|
||
- **Test Template Catalog**: Import tests from Atomic Red Team, create custom templates, and instantiate real tests from them
|
||
- **Dual Validation**: Independent approval/rejection by Red Lead and Blue Lead before a test is finalized
|
||
- **Coverage Tracking**: Track validation status for each technique (validated, partial, not covered, in progress)
|
||
- **Evidence Storage**: Secure evidence file storage with SHA256 integrity verification, separated by team (red/blue)
|
||
- **In-App Notifications**: Real-time notification bell with polling, automatic alerts on state changes
|
||
- **Reports & Export**: Coverage summary, test results, and remediation reports in JSON and CSV formats
|
||
- **Remediation Tracking**: Step-by-step remediation assignments with status tracking per test
|
||
- **Role-Based Access Control**: Granular permissions for red team, blue team, and leadership roles
|
||
- **Intel Monitoring**: Automated scanning for new threat intelligence related to techniques
|
||
- **Metrics Dashboard**: Pipeline funnel, team activity, validation rates, and recent tests
|
||
|
||
## Red Team / Blue Team Validation Flow
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────────────┐
|
||
│ TEST LIFECYCLE │
|
||
│ │
|
||
│ ┌──────┐ ┌──────────────┐ ┌─────────────────┐ ┌───────────┐ │
|
||
│ │ DRAFT│───▶│RED_EXECUTING │───▶│ BLUE_EVALUATING │───▶│ IN_REVIEW │ │
|
||
│ └──────┘ └──────────────┘ └─────────────────┘ └───────────┘ │
|
||
│ │ │
|
||
│ ┌────────────────────┤ │
|
||
│ ▼ ▼ │
|
||
│ ┌──────────┐ ┌──────────┐ │
|
||
│ │ REJECTED │ │VALIDATED │ │
|
||
│ └──────────┘ └──────────┘ │
|
||
│ │ │
|
||
│ └──────▶ Back to DRAFT │
|
||
└─────────────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### States
|
||
|
||
| State | Description | Who acts |
|
||
|-------|-------------|----------|
|
||
| `draft` | Created, pending execution | Red Tech |
|
||
| `red_executing` | Red Team documents attack & uploads evidence | Red Tech |
|
||
| `blue_evaluating` | Blue Team documents detection & uploads evidence | Blue Tech |
|
||
| `in_review` | Both managers review evidence | Red Lead, Blue Lead |
|
||
| `validated` | Approved by both managers | — (terminal) |
|
||
| `rejected` | Rejected — returns to draft for redo | Red/Blue Lead can reopen |
|
||
|
||
### Dual Validation
|
||
|
||
Both Red Lead and Blue Lead must independently vote:
|
||
- **Both approve** → test moves to `validated`
|
||
- **Either rejects** → test moves to `rejected`
|
||
- **One votes, other pending** → stays in `in_review`
|
||
|
||
## User Roles
|
||
|
||
| Role | Description | Capabilities |
|
||
|------|-------------|-------------|
|
||
| `admin` | Full system access | Everything |
|
||
| `red_tech` | Red team technician | Create tests, document attacks, upload red evidence |
|
||
| `blue_tech` | Blue team technician | Document detection, upload blue evidence |
|
||
| `red_lead` | Red team lead | Validate/reject the red side of tests |
|
||
| `blue_lead` | Blue team lead | Validate/reject the blue side of tests |
|
||
| `viewer` | Read-only access | View all data |
|
||
|
||
## Test Template Catalog
|
||
|
||
Tests can be created from predefined templates sourced from:
|
||
1. **Atomic Red Team** (Red Canary) — imported via the System admin panel
|
||
2. **Custom templates** — created by admins with suggested procedures and remediation
|
||
3. **MITRE procedures** — based on MITRE ATT&CK documentation
|
||
|
||
Templates include attack procedures, expected detections, suggested tools, severity levels, and suggested remediation steps. When instantiated, these fields are pre-populated into the new test.
|
||
|
||
## Tech Stack
|
||
|
||
- **Backend**: FastAPI (Python 3.11)
|
||
- **Database**: PostgreSQL 15
|
||
- **Object Storage**: MinIO (S3-compatible)
|
||
- **ORM**: SQLAlchemy with Alembic migrations
|
||
- **Frontend**: React 19 + TypeScript + Vite + Tailwind CSS v4 + TanStack Query
|
||
- **Scheduler**: APScheduler (MITRE sync, Intel scan, Notification cleanup)
|
||
|
||
## Quick Start
|
||
|
||
### Prerequisites
|
||
|
||
- Docker and Docker Compose
|
||
- Git
|
||
|
||
### Installation
|
||
|
||
1. Clone the repository:
|
||
```bash
|
||
git clone <repository-url>
|
||
cd Aegis
|
||
```
|
||
|
||
2. Start all services:
|
||
```bash
|
||
docker-compose up -d
|
||
```
|
||
|
||
3. Run database migrations:
|
||
```bash
|
||
docker exec -w /app aegis-backend-1 alembic upgrade head
|
||
```
|
||
|
||
4. Seed the admin user:
|
||
```bash
|
||
docker exec -w /app aegis-backend-1 python -m app.seed
|
||
```
|
||
|
||
5. Start the frontend:
|
||
```bash
|
||
cd frontend && npm install && npm run dev
|
||
```
|
||
|
||
6. Verify the installation:
|
||
```bash
|
||
curl http://localhost:8000/health
|
||
# Expected: {"status":"ok"}
|
||
|
||
# Open http://localhost:5173 — Aegis login page
|
||
```
|
||
|
||
### Authentication
|
||
|
||
JWT-based authentication. Default admin credentials after seeding:
|
||
|
||
```bash
|
||
curl -X POST http://localhost:8000/api/v1/auth/login \
|
||
-d "username=admin&password=admin123"
|
||
```
|
||
|
||
> **Important:** Change the default `admin123` password and `SECRET_KEY` in production.
|
||
|
||
## Services
|
||
|
||
| Service | Port | Description |
|
||
|---------|------|-------------|
|
||
| Frontend | 5173 | React dev server (Vite) |
|
||
| Backend | 8000 | FastAPI REST API |
|
||
| PostgreSQL | 5433 | Database |
|
||
| MinIO API | 9000 | S3-compatible object storage |
|
||
| MinIO Console | 9001 | MinIO web interface |
|
||
|
||
## API Documentation
|
||
|
||
Interactive API documentation available at:
|
||
|
||
- **Swagger UI**: http://localhost:8000/docs
|
||
- **ReDoc**: http://localhost:8000/redoc
|
||
|
||
## API Endpoints
|
||
|
||
### Auth
|
||
| Method | Route | Auth | Description |
|
||
|--------|-------|------|-------------|
|
||
| POST | `/api/v1/auth/login` | Public | Obtain JWT token |
|
||
| GET | `/api/v1/auth/me` | Authenticated | Current user profile |
|
||
|
||
### Techniques
|
||
| Method | Route | Auth | Description |
|
||
|--------|-------|------|-------------|
|
||
| GET | `/api/v1/techniques` | Authenticated | List all (filters: tactic, status, review_required) |
|
||
| GET | `/api/v1/techniques/{mitre_id}` | Authenticated | Detail with associated tests |
|
||
| POST | `/api/v1/techniques` | Admin | Create technique |
|
||
| PATCH | `/api/v1/techniques/{mitre_id}` | Admin | Update technique fields |
|
||
| PATCH | `/api/v1/techniques/{mitre_id}/review` | Lead, Admin | Mark as reviewed |
|
||
|
||
### Tests — Red/Blue Workflow
|
||
| Method | Route | Auth | Description |
|
||
|--------|-------|------|-------------|
|
||
| GET | `/api/v1/tests` | Authenticated | List with filters (state, technique, platform, creator, pending_validation_side) |
|
||
| POST | `/api/v1/tests` | Red Tech, Admin | Create test (state=draft) |
|
||
| POST | `/api/v1/tests/from-template` | Red Tech, Admin | Create from template (pre-populates fields) |
|
||
| GET | `/api/v1/tests/{id}` | Authenticated | Detail with split red/blue evidences |
|
||
| PATCH | `/api/v1/tests/{id}` | Creator, Admin | General update (draft/rejected only) |
|
||
| PATCH | `/api/v1/tests/{id}/red` | Red Tech, Admin | Red Team fields (draft, red_executing) |
|
||
| PATCH | `/api/v1/tests/{id}/blue` | Blue Tech, Admin | Blue Team fields (blue_evaluating) |
|
||
| PATCH | `/api/v1/tests/{id}/remediation` | Authenticated | Update remediation fields |
|
||
| POST | `/api/v1/tests/{id}/start-execution` | Red Tech, Admin | draft → red_executing |
|
||
| POST | `/api/v1/tests/{id}/submit-red` | Red Tech, Admin | red_executing → blue_evaluating |
|
||
| POST | `/api/v1/tests/{id}/submit-blue` | Blue Tech, Admin | blue_evaluating → in_review |
|
||
| POST | `/api/v1/tests/{id}/validate-red` | Red Lead, Admin | Red Lead approves/rejects |
|
||
| POST | `/api/v1/tests/{id}/validate-blue` | Blue Lead, Admin | Blue Lead approves/rejects |
|
||
| POST | `/api/v1/tests/{id}/reopen` | Lead, Admin | rejected → draft (clears validation) |
|
||
| GET | `/api/v1/tests/{id}/timeline` | Authenticated | Audit-log history for this test |
|
||
|
||
### Test Templates
|
||
| Method | Route | Auth | Description |
|
||
|--------|-------|------|-------------|
|
||
| GET | `/api/v1/test-templates` | Authenticated | List templates (filters: source, platform, severity, search, mitre_technique_id) |
|
||
| POST | `/api/v1/test-templates` | Admin | Create custom template |
|
||
| GET | `/api/v1/test-templates/stats` | Admin | Catalog statistics |
|
||
| GET | `/api/v1/test-templates/{id}` | Authenticated | Template detail |
|
||
| PATCH | `/api/v1/test-templates/{id}` | Admin | Update template |
|
||
| DELETE | `/api/v1/test-templates/{id}` | Admin | Soft-delete (deactivate) |
|
||
| POST | `/api/v1/test-templates/{id}/toggle-active` | Admin | Toggle active/inactive |
|
||
|
||
### Evidence
|
||
| Method | Route | Auth | Description |
|
||
|--------|-------|------|-------------|
|
||
| POST | `/api/v1/tests/{test_id}/evidence` | Authenticated | Upload evidence (team=red/blue) |
|
||
| GET | `/api/v1/evidence/{id}` | Authenticated | Metadata + presigned download URL |
|
||
|
||
### Notifications
|
||
| Method | Route | Auth | Description |
|
||
|--------|-------|------|-------------|
|
||
| GET | `/api/v1/notifications` | Authenticated | List notifications (paginated, limit=20) |
|
||
| GET | `/api/v1/notifications/unread-count` | Authenticated | Unread notification count |
|
||
| PATCH | `/api/v1/notifications/{id}/read` | Authenticated | Mark one as read |
|
||
| POST | `/api/v1/notifications/read-all` | Authenticated | Mark all as read |
|
||
|
||
### Reports
|
||
| Method | Route | Auth | Description |
|
||
|--------|-------|------|-------------|
|
||
| GET | `/api/v1/reports/coverage-summary` | Authenticated | Full coverage JSON report (filters: tactic, platform) |
|
||
| GET | `/api/v1/reports/coverage-csv` | Authenticated | CSV export of coverage |
|
||
| GET | `/api/v1/reports/test-results` | Authenticated | Test results report (filters: state, date_from, date_to) |
|
||
| GET | `/api/v1/reports/remediation-status` | Authenticated | Remediation status report (filter: status) |
|
||
|
||
### Metrics
|
||
| Method | Route | Auth | Description |
|
||
|--------|-------|------|-------------|
|
||
| GET | `/api/v1/metrics/summary` | Authenticated | Global coverage summary |
|
||
| GET | `/api/v1/metrics/by-tactic` | Authenticated | Coverage by MITRE tactic |
|
||
| GET | `/api/v1/metrics/test-pipeline` | Authenticated | Test counts by pipeline state |
|
||
| GET | `/api/v1/metrics/team-activity` | Authenticated | Red/Blue team activity |
|
||
| GET | `/api/v1/metrics/validation-rate` | Authenticated | Approval/rejection rates by lead |
|
||
| GET | `/api/v1/metrics/recent-tests` | Authenticated | Last 10 updated tests |
|
||
|
||
### System (Admin)
|
||
| Method | Route | Auth | Description |
|
||
|--------|-------|------|-------------|
|
||
| POST | `/api/v1/system/sync-mitre` | Admin | Trigger MITRE ATT&CK sync |
|
||
| POST | `/api/v1/system/run-intel-scan` | Admin | Trigger threat-intel RSS scan |
|
||
| POST | `/api/v1/system/import-atomic-red-team` | Admin | Import Atomic Red Team templates |
|
||
| GET | `/api/v1/system/scheduler-status` | Admin | Background scheduler health |
|
||
|
||
### Users (Admin)
|
||
| Method | Route | Auth | Description |
|
||
|--------|-------|------|-------------|
|
||
| GET | `/api/v1/users` | Admin | List all users |
|
||
| POST | `/api/v1/users` | Admin | Create new user |
|
||
| GET | `/api/v1/users/{id}` | Admin | Get user by ID |
|
||
| PATCH | `/api/v1/users/{id}` | Admin | Update user |
|
||
|
||
### Audit Logs (Admin)
|
||
| Method | Route | Auth | Description |
|
||
|--------|-------|------|-------------|
|
||
| GET | `/api/v1/audit-logs` | Admin | List audit logs (filters: action, entity_type, dates) |
|
||
| GET | `/api/v1/audit-logs/actions` | Admin | List distinct action types |
|
||
| GET | `/api/v1/audit-logs/entity-types` | Admin | List distinct entity types |
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
Aegis/
|
||
├── docker-compose.yml
|
||
├── backend/
|
||
│ ├── Dockerfile
|
||
│ ├── requirements.txt
|
||
│ ├── alembic.ini
|
||
│ ├── alembic/versions/ # b001–b007 migration files
|
||
│ └── app/
|
||
│ ├── main.py # FastAPI app with all routers
|
||
│ ├── config.py # Settings from environment
|
||
│ ├── database.py # SQLAlchemy engine + session
|
||
│ ├── storage.py # MinIO/S3 helpers
|
||
│ ├── models/
|
||
│ │ ├── user.py # User with roles
|
||
│ │ ├── technique.py # MITRE ATT&CK techniques
|
||
│ │ ├── test.py # Tests with Red/Blue + remediation fields
|
||
│ │ ├── test_template.py # Template catalog
|
||
│ │ ├── evidence.py # Evidence files (team-separated)
|
||
│ │ ├── notification.py # In-app notifications
|
||
│ │ ├── intel.py # Threat intelligence
|
||
│ │ ├── audit.py # Audit logging
|
||
│ │ └── enums.py # Shared enumerations
|
||
│ ├── schemas/ # Pydantic schemas
|
||
│ │ ├── test.py # TestCreate/Red/Blue/Validate/Remediation
|
||
│ │ ├── test_template.py # Template CRUD schemas
|
||
│ │ ├── notification.py # NotificationOut, UnreadCountOut
|
||
│ │ └── metrics.py # Pipeline, TeamActivity, ValidationRate
|
||
│ ├── routers/ # API endpoints
|
||
│ │ ├── tests.py # Full Red/Blue workflow endpoints
|
||
│ │ ├── test_templates.py # Template CRUD + import + stats
|
||
│ │ ├── notifications.py # Notification list/read/mark
|
||
│ │ ├── reports.py # Coverage/results/remediation reports
|
||
│ │ ├── metrics.py # V1 + V2 metrics endpoints
|
||
│ │ └── ... # auth, techniques, evidence, system, users, audit
|
||
│ ├── services/
|
||
│ │ ├── test_workflow_service.py # State machine + dual validation
|
||
│ │ ├── notification_service.py # Create/read/cleanup notifications
|
||
│ │ ├── status_service.py # Technique status recalculation
|
||
│ │ └── ... # audit, mitre_sync, intel
|
||
│ └── jobs/
|
||
│ └── mitre_sync_job.py # Scheduler: MITRE sync, Intel scan, Notification cleanup
|
||
├── frontend/src/
|
||
│ ├── App.tsx # Routes including /reports
|
||
│ ├── api/ # API clients
|
||
│ │ ├── notifications.ts # Notification API
|
||
│ │ ├── reports.ts # Report API
|
||
│ │ └── ...
|
||
│ ├── components/
|
||
│ │ ├── Layout.tsx # Sidebar + header + NotificationBell
|
||
│ │ ├── Sidebar.tsx # Collapsible nav with admin section
|
||
│ │ ├── NotificationBell.tsx # Bell icon with badge (polls every 30s)
|
||
│ │ ├── NotificationDropdown.tsx # Notification list dropdown
|
||
│ │ ├── ConfirmDialog.tsx # Reusable confirmation modal
|
||
│ │ ├── Toast.tsx # Toast notification system
|
||
│ │ └── test-detail/ # Test detail sub-components
|
||
│ └── pages/
|
||
│ ├── DashboardPage.tsx # Pipeline funnel, team activity, validation rates
|
||
│ ├── TestsPage.tsx # Filters, state counters, pending tasks
|
||
│ ├── TestDetailPage.tsx # Red/Blue tabs, validation, evidence
|
||
│ ├── TestCatalogPage.tsx # Browse & use templates
|
||
│ ├── ReportsPage.tsx # Coverage, results, remediation reports
|
||
│ └── SystemPage.tsx # Template admin, import Atomic Red Team
|
||
└── backend/tests/ # Test suite
|
||
├── test_workflow.py # Red/Blue workflow tests
|
||
├── test_templates_crud.py # Template CRUD tests
|
||
├── test_metrics_v2.py # V2 metrics tests
|
||
└── test_integration_v2.py # Full integration E2E tests
|
||
```
|
||
|
||
## Database Schema
|
||
|
||
| Table | Description |
|
||
|-------|-------------|
|
||
| `users` | User accounts with role-based access |
|
||
| `techniques` | MITRE ATT&CK techniques with coverage status |
|
||
| `tests` | Security tests with Red/Blue fields, dual validation, and remediation |
|
||
| `test_templates` | Predefined test catalog (Atomic Red Team, custom) |
|
||
| `evidences` | File evidence separated by team (red/blue) |
|
||
| `notifications` | In-app notifications with read status |
|
||
| `intel_items` | Threat intelligence items linked to techniques |
|
||
| `audit_logs` | System-wide audit trail |
|
||
|
||
## Configuration
|
||
|
||
| Variable | Default | Description |
|
||
|----------|---------|-------------|
|
||
| `DATABASE_URL` | `postgresql://postgres:postgres@postgres:5432/attackdb` | PostgreSQL connection |
|
||
| `SECRET_KEY` | `change-me-in-production` | JWT signing key |
|
||
| `ALGORITHM` | `HS256` | JWT signing algorithm |
|
||
| `ACCESS_TOKEN_EXPIRE_MINUTES` | `60` | Token lifetime |
|
||
| `MINIO_ENDPOINT` | `minio:9000` | MinIO server |
|
||
| `MINIO_ACCESS_KEY` | `minioadmin` | MinIO access key |
|
||
| `MINIO_SECRET_KEY` | `minioadmin` | MinIO secret key |
|
||
| `MINIO_BUCKET` | `evidence` | Evidence bucket |
|
||
|
||
## Development
|
||
|
||
### Running Migrations
|
||
|
||
```bash
|
||
docker exec -w /app aegis-backend-1 alembic upgrade head
|
||
docker exec -w /app aegis-backend-1 alembic revision --autogenerate -m "description"
|
||
docker exec -w /app aegis-backend-1 alembic downgrade -1
|
||
```
|
||
|
||
### Running Tests
|
||
|
||
```bash
|
||
# Run standalone tests (no database required)
|
||
cd backend && python tests/test_workflow.py
|
||
cd backend && python tests/test_templates_crud.py
|
||
cd backend && python tests/test_metrics_v2.py
|
||
cd backend && python tests/test_integration_v2.py
|
||
|
||
# Run with pytest (requires PostgreSQL)
|
||
docker exec -w /app aegis-backend-1 pytest -v
|
||
```
|
||
|
||
## License
|
||
|
||
This project is proprietary software. All rights reserved.
|