Aegis/README.md

Aegis — MITRE ATT&CK Coverage Platform

Continuous integration (lint + tests against PostgreSQL and Redis) is defined in .github/workflows/ci.yml.

Aegis is a comprehensive platform for tracking and managing security coverage against the MITRE ATT&CK framework. It enables security teams to document, validate, and visualize their defensive capabilities against known adversary techniques through a structured Red Team / Blue Team validation workflow.

Features

Core (V1)

• MITRE ATT&CK Integration — Automatic synchronization via TAXII 2.0 (with GitHub fallback), scheduled every 24h
• Red/Blue Validation Workflow — Structured dual-validation lifecycle: draft → red_executing → blue_evaluating → in_review → validated/rejected
• Dual Validation — Independent Red Lead / Blue Lead approval before finalization
• Coverage Tracking — Per-technique status (validated, partial, not covered, in progress)
• Evidence Storage — Secure evidence with SHA256 integrity, separated by team (red/blue)
• Role-Based Access Control — Granular permissions for 6 roles (admin, red_tech, blue_tech, red_lead, blue_lead, viewer)

Enhanced (V2)

• Test Template Catalog — Import from Atomic Red Team, CALDERA, LOLBAS, GTFOBins; create custom templates; bulk activate/deactivate
• In-App Notifications — Real-time notification bell with polling and automatic state-change alerts
• Reports & Export — Coverage summary, test results, and remediation reports in JSON and CSV
• Remediation Tracking — Step-by-step remediation assignments with status tracking
• Metrics Dashboard — Pipeline funnel, team activity, validation rates

Advanced (V3)

• Multi-Source Data Import — Sigma, CALDERA, LOLBAS, GTFOBins, D3FEND, MITRE CTI threat actors, compliance mappings (NIST 800-53, CIS Controls v8)
• Detection Rule Tracking — Import and evaluate Sigma/Elastic detection rules per test
• ATT&CK Heatmap — Interactive Navigator-style heatmap with layers, filters, and export
• Threat Actor Intelligence — Track intrusion sets and their technique coverage
• Campaign Management — Group tests into campaigns with dependencies, scheduling, and recurring execution
• Compliance Mapping — Map NIST 800-53 and CIS Controls v8 to ATT&CK techniques with gap analysis
• Granular Scoring — 0–100 scoring for techniques, tactics, actors, and organization with configurable weights
• Operational Metrics — MTTD, MTTR, detection efficacy, alert fidelity, coverage velocity
• Executive Dashboard — High-level KPIs for leadership (leads + admin)
• Temporal Comparison — Coverage snapshots with historical comparison and trend analysis
• Auto Re-testing — Automatic retest creation after remediation completion (configurable limit)
• Performance Optimizations — Score caching, lazy loading, React.memo, database index optimization

Red Team / Blue Team Validation Flow

┌──────┐    ┌──────────────┐    ┌─────────────────┐    ┌───────────┐
│ DRAFT│───▶│RED_EXECUTING │───▶│ BLUE_EVALUATING  │───▶│ IN_REVIEW │
└──────┘    └──────────────┘    └─────────────────┘    └─────┬─────┘
                                                             │
                                         ┌───────────────────┤
                                         ▼                   ▼
                                   ┌──────────┐       ┌──────────┐
                                   │ REJECTED │       │VALIDATED │
                                   └────┬─────┘       └──────────┘
                                        │                    │
                                        └──▶ Back to DRAFT   ├──▶ Remediation
                                                             └──▶ Auto Re-test

States

State	Description	Who acts
draft	Created, pending execution	Red Tech
red_executing	Red Team documents attack & uploads evidence	Red Tech
blue_evaluating	Blue Team documents detection & uploads evidence	Blue Tech
in_review	Both managers review evidence	Red Lead, Blue Lead
validated	Approved by both managers	— (terminal)
rejected	Rejected — returns to draft for redo	Red/Blue Lead can reopen

Dual Validation

Both Red Lead and Blue Lead must independently vote:

• Both approve → test moves to validated
• Either rejects → test moves to rejected
• One votes, other pending → stays in in_review

User Roles

Role	Description	Capabilities
admin	Full system access	Everything
red_tech	Red team technician	Create tests, document attacks, upload red evidence
blue_tech	Blue team technician	Document detection, upload blue evidence
red_lead	Red team lead	Validate/reject the red side of tests
blue_lead	Blue team lead	Validate/reject the blue side of tests
viewer	Read-only access	View all data

Tech Stack

• Backend: FastAPI (Python 3.11) — Clean Modular Monolith with domain entities, services, and repository pattern
• Database: PostgreSQL 16 with UUID primary keys and JSONB columns
• Object Storage: MinIO (S3-compatible)
• ORM: SQLAlchemy 2.x with Alembic migrations
• Frontend: React 19 + TypeScript + Vite 7 + Tailwind CSS v4 + TanStack Query + TanStack Virtual
• Cache / Token Store: Redis (token blacklist, score caching)
• Scheduler: APScheduler (MITRE sync, Intel scan, Notification cleanup, Snapshots, Recurring campaigns)
• Testing: Pytest (367+ tests), Ruff (linting), GitHub Actions CI
• Charts: Recharts

Quick Start

Prerequisites

• Docker and Docker Compose
• Git
• Linux / macOS (or WSL on Windows)

Production Deployment

The recommended way to deploy Aegis in production:

git clone <repository-url>
cd Aegis
chmod +x scripts/install.sh
./scripts/install.sh

The interactive install wizard will guide you through:

1. Domain configuration — your domain or IP, protocol (HTTP/HTTPS), and port
2. Admin account — custom username and password (or auto-generated secure password)
3. Database — name, user, and password (or auto-generated)
4. Session duration — JWT token expiry (default: 15 minutes)
5. MITRE ATT&CK sync — optionally import ~700 techniques on first run

The script automatically generates cryptographically secure random secrets for SECRET_KEY, database password, and MinIO credentials. A summary with all credentials is displayed at the end of the installation.

Access the application at the URL shown in the installation summary.

Development Setup

For local development with hot-reload:

git clone <repository-url>
cd Aegis
docker-compose up -d
./scripts/init.sh

Access at http://localhost:5173 (frontend dev server) and http://localhost:8000/docs (API docs).

Authentication

JWT-based authentication with HttpOnly cookies. Admin credentials are configured during installation:

• If you set a custom password in the wizard, use that.
• If you left it blank, a secure random password was auto-generated and displayed in the installation summary and backend logs.

To retrieve auto-generated credentials after installation:

docker logs aegis-backend 2>&1 | grep -A 5 "Initial Admin User Created"

Note: Passwords must meet complexity requirements: minimum 12 characters with at least one uppercase letter, one lowercase letter, one digit, and one special character.

Importing Data Sources

On startup, the backend automatically seeds the initial data sources (Atomic Red Team, SigmaHQ, CALDERA, LOLBAS, GTFOBins, D3FEND). You can then sync each source from the UI:

1. Navigate to Data Sources in the sidebar
2. Click Sync on each data source to import its content
3. Trigger a MITRE ATT&CK Sync from the System page

Alternatively, use the API:

# Sync MITRE ATT&CK techniques
curl -X POST http://your-server/api/v1/system/sync-mitre -H "Authorization: Bearer $TOKEN"

# Sync all data sources at once
curl -X POST http://your-server/api/v1/data-sources/sync-all -H "Authorization: Bearer $TOKEN"

Production Considerations

• HTTPS/TLS: For internet-facing deployments, place a reverse proxy with TLS in front (e.g., Traefik, Caddy, or Nginx with Let's Encrypt). Uncomment the HSTS header in frontend/nginx.conf once HTTPS is configured.
• Backups: Set up regular PostgreSQL backups: docker exec aegis-postgres pg_dump -U postgres attackdb > backup.sql
• Updates: To update, pull the latest code and run: docker compose -f docker-compose.prod.yml up -d --build
• Firewall: Only expose port 80/443. All other services (DB, MinIO, backend) are internal only.
• Reconfigure: Run ./scripts/install.sh again to reconfigure the environment (domain, credentials, etc.).

Configuring Scoring Weights

Default scoring weights can be adjusted via environment variables:

SCORING_WEIGHT_TESTS=40
SCORING_WEIGHT_DETECTION_RULES=20
SCORING_WEIGHT_D3FEND=15
SCORING_WEIGHT_FRESHNESS=15
SCORING_WEIGHT_PLATFORM_DIVERSITY=10

Or at runtime via the admin API — see docs/SCORING.md.

Configuring Campaigns

1. Navigate to Campaigns in the sidebar
2. Create a new campaign (custom or from a threat actor)
3. Add tests with dependencies and phases
4. Optionally enable recurring scheduling (weekly, monthly, quarterly)

Services

Service	Port	Description
Frontend	5173	React dev server (Vite)
Backend	8000	FastAPI REST API
PostgreSQL	5433	Database
MinIO API	9000	S3-compatible object storage
MinIO Console	9001	MinIO web interface

Navigation

📊 Dashboard
📊 Executive Dashboard          (leads + admin)
🔲 ATT&CK Matrix                (advanced heatmap)
🧪 Tests
  ├─ All Tests
  ├─ My Pending Tasks
  └─ Test Catalog
📋 Campaigns
👤 Threat Actors
📜 Compliance
📈 Comparison                    (leads + admin)
📄 Reports
⚙️ System                       (admin only)
  ├─ Data Sources               (sync Atomic, Sigma, CALDERA, LOLBAS, GTFOBins, D3FEND)
  ├─ MITRE Sync                 (ATT&CK sync, intel scan, template management)
  ├─ Users
  └─ Audit Log

API Documentation

Interactive API documentation is available in development only (disabled in production for security):

• Swagger UI: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc

In production (AEGIS_ENV=production), these endpoints are disabled. Use the development environment or refer to docs/API.md.

API Endpoints

Group	Prefix	Key Operations
Auth	/api/v1/auth	Login, get current user
Techniques	/api/v1/techniques	CRUD, list with filters, mark reviewed
Tests	/api/v1/tests	Full Red/Blue workflow, remediation, retest chain
Test Templates	/api/v1/test-templates	CRUD, stats, toggle active, bulk activate/deactivate
Evidence	/api/v1/tests/{id}/evidence	Upload evidence, get presigned URLs
Campaigns	/api/v1/campaigns	CRUD, scheduling, history
Threat Actors	/api/v1/threat-actors	CRUD, technique mappings
Detection Rules	/api/v1/detection-rules	List, filter by source/technique
D3FEND	/api/v1/d3fend	Defensive techniques and mappings
Compliance	/api/v1/compliance	Frameworks, controls, gaps
Scores	/api/v1/scores	Technique/tactic/actor/org scores, config
Operational Metrics	/api/v1/metrics/operational	MTTD, MTTR, trends, team breakdown
Heatmap	/api/v1/heatmap	ATT&CK Navigator-style data
Snapshots	/api/v1/snapshots	Create, compare, list snapshots
Reports	/api/v1/reports	Coverage, results, remediation exports
Notifications	/api/v1/notifications	List, read, mark all read
Metrics	/api/v1/metrics	Summary, by-tactic, pipeline, team activity
System	/api/v1/system	MITRE sync, import, scheduler status
Users	/api/v1/users	User CRUD (admin)
Audit Logs	/api/v1/audit-logs	Audit trail (admin)
Data Sources	/api/v1/data-sources	Data source management (admin)

See docs/API.md for the full endpoint reference.

Configuration

All variables are configured automatically by scripts/install.sh. For manual setup, copy .env.example to .env and fill in the values.

Required (production)

Variable	Description
SECRET_KEY	JWT signing key — required in production (app refuses to start without it). Generate with openssl rand -hex 32
DB_PASSWORD	PostgreSQL password
MINIO_SECRET_KEY	MinIO secret key

Security & Auth

Variable	Default	Description
AEGIS_ENV	—	Set to production to enforce security settings
ADMIN_USERNAME	admin	Initial admin account username
ADMIN_PASSWORD	(auto-generated)	Initial admin password. If empty, a secure random password is generated and shown in logs
ACCESS_TOKEN_EXPIRE_MINUTES	15	JWT token lifetime in minutes
CORS_ORIGINS	http://localhost:5173	Comma-separated list of allowed frontend origins

Infrastructure

Variable	Default	Description
DB_USER	postgres	PostgreSQL username
DB_NAME	attackdb	PostgreSQL database name
MINIO_ENDPOINT	minio:9000	MinIO server address
MINIO_ACCESS_KEY	minioadmin	MinIO access key
MINIO_BUCKET	evidence	MinIO bucket for evidence files
MINIO_SECURE	false	Set to true if MinIO is behind TLS
FRONTEND_PORT	80	Port exposed by the frontend container

Scoring Weights

Variable	Default	Description
SCORING_WEIGHT_TESTS	40	Weight for test validation component
SCORING_WEIGHT_DETECTION_RULES	20	Weight for detection rules component
SCORING_WEIGHT_D3FEND	15	Weight for D3FEND coverage component
SCORING_WEIGHT_FRESHNESS	15	Weight for freshness component
SCORING_WEIGHT_PLATFORM_DIVERSITY	10	Weight for platform diversity component
MAX_RETEST_COUNT	3	Max automatic retests per original test

Security

Aegis includes several security hardening measures:

• Authentication: JWT tokens stored in HttpOnly/Secure/SameSite cookies (immune to XSS theft). Token revocation via Redis-backed blacklist on logout.
• Rate limiting: Login endpoint limited to 5 attempts per minute per IP (via slowapi).
• Password policy: Minimum 12 characters with uppercase, lowercase, digit, and special character.
• CORS: Configurable origins via CORS_ORIGINS environment variable. Restrictive method and header lists.
• Nginx security headers: CSP, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy.
• Non-root container: Backend runs as appuser (UID 1001), not root.
• File uploads: 50 MB size limit, extension whitelist, filename sanitization.
• ZIP imports: Zip Slip (path traversal) and Zip Bomb (size/entry limit) protection.
• API surface: Swagger UI, ReDoc, and OpenAPI schema disabled in production.
• Health endpoint: Restricted to internal networks via Nginx ACL.
• Input sanitization: LIKE wildcard escaping in all search queries; Pydantic schemas on all endpoints.
• XML parsing: Uses defusedxml to prevent Billion Laughs / XXE attacks.
• Error handling: Internal exception details are logged server-side only, never exposed to clients.

Project Structure

Aegis/
├── docker-compose.yml
├── docker-compose.prod.yml
├── .github/workflows/ci.yml      # GitHub Actions: ruff + pytest on PostgreSQL + Redis
├── docs/
│   ├── API.md                     # Full API endpoint reference
│   ├── ARCHITECTURE.md            # System architecture, DB schema, service map
│   ├── ADR.md                     # Architecture Decision Records
│   ├── DATA_SOURCES.md            # External data source documentation
│   ├── SCORING.md                 # Scoring system and metrics
│   ├── TECHNOLOGY_JUSTIFICATION.md
│   ├── C4_CONTEXT_DIAGRAM.md      # System context (C4 Level 1)
│   └── C4_CONTAINER_DIAGRAM.md    # Container architecture (C4 Level 2)
├── backend/
│   ├── Dockerfile
│   ├── requirements.txt
│   ├── alembic.ini
│   ├── alembic/versions/          # Database migration files
│   ├── pytest.ini
│   ├── tests/                     # 367+ pytest tests (domain, service, API)
│   └── app/
│       ├── main.py                # FastAPI app with all routers + lifespan
│       ├── config.py              # Pydantic Settings from environment
│       ├── database.py            # SQLAlchemy engine + session (lazy init)
│       ├── storage.py             # MinIO/S3 helpers
│       ├── auth.py                # Password hashing + JWT tokens
│       ├── domain/                # Pure business logic (zero framework imports)
│       │   ├── entities/          # Rich domain entities (Technique, Campaign, etc.)
│       │   ├── ports/             # Protocol interfaces (repos, ImportService)
│       │   ├── value_objects/     # Immutable types (MitreId, ScoringWeights)
│       │   ├── errors.py          # Domain exception hierarchy
│       │   └── unit_of_work.py    # Transaction management
│       ├── infrastructure/        # SQLAlchemy repos, Redis, mappers
│       ├── models/                # SQLAlchemy ORM models
│       ├── schemas/               # Pydantic request/response schemas
│       ├── routers/               # 27 thin HTTP adapter routers
│       ├── services/              # 46 framework-agnostic business services
│       ├── middleware/            # Error handler (domain exceptions → HTTP)
│       ├── dependencies/          # FastAPI dependency injection (auth, repos)
│       └── jobs/                  # APScheduler background jobs
└── frontend/src/
    ├── App.tsx                    # Routes with lazy loading + role protection
    ├── api/                       # API client modules (Axios + TanStack Query)
    ├── components/                # Reusable UI components
    ├── hooks/                     # Custom hooks (useDebounce, etc.)
    ├── context/                   # Auth state management
    └── pages/                     # Page components

Development

Running Migrations

docker exec aegis-backend alembic upgrade head
docker exec aegis-backend alembic revision --autogenerate -m "description"
docker exec aegis-backend alembic downgrade -1

Running Tests

# Run all V3 tests inside the container (recommended)
docker exec aegis-backend python -m pytest tests/ -v --tb=short

# Run specific test suites
docker exec aegis-backend python -m pytest tests/test_data_sources.py -v
docker exec aegis-backend python -m pytest tests/test_scoring_and_compliance.py -v
docker exec aegis-backend python -m pytest tests/test_campaigns_and_snapshots.py -v

# Skip integration tests (require network)
docker exec aegis-backend python -m pytest tests/ -v -m "not integration"

Generating Reports

# Coverage summary (JSON)
GET /api/v1/reports/coverage-summary

# Coverage CSV export
GET /api/v1/reports/coverage-csv

# Compliance gap analysis
GET /api/v1/compliance/{framework_id}/gaps

Further Documentation

• Architecture — Database schema, backend layers, domain entities, service map
• API Reference — Full endpoint documentation
• Scoring — Scoring system explained with examples and configuration
• Data Sources — All external data sources with import instructions
• ADRs — Architecture Decision Records
• Technology Justification — Technology choices and rationale
• C4 Diagrams — System context and container architecture

License

This project is proprietary software. All rights reserved.

Disclaimer

This project has been developed with the assistance of Cursor and Claude Opus 4.6 (Anthropic).