Add wiki page: Architecture
226
Architecture.md
226
Architecture.md
@@ -0,0 +1,226 @@
|
||||
# Architecture
|
||||
|
||||
This page describes the technical architecture of Aegis: its components, data flow,
|
||||
deployment model, and backend code structure.
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack
|
||||
|
||||
| Layer | Technology | Version |
|
||||
|-------|------------|---------|
|
||||
| Backend framework | FastAPI | Latest (Python 3.11) |
|
||||
| Database | PostgreSQL | 16 |
|
||||
| Object storage | MinIO | Latest |
|
||||
| Cache / token blacklist | Redis | 7 |
|
||||
| Frontend framework | React | 19 |
|
||||
| Language (frontend) | TypeScript | 5 |
|
||||
| Build tool | Vite | 7 |
|
||||
| CSS framework | Tailwind CSS | v4 |
|
||||
| Data fetching | TanStack Query | v5 |
|
||||
| ORM | SQLAlchemy | 2.x |
|
||||
| Migrations | Alembic | Latest |
|
||||
| Task scheduler | APScheduler | 3.x |
|
||||
| Linting | Ruff | Latest |
|
||||
| Testing | Pytest | 8.x |
|
||||
|
||||
---
|
||||
|
||||
## Docker Services
|
||||
|
||||
All services are defined in `docker-compose.yml` (development) and
|
||||
`docker-compose.prod.yml` (production).
|
||||
|
||||
| Service name | Image | Internal port | Dev exposed port | Role |
|
||||
|---|---|---|---|---|
|
||||
| `aegis-backend` | Custom (Dockerfile) | 8000 | 8000 | FastAPI application + APScheduler |
|
||||
| `aegis-frontend` | Custom (Dockerfile.prod / vite dev) | 5173 / 80 | 5173 | React SPA |
|
||||
| `aegis-postgres` | `postgres:16` | 5432 | 5433 | Primary relational database |
|
||||
| `aegis-redis` | `redis:7` | 6379 | 6379 | Token blacklist, rate limiting, cache |
|
||||
| `aegis-minio` | `minio/minio` | 9000 (API), 9001 (Console) | 9000 / 9001 | Evidence file storage (S3-compatible) |
|
||||
|
||||
### Health checks
|
||||
|
||||
- Backend: `GET /health` returns `{"status": "ok"}`
|
||||
- Postgres: `pg_isready` via Docker healthcheck
|
||||
- Redis: `PING` via Docker healthcheck
|
||||
|
||||
---
|
||||
|
||||
## Backend Folder Structure
|
||||
|
||||
```
|
||||
backend/
|
||||
├── app/
|
||||
│ ├── domain/
|
||||
│ │ ├── entities/ # Core domain entities (Test, Technique, User, ...)
|
||||
│ │ ├── errors/ # Custom domain exception classes
|
||||
│ │ ├── ports/ # Abstract interfaces (repositories, storage)
|
||||
│ │ └── value_objects/ # Immutable value types (CoverageStatus, Role, ...)
|
||||
│ │
|
||||
│ ├── services/ # Business logic layer (~45 services)
|
||||
│ │ ├── test_service.py
|
||||
│ │ ├── coverage_service.py
|
||||
│ │ ├── campaign_service.py
|
||||
│ │ ├── alert_service.py
|
||||
│ │ ├── report_service.py
|
||||
│ │ └── ...
|
||||
│ │
|
||||
│ ├── routers/ # FastAPI route handlers (~28 routers)
|
||||
│ │ ├── auth.py # /api/v1/auth
|
||||
│ │ ├── tests.py # /api/v1/tests
|
||||
│ │ ├── techniques.py # /api/v1/techniques
|
||||
│ │ ├── campaigns.py # /api/v1/campaigns
|
||||
│ │ └── ... # one file per module
|
||||
│ │
|
||||
│ ├── models/ # SQLAlchemy ORM model classes
|
||||
│ │ ├── test.py
|
||||
│ │ ├── technique.py
|
||||
│ │ ├── user.py
|
||||
│ │ └── ...
|
||||
│ │
|
||||
│ ├── schemas/ # Pydantic v2 request/response schemas
|
||||
│ │ ├── test_schemas.py
|
||||
│ │ ├── user_schemas.py
|
||||
│ │ └── ...
|
||||
│ │
|
||||
│ ├── infrastructure/ # Adapters for external systems
|
||||
│ │ ├── redis_client.py # Redis connection + helpers
|
||||
│ │ ├── minio_client.py # MinIO / S3 evidence storage
|
||||
│ │ └── persistence/ # SQLAlchemy session management
|
||||
│ │
|
||||
│ ├── jobs/ # APScheduler background jobs
|
||||
│ │ ├── mitre_sync.py # Hourly MITRE ATT&CK data sync
|
||||
│ │ ├── alert_eval.py # Hourly alert rule evaluation
|
||||
│ │ └── retention.py # Snapshot retention cleanup
|
||||
│ │
|
||||
│ ├── core/
|
||||
│ │ ├── config.py # Settings via pydantic-settings / .env
|
||||
│ │ ├── security.py # JWT creation/validation, password hashing
|
||||
│ │ └── dependencies.py # FastAPI dependency injection (get_db, get_current_user)
|
||||
│ │
|
||||
│ └── main.py # App factory, router registration, CORS, middleware
|
||||
│
|
||||
├── alembic/ # Migration scripts
|
||||
│ └── versions/
|
||||
├── tests/ # Pytest test suite (367+ tests)
|
||||
│ ├── test_auth.py
|
||||
│ ├── test_tests.py
|
||||
│ └── ...
|
||||
├── entrypoint.prod.sh # Production startup: run migrations then uvicorn
|
||||
├── requirements.txt
|
||||
└── Dockerfile
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API URL Structure
|
||||
|
||||
Every API endpoint is prefixed with `/api/v1/`. Examples:
|
||||
|
||||
```
|
||||
GET /api/v1/techniques
|
||||
POST /api/v1/tests
|
||||
GET /api/v1/dashboard/kpis
|
||||
POST /api/v1/campaigns/{id}/complete
|
||||
```
|
||||
|
||||
The Swagger UI (development only) is available at `http://localhost:8000/docs`.
|
||||
ReDoc is at `http://localhost:8000/redoc`.
|
||||
In production (`AEGIS_ENV=production`) both are disabled.
|
||||
|
||||
---
|
||||
|
||||
## Authentication Flow
|
||||
|
||||
```
|
||||
Client Backend Redis
|
||||
│ │ │
|
||||
│ POST /auth/login │ │
|
||||
│ (username + password) │ │
|
||||
│──────────────────────────>│ │
|
||||
│ │ verify password hash │
|
||||
│ │ create JWT (jti claim) │
|
||||
│ Set-Cookie: aegis_token │ │
|
||||
│<──────────────────────────│ │
|
||||
│ │ │
|
||||
│ GET /api/v1/tests │ │
|
||||
│ (Cookie: aegis_token) │ │
|
||||
│──────────────────────────>│ │
|
||||
│ │ decode JWT │
|
||||
│ │ check blacklist (jti) ────>│
|
||||
│ │<─────────────────── not found │
|
||||
│ │ inject current_user │
|
||||
│ 200 OK │ │
|
||||
│<──────────────────────────│ │
|
||||
│ │ │
|
||||
│ POST /auth/logout │ │
|
||||
│──────────────────────────>│ │
|
||||
│ │ store jti in blacklist ───>│
|
||||
│ │ (TTL = token exp) │
|
||||
│ Clear cookie │ │
|
||||
│<──────────────────────────│ │
|
||||
```
|
||||
|
||||
Alternative: pass token as `Authorization: Bearer <token>` header
|
||||
(e.g. for API clients or when cookies are not practical).
|
||||
|
||||
---
|
||||
|
||||
## Background Jobs (APScheduler)
|
||||
|
||||
Aegis uses APScheduler (in-process) for recurring tasks:
|
||||
|
||||
| Job ID | Schedule | Description |
|
||||
|--------|----------|-------------|
|
||||
| `mitre_sync` | Every hour | Fetches latest MITRE ATT&CK STIX data from GitHub, upserts techniques |
|
||||
| `alert_evaluation` | Every hour | Evaluates all enabled alert rules against current data |
|
||||
| `snapshot_retention` | Daily | Deletes old snapshots beyond the configured retention window |
|
||||
|
||||
Jobs are registered in `app/jobs/` and started in `main.py` during application startup.
|
||||
Admin can check job status at `GET /api/v1/system/scheduler-status`.
|
||||
Manual MITRE sync: `POST /api/v1/system/sync-mitre` (admin only).
|
||||
|
||||
---
|
||||
|
||||
## Data Flow — Test Execution
|
||||
|
||||
```
|
||||
red_tech Backend DB MinIO
|
||||
│ │ │ │
|
||||
│ POST /tests/{id}/ │ │ │
|
||||
│ start-execution │ │ │
|
||||
│───────────────────────>│ check role (red_tech+) │ │
|
||||
│ │ validate state = draft │ │
|
||||
│ │ UPDATE state=red_executing│ │
|
||||
│ │──────────────────────────>│ │
|
||||
│ 200 OK │ │ │
|
||||
│<───────────────────────│ │ │
|
||||
│ │ │ │
|
||||
│ POST /tests/{id}/ │ │ │
|
||||
│ evidence │ │ │
|
||||
│ (multipart file) │ │ │
|
||||
│───────────────────────>│ check role + state │ │
|
||||
│ │ upload file ──────────────────────────>│
|
||||
│ │ store metadata ───────────>│ │
|
||||
│ 201 Created │ │ │
|
||||
│<───────────────────────│ │ │
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Production vs Development Differences
|
||||
|
||||
| Feature | Development | Production |
|
||||
|---------|-------------|------------|
|
||||
| Swagger UI (`/docs`) | Enabled | Disabled |
|
||||
| ReDoc (`/redoc`) | Enabled | Disabled |
|
||||
| Cookie `secure` flag | False (HTTP allowed) | True (HTTPS required) |
|
||||
| `AEGIS_ENV` | `development` | `production` |
|
||||
| `SECURE_COOKIES` | `auto` or `false` | `auto` or `true` |
|
||||
| Debug logging | Verbose | Structured JSON |
|
||||
| CORS origins | Permissive | Restricted |
|
||||
| DB migrations | Manual or auto | Auto on startup via `entrypoint.prod.sh` |
|
||||
| Secret key | Can be default | **Must** be set via `SECRET_KEY` env var |
|
||||
|
||||
See [Deployment-Guide](Deployment-Guide) for full configuration details.
|
||||
|
||||
Reference in New Issue
Block a user