Add wiki page: Architecture
226
Architecture.md
226
Architecture.md
@@ -0,0 +1,226 @@
|
|||||||
|
# Architecture
|
||||||
|
|
||||||
|
This page describes the technical architecture of Aegis: its components, data flow,
|
||||||
|
deployment model, and backend code structure.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Tech Stack
|
||||||
|
|
||||||
|
| Layer | Technology | Version |
|
||||||
|
|-------|------------|---------|
|
||||||
|
| Backend framework | FastAPI | Latest (Python 3.11) |
|
||||||
|
| Database | PostgreSQL | 16 |
|
||||||
|
| Object storage | MinIO | Latest |
|
||||||
|
| Cache / token blacklist | Redis | 7 |
|
||||||
|
| Frontend framework | React | 19 |
|
||||||
|
| Language (frontend) | TypeScript | 5 |
|
||||||
|
| Build tool | Vite | 7 |
|
||||||
|
| CSS framework | Tailwind CSS | v4 |
|
||||||
|
| Data fetching | TanStack Query | v5 |
|
||||||
|
| ORM | SQLAlchemy | 2.x |
|
||||||
|
| Migrations | Alembic | Latest |
|
||||||
|
| Task scheduler | APScheduler | 3.x |
|
||||||
|
| Linting | Ruff | Latest |
|
||||||
|
| Testing | Pytest | 8.x |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Docker Services
|
||||||
|
|
||||||
|
All services are defined in `docker-compose.yml` (development) and
|
||||||
|
`docker-compose.prod.yml` (production).
|
||||||
|
|
||||||
|
| Service name | Image | Internal port | Dev exposed port | Role |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| `aegis-backend` | Custom (Dockerfile) | 8000 | 8000 | FastAPI application + APScheduler |
|
||||||
|
| `aegis-frontend` | Custom (Dockerfile.prod / vite dev) | 5173 / 80 | 5173 | React SPA |
|
||||||
|
| `aegis-postgres` | `postgres:16` | 5432 | 5433 | Primary relational database |
|
||||||
|
| `aegis-redis` | `redis:7` | 6379 | 6379 | Token blacklist, rate limiting, cache |
|
||||||
|
| `aegis-minio` | `minio/minio` | 9000 (API), 9001 (Console) | 9000 / 9001 | Evidence file storage (S3-compatible) |
|
||||||
|
|
||||||
|
### Health checks
|
||||||
|
|
||||||
|
- Backend: `GET /health` returns `{"status": "ok"}`
|
||||||
|
- Postgres: `pg_isready` via Docker healthcheck
|
||||||
|
- Redis: `PING` via Docker healthcheck
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Backend Folder Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
backend/
|
||||||
|
├── app/
|
||||||
|
│ ├── domain/
|
||||||
|
│ │ ├── entities/ # Core domain entities (Test, Technique, User, ...)
|
||||||
|
│ │ ├── errors/ # Custom domain exception classes
|
||||||
|
│ │ ├── ports/ # Abstract interfaces (repositories, storage)
|
||||||
|
│ │ └── value_objects/ # Immutable value types (CoverageStatus, Role, ...)
|
||||||
|
│ │
|
||||||
|
│ ├── services/ # Business logic layer (~45 services)
|
||||||
|
│ │ ├── test_service.py
|
||||||
|
│ │ ├── coverage_service.py
|
||||||
|
│ │ ├── campaign_service.py
|
||||||
|
│ │ ├── alert_service.py
|
||||||
|
│ │ ├── report_service.py
|
||||||
|
│ │ └── ...
|
||||||
|
│ │
|
||||||
|
│ ├── routers/ # FastAPI route handlers (~28 routers)
|
||||||
|
│ │ ├── auth.py # /api/v1/auth
|
||||||
|
│ │ ├── tests.py # /api/v1/tests
|
||||||
|
│ │ ├── techniques.py # /api/v1/techniques
|
||||||
|
│ │ ├── campaigns.py # /api/v1/campaigns
|
||||||
|
│ │ └── ... # one file per module
|
||||||
|
│ │
|
||||||
|
│ ├── models/ # SQLAlchemy ORM model classes
|
||||||
|
│ │ ├── test.py
|
||||||
|
│ │ ├── technique.py
|
||||||
|
│ │ ├── user.py
|
||||||
|
│ │ └── ...
|
||||||
|
│ │
|
||||||
|
│ ├── schemas/ # Pydantic v2 request/response schemas
|
||||||
|
│ │ ├── test_schemas.py
|
||||||
|
│ │ ├── user_schemas.py
|
||||||
|
│ │ └── ...
|
||||||
|
│ │
|
||||||
|
│ ├── infrastructure/ # Adapters for external systems
|
||||||
|
│ │ ├── redis_client.py # Redis connection + helpers
|
||||||
|
│ │ ├── minio_client.py # MinIO / S3 evidence storage
|
||||||
|
│ │ └── persistence/ # SQLAlchemy session management
|
||||||
|
│ │
|
||||||
|
│ ├── jobs/ # APScheduler background jobs
|
||||||
|
│ │ ├── mitre_sync.py # Hourly MITRE ATT&CK data sync
|
||||||
|
│ │ ├── alert_eval.py # Hourly alert rule evaluation
|
||||||
|
│ │ └── retention.py # Snapshot retention cleanup
|
||||||
|
│ │
|
||||||
|
│ ├── core/
|
||||||
|
│ │ ├── config.py # Settings via pydantic-settings / .env
|
||||||
|
│ │ ├── security.py # JWT creation/validation, password hashing
|
||||||
|
│ │ └── dependencies.py # FastAPI dependency injection (get_db, get_current_user)
|
||||||
|
│ │
|
||||||
|
│ └── main.py # App factory, router registration, CORS, middleware
|
||||||
|
│
|
||||||
|
├── alembic/ # Migration scripts
|
||||||
|
│ └── versions/
|
||||||
|
├── tests/ # Pytest test suite (367+ tests)
|
||||||
|
│ ├── test_auth.py
|
||||||
|
│ ├── test_tests.py
|
||||||
|
│ └── ...
|
||||||
|
├── entrypoint.prod.sh # Production startup: run migrations then uvicorn
|
||||||
|
├── requirements.txt
|
||||||
|
└── Dockerfile
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API URL Structure
|
||||||
|
|
||||||
|
Every API endpoint is prefixed with `/api/v1/`. Examples:
|
||||||
|
|
||||||
|
```
|
||||||
|
GET /api/v1/techniques
|
||||||
|
POST /api/v1/tests
|
||||||
|
GET /api/v1/dashboard/kpis
|
||||||
|
POST /api/v1/campaigns/{id}/complete
|
||||||
|
```
|
||||||
|
|
||||||
|
The Swagger UI (development only) is available at `http://localhost:8000/docs`.
|
||||||
|
ReDoc is at `http://localhost:8000/redoc`.
|
||||||
|
In production (`AEGIS_ENV=production`) both are disabled.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Authentication Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
Client Backend Redis
|
||||||
|
│ │ │
|
||||||
|
│ POST /auth/login │ │
|
||||||
|
│ (username + password) │ │
|
||||||
|
│──────────────────────────>│ │
|
||||||
|
│ │ verify password hash │
|
||||||
|
│ │ create JWT (jti claim) │
|
||||||
|
│ Set-Cookie: aegis_token │ │
|
||||||
|
│<──────────────────────────│ │
|
||||||
|
│ │ │
|
||||||
|
│ GET /api/v1/tests │ │
|
||||||
|
│ (Cookie: aegis_token) │ │
|
||||||
|
│──────────────────────────>│ │
|
||||||
|
│ │ decode JWT │
|
||||||
|
│ │ check blacklist (jti) ────>│
|
||||||
|
│ │<─────────────────── not found │
|
||||||
|
│ │ inject current_user │
|
||||||
|
│ 200 OK │ │
|
||||||
|
│<──────────────────────────│ │
|
||||||
|
│ │ │
|
||||||
|
│ POST /auth/logout │ │
|
||||||
|
│──────────────────────────>│ │
|
||||||
|
│ │ store jti in blacklist ───>│
|
||||||
|
│ │ (TTL = token exp) │
|
||||||
|
│ Clear cookie │ │
|
||||||
|
│<──────────────────────────│ │
|
||||||
|
```
|
||||||
|
|
||||||
|
Alternative: pass token as `Authorization: Bearer <token>` header
|
||||||
|
(e.g. for API clients or when cookies are not practical).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Background Jobs (APScheduler)
|
||||||
|
|
||||||
|
Aegis uses APScheduler (in-process) for recurring tasks:
|
||||||
|
|
||||||
|
| Job ID | Schedule | Description |
|
||||||
|
|--------|----------|-------------|
|
||||||
|
| `mitre_sync` | Every hour | Fetches latest MITRE ATT&CK STIX data from GitHub, upserts techniques |
|
||||||
|
| `alert_evaluation` | Every hour | Evaluates all enabled alert rules against current data |
|
||||||
|
| `snapshot_retention` | Daily | Deletes old snapshots beyond the configured retention window |
|
||||||
|
|
||||||
|
Jobs are registered in `app/jobs/` and started in `main.py` during application startup.
|
||||||
|
Admin can check job status at `GET /api/v1/system/scheduler-status`.
|
||||||
|
Manual MITRE sync: `POST /api/v1/system/sync-mitre` (admin only).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Flow — Test Execution
|
||||||
|
|
||||||
|
```
|
||||||
|
red_tech Backend DB MinIO
|
||||||
|
│ │ │ │
|
||||||
|
│ POST /tests/{id}/ │ │ │
|
||||||
|
│ start-execution │ │ │
|
||||||
|
│───────────────────────>│ check role (red_tech+) │ │
|
||||||
|
│ │ validate state = draft │ │
|
||||||
|
│ │ UPDATE state=red_executing│ │
|
||||||
|
│ │──────────────────────────>│ │
|
||||||
|
│ 200 OK │ │ │
|
||||||
|
│<───────────────────────│ │ │
|
||||||
|
│ │ │ │
|
||||||
|
│ POST /tests/{id}/ │ │ │
|
||||||
|
│ evidence │ │ │
|
||||||
|
│ (multipart file) │ │ │
|
||||||
|
│───────────────────────>│ check role + state │ │
|
||||||
|
│ │ upload file ──────────────────────────>│
|
||||||
|
│ │ store metadata ───────────>│ │
|
||||||
|
│ 201 Created │ │ │
|
||||||
|
│<───────────────────────│ │ │
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Production vs Development Differences
|
||||||
|
|
||||||
|
| Feature | Development | Production |
|
||||||
|
|---------|-------------|------------|
|
||||||
|
| Swagger UI (`/docs`) | Enabled | Disabled |
|
||||||
|
| ReDoc (`/redoc`) | Enabled | Disabled |
|
||||||
|
| Cookie `secure` flag | False (HTTP allowed) | True (HTTPS required) |
|
||||||
|
| `AEGIS_ENV` | `development` | `production` |
|
||||||
|
| `SECURE_COOKIES` | `auto` or `false` | `auto` or `true` |
|
||||||
|
| Debug logging | Verbose | Structured JSON |
|
||||||
|
| CORS origins | Permissive | Restricted |
|
||||||
|
| DB migrations | Manual or auto | Auto on startup via `entrypoint.prod.sh` |
|
||||||
|
| Secret key | Can be default | **Must** be set via `SECRET_KEY` env var |
|
||||||
|
|
||||||
|
See [Deployment-Guide](Deployment-Guide) for full configuration details.
|
||||||
|
|||||||
Reference in New Issue
Block a user