Commit Graph

144 Commits

Author SHA1 Message Date
64d64080e0 fix: resolve 20 security vulnerabilities from comprehensive audit
Critical (1-3):
- Replace hardcoded admin credentials with secure auto-generation (seed.py)
- Enforce SECRET_KEY configuration, fail in production if missing (config.py)
- Add Zip Slip and Zip Bomb protection to all ZIP import services

High/Medium (4-9):
- Add 50MB file size limit and extension whitelist to evidence uploads
- Configure CORS origins via environment variable instead of hardcoded
- Migrate JWT storage from localStorage to HttpOnly cookies (frontend+backend)
- Add rate limiting (5/min) on login endpoint via slowapi
- Replace generic dict payloads with Pydantic schemas (mass assignment)

Medium (10-17):
- Check is_active on login to prevent disabled users from authenticating
- Sanitize exception messages in API responses (system, data_sources)
- Escape LIKE wildcards in all ilike search filters across 8 routers
- Run Docker container as non-root user (appuser)
- Make MINIO_SECURE configurable via environment variable
- Add password complexity policy (12+ chars, upper/lower/digit/special)
- Implement JWT token revocation via in-memory blacklist + reduce TTL to 15min
- Replace xml.etree with defusedxml to prevent Billion Laughs attacks

Low (18-20):
- Add security headers to Nginx (CSP, X-Frame-Options, HSTS-ready, etc.)
- Disable Swagger UI/ReDoc/OpenAPI in production
- Restrict /health endpoint to internal networks via Nginx ACL

Also: rewrite install.sh as interactive wizard for guided deployment,
fix test-from-template validation error (technique_id UUID vs MITRE ID)
2026-02-11 08:56:26 +01:00
e7e63161e8 fix: increase Nginx proxy timeout to 300s for long-running API calls
MITRE sync and data source imports can take over 60s on first run,
causing Nginx to return 504 Gateway Timeout to the frontend.
2026-02-10 16:34:23 +01:00
38285f885c fix: TypeScript errors blocking production build
- Replace process.env.NODE_ENV with import.meta.env.DEV (Vite compatible)
- Fix undefined not assignable to boolean|null with nullish coalescing
2026-02-10 16:18:34 +01:00
cc0bbdf797 fix: auto-detect Docker API version to avoid client/server mismatch 2026-02-10 16:14:21 +01:00
de6f3fbea4 fix: install script cd to project root and improve error handling
- Auto-detect project root from script location so it works from any dir
- Fail properly if docker-compose build fails (was hidden by pipe)
- Use docker exec for backend health checks (port 8000 not exposed in prod)
- Add fallback API access via docker exec if nginx not reachable yet
- Show backend logs during wait for better debugging
- Increase timeouts for MITRE sync and data source sync
2026-02-10 16:10:09 +01:00
8aec3581a0 feat: production deployment setup and hardcoded URL fixes
- Fix hardcoded localhost:8000 URLs in frontend to use relative /api/v1
  path (works with Nginx proxy in prod and VITE_API_URL in dev)
- Create production entrypoint (entrypoint.prod.sh) that runs migrations,
  seeds, and starts uvicorn with 4 workers (no --reload)
- Create comprehensive install.sh script for production deployment that
  generates secure .env, builds containers, waits for health, and
  optionally triggers initial MITRE sync
- Update docker-compose.prod.yml to use production entrypoint
- Update Dockerfile to make both entrypoints executable
- Remove init.ps1 (production will always be Linux)
- Update README with production deployment instructions
2026-02-10 16:04:16 +01:00
a3f83c316a fix: D3FEND ontology-based import, template management UX, and branding
- Rewrite D3FEND import to use tactic-level APIs for reliable technique
  fetching with proper ontology IRIs, descriptions, and tactic assignments
- Fix D3FEND technique URLs to use canonical IRI (no more 404s)
- All 255 D3FEND techniques now have descriptions from the official API
- Change Deactivate button color to red in template management table
- Add custom Aegis logo and favicon replacing default Vite assets
- Remove unused old API parsing code and clean up fallback list
2026-02-10 15:53:24 +01:00
c2e9c687f4 fix: D3FEND expandable cards, System page cleanup, and multi-source improvements
- Make D3FEND defense cards clickable with expandable details and external link
- Fix D3FEND URLs to use PascalCase technique names matching the ontology
- Remove duplicate Import Atomic Red Team from System page (use Data Sources)
- Add bulk Activate All / Deactivate All buttons with confirmation modal
- Fix template admin list to show both active and inactive templates
- Add PATCH /test-templates/bulk-activate backend endpoint
- Auto-seed data sources on container startup via entrypoint.sh
- Fix SigmaHQ, CALDERA, GTFOBins import issues
- Register D3FEND sync handler in data sources router
- Add CIS Controls v8 compliance framework import
- Expand Test Catalog source filters (CALDERA, LOLBAS, GTFOBins)
- Campaign Generate from Threat Actor now opens actor selector modal
- Add coverage snapshot creation button to Comparison page
- Update README with accurate data source and feature documentation
2026-02-10 13:22:23 +01:00
8032b67fab docs: add disclaimer about Cursor and Claude Opus 4.6 assistance 2026-02-10 10:52:53 +01:00
14f8485f06 feat(phase-33): final polish V3 - navigation, performance, and documentation (T-238 to T-240) 2026-02-10 09:21:35 +01:00
35983de67e feat(phase-32): add automated tests V3 for data sources, scoring, campaigns and snapshots (T-235 to T-237) 2026-02-10 09:07:43 +01:00
02034d60f0 feat(phase-31): add campaign scheduling and recurring automation (T-233 to T-234) 2026-02-10 08:38:00 +01:00
4d124b42dd feat(phase-30): add coverage snapshots, temporal comparison and auto re-testing (T-230 to T-232) 2026-02-10 08:34:29 +01:00
2ac8e7f4a5 feat(phase-29): add compliance framework mapping, reports and UI (T-227 to T-229) 2026-02-09 18:41:24 +01:00
12f33307fd feat(phase-28): add scoring system, operational metrics and executive dashboard (T-224 to T-226) 2026-02-09 17:24:44 +01:00
a911ddeb52 feat(phase-27): add advanced ATT&CK Navigator-style heatmap with layers, filters and export (T-221 to T-223) 2026-02-09 17:16:59 +01:00
57b47c296d feat(phase-26): add Campaign models, endpoints, service with kill chain timeline UI (T-217 to T-220) 2026-02-09 16:52:52 +01:00
f4de12d8ab feat(phase-25): add detection rule associations, checklist UI and evaluation workflow (T-215, T-216) 2026-02-09 16:44:35 +01:00
cd124b655b feat(phase-24): integrate MITRE D3FEND defensive techniques with ATT&CK mapping (T-213, T-214) 2026-02-09 16:38:59 +01:00
2fc0e2cafd feat(phase-23): add Threat Actor profiles with MITRE CTI import, API, heatmap and gap analysis (T-208 to T-212) 2026-02-09 16:27:38 +01:00
f4c8cbf768 feat(phase-22): add import services for Sigma, LOLBAS, GTFOBins, CALDERA, Elastic and data sources panel (T-203 to T-207) 2026-02-09 16:19:44 +01:00
022c4f2886 feat(phase-21): add V3 demo seed, DataSource and DetectionRule models (T-200, T-201, T-202) 2026-02-09 16:06:44 +01:00
29eab4ef77 feat(phase-20): navigation, error handling, integration tests, and V2 docs (T-132 to T-135) 2026-02-09 14:19:42 +01:00
9ea6ce1326 feat(phase-19): add remediation fields and reports system (T-130, T-131) 2026-02-09 13:58:35 +01:00
fb7f340038 feat(phase-18): add in-app notification system (T-128, T-129) 2026-02-09 13:52:04 +01:00
cda59de426 test(phase-17): add automated tests for Red/Blue workflow, templates CRUD, and V2 metrics (T-125, T-126, T-127) 2026-02-09 13:35:40 +01:00
a95defcee4 feat(phase-16): enhanced Tests view, Red/Blue dashboard metrics, and Template admin panel (T-122, T-123, T-124) 2026-02-09 13:00:07 +01:00
fd7f855008 feat(phase-15): add Test Catalog page, template instantiation, and auto-migration entrypoint (T-119, T-120, T-121)
T-119: TestCatalogPage with search, filters (source/platform/severity), template cards grid, and pagination

T-120: TestFromTemplateForm modal with pre-filled fields from template, required field validation, and redirect on creation

T-121: Integrate Available Test Templates section in TechniqueDetailPage with Run This Test buttons; fix missing testStateBadgeColors for new states

Also: add backend entrypoint.sh for automatic Alembic migrations + seed on container startup, add curl to Dockerfile for healthcheck
2026-02-09 12:22:29 +01:00
cea470053f feat(phase-14): redesign Test Detail page with Red/Blue tabs and dual validation (T-115, T-116, T-117, T-118)
T-115: TestDetailHeader with progress bar, contextual action buttons, and dual validation indicators

T-116: TeamTabs component with Red Team, Blue Team, Summary, and Timeline tabs

T-117: Redesigned TestDetailPage integrating new components with react-query mutations, toast notifications, and role/state-based permissions

T-118: ValidationModal for dual Red Lead / Blue Lead approval with required notes on rejection
2026-02-09 11:14:44 +01:00
d660bceeb4 feat(phase-13): update frontend types and API clients for Red/Blue workflow (T-113, T-114)
T-113: Rewrite models.ts with v2 types - TestState now includes red_executing/blue_evaluating, add TeamSide, ValidationStatus, TestTemplate, TestTemplateSummary, TestTimelineEntry types, RED_EDITABLE_STATES/BLUE_EDITABLE_STATES constants, and dual validation fields on Test interface. Remove old validated_by/validated_at references from TestDetailPage and techniques API.

T-114: Rewrite tests.ts API client with 16 functions covering full Red/Blue workflow (createTestFromTemplate, updateTestRed/Blue, startExecution, submitRed/Blue, validateAsRedLead/BlueLead, reopenTest, getTestTimeline). Rewrite evidence.ts with team parameter on upload/list and new deleteEvidence. Create test-templates.ts with getTemplates, getTemplateById, getTemplatesByTechnique, createTemplate, importAtomicTests.
2026-02-09 10:57:48 +01:00
9d7832c571 feat(phase-12): implement Red/Blue API endpoints (T-109, T-110, T-111, T-112)
T-109: Rewrite tests router with full Red/Blue workflow endpoints - list with filters, create from template, Red/Blue team updates with state guards, start-execution, submit-red, submit-blue, validate-red, validate-blue, reopen, and timeline. All using workflow service from Phase 11.

T-110: Rewrite evidence router with Red/Blue separation - upload with team field, list with team filter, delete with state-based permissions. Red Team edits in draft/red_executing, Blue Team in blue_evaluating, admin bypasses all.

T-111: Create test_templates router with full CRUD - paginated list with source/platform/severity/search filters, by-technique lookup, admin-only create/update, and soft delete. Registered in main.py.

T-112: Add POST /system/import-atomic-tests endpoint to system router - admin-only trigger for Atomic Red Team import with error handling and statistics response.

Includes validation tests for all four tasks (35 checks total).
2026-02-09 10:45:33 +01:00
7af6be10be feat(phase-11): implement Red/Blue business logic services (T-106, T-107, T-108)
T-106: Create test_workflow_service.py with state-machine transitions for the complete test lifecycle (draft -> red_executing -> blue_evaluating -> in_review -> validated/rejected), dual validation by Red/Blue leads, and reopen capability with field cleanup.

T-107: Update status_service.py to use detection_result from Blue Team instead of legacy result field, and differentiate between partial progress (some validated) vs all-in-progress states.

T-108: Create atomic_import_service.py that downloads the Atomic Red Team repo as a ZIP (avoiding API rate limits), parses all atomics YAML files, and creates idempotent TestTemplate records mapped to MITRE techniques.

Includes validation tests for all three tasks (19 checks total).
2026-02-09 09:58:54 +01:00
086cc5c8bc fix: add .dockerignore files to exclude node_modules from build 2026-02-06 16:59:50 +01:00
ce46314afb feat: add complete Docker setup for testing
- Update docker-compose.yml with frontend service and healthchecks

- Add frontend Dockerfile with dev and production stages

- Add nginx.conf for production frontend serving

- Add docker-compose.prod.yml for production deployment

- Add .env.example with all configuration options

- Add init scripts (init.sh, init.ps1) for easy setup
2026-02-06 16:33:22 +01:00
174919da4e feat(phase-9): implement MVP polishing and closure
T-032: User management admin panel - backend users router with CRUD, frontend UsersPage with modals

T-033: Audit log viewer - backend audit router with filters/pagination, frontend AuditLogPage

T-034: Global error handling - ErrorBoundary, LoadingSpinner, ErrorMessage, Toast components

T-035: Backend tests - pytest setup with SQLite, tests for health/auth/techniques/tests

T-036: Documentation - Updated README with testing section, created docs/API.md
2026-02-06 16:30:35 +01:00
cb447f3803 feat: Phase 8 - Frontend main views (T-026 to T-031)
Implement all main frontend views for the MITRE ATT&CK coverage platform:

- T-026: Dashboard with coverage summary cards and tactic breakdown table

- T-027: Interactive ATT&CK matrix with filtering by status, tactic, platform

- T-028: Technique detail page with tests, intel items, and review actions

- T-029: Test creation form with technique selector and validation

- T-030: Test detail page with drag and drop evidence upload and download

- T-031: System admin panel with MITRE sync and intel scan controls

New components: CoverageSummaryCard, TacticCoverageChart, AttackMatrix, TechniqueCell, TestForm, EvidenceUpload, EvidenceList

New API modules: metrics.ts, techniques.ts, tests.ts, evidence.ts, system.ts

All views use TanStack Query for data fetching with proper loading and error states. Role-based UI controls for admin/lead actions.
2026-02-06 16:21:14 +01:00
591b5df250 feat: Phase 7 - Frontend scaffolding and auth (T-023, T-024, T-025)
T-023: Initialize React project
- Vite + React 19 + TypeScript scaffold
- Tailwind CSS v4 with @tailwindcss/vite plugin
- Dependencies: react-router-dom, axios, @tanstack/react-query, lucide-react
- Project structure: api/, components/, pages/, context/, types/, hooks/, lib/

T-024: API client and auth context
- Axios client with JWT interceptor (auto-attach token, clear on 401)
- login() and getMe() API functions
- AuthContext: user state, login, logout, isAuthenticated, isLoading
- Token persistence via localStorage with hydration on mount
- TypeScript types for all backend models

T-025: Login page and layout
- LoginPage with form, error handling, redirect on success
- Layout with sidebar + header + Outlet
- Sidebar with role-aware navigation (System only for admin)
- ProtectedRoute wrapper with role-based access control
- Routes: /login, /dashboard, /techniques, /tests, /system
2026-02-06 16:09:50 +01:00
52d230628d feat: Phase 6 - Automated intel scanning (T-021, T-022)
- Add intel_service.py: RSS feed scanner for threat intelligence
  Searches CISA, NIST NVD, SANS ISC, BleepingComputer, The Hacker News,
  Krebs on Security for mentions of MITRE technique IDs and names
- New intel items stored in intel_items table with URL deduplication
- Techniques with new intel flagged with review_required=True
- Add POST /system/run-intel-scan endpoint (admin only)
- Register weekly intel scan job in APScheduler (every 7 days)
- Audit log records each scan execution with summary stats
- Update README with new endpoint and project structure
2026-02-06 15:48:57 +01:00
abdb23be33 feat: Phase 5 - Metrics and dashboard API (T-020)
- Add GET /metrics/summary endpoint with global coverage counts and percentage
- Add GET /metrics/by-tactic endpoint with per-tactic coverage breakdown
- Handle multi-tactic techniques (comma-separated) counting in each tactic
- Add CoverageSummary and TacticCoverage Pydantic schemas
- Update README with metrics endpoints and project structure
2026-02-06 15:33:37 +01:00
b11854fdab feat: Phase 4 - MITRE ATT&CK sync and scheduled job (T-018, T-019)
- Add MITRE sync service via TAXII 2.0 with GitHub fallback
- Upsert attack-pattern objects into techniques table (691 techniques)
- Detect name/description changes and flag review_required on re-sync
- Add APScheduler background job running every 24h
- Add POST /system/sync-mitre endpoint (admin only)
- Add GET /system/scheduler-status endpoint (admin only)
- Configure logging for scheduler and sync visibility
- Update README with new endpoints and project structure
2026-02-06 15:28:53 +01:00
4f6dd838fd feat: Phase 3 - CRUD core for Techniques, Tests and Evidence (T-014 to T-017)
- Add Pydantic schemas for Technique, Test and Evidence
- Add CRUD endpoints for Techniques (list with filters, detail, create, update, review)
- Add CRUD endpoints for Tests (create, detail, update, validate, reject)
- Add evidence upload with SHA-256 integrity and presigned download URLs
- Add MinIO/S3 storage client with bucket auto-creation on startup
- Add status_service to recalculate technique coverage from test results
- Add require_any_role RBAC dependency for multi-role authorization
- Update README with API endpoints reference and project structure
2026-02-06 13:52:27 +01:00
508f0723af feat: Phase 2 - Authentication and authorization (T-010 to T-013) 2026-02-06 13:15:25 +01:00
ec65991ac1 feat: Phase 1 - Data models and migrations (T-004 to T-009)
Implements all database models for the Aegis platform with full
Alembic migration support.

Models created:
- User: Authentication with role-based access control
- Technique: MITRE ATT&CK techniques with coverage status tracking
- Test: Security tests with validation workflow (draft/review/validated)
- Evidence: File metadata for test evidence (stored in MinIO)
- IntelItem: Threat intelligence items linked to techniques
- AuditLog: System-wide audit trail with JSONB details

Enumerations:
- TechniqueStatus: not_evaluated, in_progress, validated, partial, etc.
- TestState: draft, in_review, validated, rejected
- TestResult: detected, not_detected, partially_detected

Services:
- audit_service.py: log_action() helper for audit logging

All models include proper foreign key relationships and PostgreSQL
enum types are managed correctly in migrations (create/drop).
2026-02-06 12:26:26 +01:00
b479acdea0 feat: Phase 0 - Infrastructure and scaffolding (T-001 to T-003)
This commit establishes the foundational infrastructure for the Aegis
MITRE ATT&CK Coverage Platform.

T-001: Initialize project and Docker Compose
- Set up Docker Compose with PostgreSQL 15, MinIO, and FastAPI backend
- Create basic FastAPI application with health endpoint
- Configure persistent volumes for data storage

T-002: Configuration and database connection
- Add centralized configuration using pydantic-settings
- Implement SQLAlchemy database connection with session management
- Configure MinIO and JWT settings

T-003: Initialize Alembic for migrations
- Set up Alembic with PostgreSQL connection from settings
- Create initial empty migration
- Configure autogenerate support for future models

Also includes:
- Professional README with setup instructions
- Comprehensive .gitignore for Python/Node/Docker
- Project task plan (AegisTestPlan.md)
2026-02-06 11:28:30 +01:00