Move layer dispatch, entity-not-found checks, and validation from router to heatmap_service. Router now only validates requests, calls service, and formats responses (no HTTPException, no business logic). Service raises EntityNotFoundError/BusinessRuleViolation instead of returning None. Add build_navigator_export() for centralized dispatch. 29 new tests (253 total, 0 failures).
validate_as_red/blue_lead now delegate to TestEntity. check_dual_validation routes through entity instead of assigning test.state directly. Side effects dispatched via domain events. Entity raises InvalidOperationError for backward compat. Removed 4 dead V1 xfail tests, fixed 2 real test issues. 224 passed, 0 xfailed.
- Add test_test_entity.py with 46 pure unit tests covering the full domain entity
- Fix _FakeSettings in 11 test files (REPORT_TEMPLATES_DIR, JIRA, TEMPO)
- Fix stale db.commit assertions to db.flush after UoW refactor
- Add missing mock fields for TestEntity.from_orm compatibility
- Make database.py skip pool args for SQLite in test environment
- Disable slowapi rate limiter in test client fixture
- Inject test engine into app.database to fix threading errors
- Update role assertions to match current require_any_role policy
- Mark 6 legacy V1 endpoint tests as xfail (replaced by V2 workflow)
str() on models.enums.TestState produces 'TestState.red_executing' instead of 'red_executing'. Use .value to extract the plain string before constructing the domain TestState.
transition_state() now hydrates a TestEntity from the ORM model and delegates state validation to entity.transition_to(). The entity is authoritative for which transitions are valid; VALID_TRANSITIONS and can_transition() are kept for backward compatibility.
Also adds public transition_to() method to TestEntity as the stable API surface for callers that need a single validated transition without lifecycle side-effects.
- Create domain/errors.py as canonical error hierarchy: DomainError, InvalidStateTransition, PermissionViolation, BusinessRuleViolation, EntityNotFoundError, DuplicateEntityError
- InvalidOperationError now inherits from BusinessRuleViolation for semantic consistency
- Convert domain/exceptions.py to backward-compatible re-export shim with legacy aliases (DomainException, InvalidTransitionError, AuthorizationError)
- Update error_handler.py to import from domain/errors.py and map all new error types
- Update main.py to register DomainError (new base) as the exception handler root
- Add UniqueConstraint(test_id, detection_rule_id) named uq_tdr_test_rule to TestDetectionResult model
- Alembic b025: safely deduplicate existing rows before creating constraint
- Create heatmap_service.py with all layer-building logic (coverage, threat-actor, detection-rules, campaign)
- Service is framework-agnostic: no FastAPI imports, no HTTPException, no db.commit()
- Fix N+1 in coverage and threat-actor layers: bulk-fetch test_counts and rule_counts with GROUP BY
- Router reduced from 528 to 140 lines: validates request, calls service, returns response
- Add UnitOfWork context manager in domain/unit_of_work.py with commit/rollback/flush API and auto-rollback on exception
- Remove all db.commit() from test_workflow_service (8 calls), notification_service (4 calls), status_service (1 call)
- Services now only stage changes via db.add/db.flush; caller owns the transaction boundary
- Update routers/tests.py: wrap 9 workflow endpoints in UnitOfWork context managers
- Update routers/notifications.py: wrap mark_as_read and mark_all_as_read in UnitOfWork
- Replace per-technique calculate_technique_score loop with bulk_technique_scores() from scoring_service
- Snapshot creation now runs ~10 fixed queries instead of N*5+N*5 (was ~2000+ for 200 techniques)
- Add bulk_technique_scores() that pre-fetches all scoring data in 5 aggregated GROUP BY queries instead of N*5 per-technique queries
- Rewrite calculate_organization_score to use bulk data (N*5+5 queries -> 10 fixed queries)
- Rewrite calculate_tactic_score and calculate_actor_coverage_score to use bulk data
- Preserve calculate_technique_score single-technique API for router-level calls
- Declare __table_args__ on Test with 5 indexes: technique_id, state, created_at, (technique_id,state), (state,created_at)
- Declare __table_args__ on AuditLog with 3 indexes: (entity_type,entity_id), timestamp, (entity_type,entity_id,action)
- Alembic b024: create only the 2 new indexes (ix_tests_created_at, ix_tests_state_created_at); existing indexes from b005/b018/b019 are preserved
- Model index names aligned with existing migration names to prevent duplicates
- Replace default=datetime.utcnow with server_default=func.now() across all 16 models (17 columns) for consistent, timezone-aware timestamps from PostgreSQL
- Upgrade DateTime columns to DateTime(timezone=True) for timestamptz storage
- Configure SQLAlchemy engine pool: pool_size=20, max_overflow=10, pool_recycle=3600, pool_pre_ping=True
- Remove unused datetime imports from model files
- Add must_change_password field to User model with migration b023
- Add POST /auth/change-password endpoint with password policy validation
- Add require_password_changed dependency to block requests until password is changed
- Add ChangePasswordModal with live password policy checklist (forced on first login)
- Show password policy in CreateUserModal and EditUserModal
- Fix backend permissions: tests, campaigns, templates, reports, evidence, worklogs
- red_tech/blue_tech: execute only, cannot create tests/campaigns/templates
- red_lead/blue_lead: create/edit tests/campaigns/templates, generate reports, no system access
- viewer: read-only everywhere, can generate reports
- Fix frontend role checks across TestDetailPage, TestDetailHeader, TeamTabs, TestsPage, CampaignsPage, CampaignDetailPage, Sidebar
Tarea 4.1 — OSINT Enrichment:
- Add OsintItem model with source_type, severity, CVSS metadata, review flag
- Add Alembic migration b022 with osint_items table and optimized indexes
- Add osint_enrichment_service with NVD API integration, deduplication, rate limiting
- Add OSINT router: GET /osint/items, /osint/summary, /osint/technique/{id}
- Add POST /osint/items/{id}/review to mark items as reviewed
- Add POST /osint/enrich/{technique_id} for manual single-technique enrichment
- Techniques with new CVEs are automatically flagged review_required=True
- Register weekly enrichment job in APScheduler
- Add NVD_API_KEY config setting for optional increased rate limits
Tarea 4.2 — Stale Coverage Detection:
- Add stale_detection_service that flags techniques with no validated test
in the last N days, or never-validated but with a coverage status
- Configurable threshold via STALE_THRESHOLD_DAYS setting (default 365)
- Register daily stale detection job in APScheduler
- Only flags techniques not already marked review_required
Pause/Resume timer:
- Add paused_at, red_paused_seconds, blue_paused_seconds fields to Test model
- Add pause_timer/resume_timer workflow functions with accumulated pause tracking
- Auto-resume on phase submit; subtract paused time from worklog duration
- Add POST /tests/{id}/pause-timer and resume-timer endpoints
- Update LiveTimer component with pause/resume button and paused visual state
- Wire pause/resume mutations through TestDetailPage and TestDetailHeader
Professional Reporting Engine - Fase 2:
- Add ReportEngine service with Jinja2 HTML rendering, WeasyPrint PDF, and docxtpl DOCX
- Add corporate CSS stylesheet with cover page, data tables, stats grid, findings
- Create purple_campaign, coverage_report, and executive_summary HTML templates
- Add report_generation_service collecting domain data for each report type
- Add professional_reports router: GET /reports/generate/purple-campaign/{id}, coverage-summary, executive-summary
- Add analytics router with flat JSON endpoints for PowerBI: /coverage, /tests, /trends, /operators
- Add advanced_metrics router: /coverage-by-tactic, /never-tested, /avg-validation-time, /detection-rate-trend
- Add weasyprint and docxtpl to requirements.txt
- Add REPORT_TEMPLATES_DIR, REPORT_OUTPUT_DIR, COMPANY_NAME, COMPANY_LOGO_PATH to config
- Add red_started_at/blue_started_at timing fields to Test model with Alembic migration
- Modify workflow transitions to auto-create integrity-hashed worklogs: Start Execution records red_started_at, Submit to Blue Team stops Red timer and creates worklog then starts Blue timer, Submit for Review stops Blue timer and creates worklog
- Auto-sync worklogs to Tempo when test has a Jira link
- Add LiveTimer component showing real-time elapsed counter during active phases
- Clear timing fields on test reopen
- Fix campaign test management: replace broken navigate-to-tests flow with AddTestToCampaignModal that lets users search and add existing tests directly from the campaign detail page
Replace all sa.Enum / op.create_table / ALTER TABLE approach with a
single op.execute() containing raw DDL. This sidesteps every
SQLAlchemy hook (enum auto-create, default cast conflicts) by letting
PostgreSQL handle CREATE TYPE IF NOT EXISTS, CREATE TABLE IF NOT
EXISTS, and CREATE INDEX IF NOT EXISTS directly.
Replace sa.Enum column types with raw SQL DO $$ IF NOT EXISTS blocks
for enum creation, then sa.Text columns + ALTER TYPE USING casts.
This completely bypasses SQLAlchemy's _on_table_create hook that
triggers CREATE TYPE without checkfirst, causing DuplicateObject
on PostgreSQL when the entrypoint retries after a failed migration.
Use create_type=False on sa.Enum column references inside
op.create_table so PostgreSQL does not attempt to CREATE TYPE
again after we already created them with checkfirst=True.
Full Jira/Tempo pipeline: link Aegis entities to Jira issues, auto-sync
status hourly, log time internally with integrity hashing, and optionally
push worklogs to Tempo.
- 1.1 JiraLink model + Worklog model: Alembic migration b020 with indexes,
enums (jiralinkentitytype, jirasyncdirection), and integrity_hash column
- 1.2 Jira service: atlassian-python-api wrapper with lazy singleton client,
search/create/sync operations, feature-flagged via JIRA_ENABLED
- 1.3 Jira router: CRUD endpoints for /jira/links, /jira/search,
/jira/create-issue with audit logging and entity-to-issue auto-creation
- 1.4 Tempo service: worklog push via tempo-api-python-client, auto-log from
test completions when TEMPO_ENABLED, graceful fallback on failure
- 1.5 Worklog service + router: immutable internal time records with SHA-256
integrity hash, CRUD at /worklogs, /worklogs/{id}/verify endpoint
- 1.6 Frontend: JiraLinkPanel component (search, link, sync, unlink) and
WorklogTimeline component (timeline view, manual log form) integrated into
TestDetailPage sidebar, CampaignDetailPage grid, TechniqueDetailPage
- 1.7 Jira sync job: APScheduler hourly job syncs all links from Jira,
registered in background scheduler alongside existing jobs
Foundational changes required before any new feature work can begin.
- 0.1 Redis infrastructure: add redis:7-alpine to docker-compose dev and prod,
REDIS_URL config, singleton client in app/infrastructure/redis_client.py
- 0.2 Token blacklist on Redis SEC-001: replace in-memory dict with Redis SETEX
keyed by jti, auto-expiring TTL derived from token exp
- 0.3 Database indexes SR-006: Alembic migration b019 with 5 composite indexes
for scoring, MTTD/MTTR, remediation, and notification queries
- 0.4 Domain exceptions TD-003: app/domain/exceptions.py with typed errors,
error_handler middleware mapping them to HTTP, services decoupled from FastAPI
- 0.5 Fix silenced exceptions TD-007: replace 4 bare except-pass blocks in
test_workflow_service with logger.warning with exc_info
- 0.6 CI pipeline TD-009: GitHub Actions workflow with Postgres and Redis
service containers, ruff lint, pytest; ruff.toml for baseline config
Critical (1-3):
- Replace hardcoded admin credentials with secure auto-generation (seed.py)
- Enforce SECRET_KEY configuration, fail in production if missing (config.py)
- Add Zip Slip and Zip Bomb protection to all ZIP import services
High/Medium (4-9):
- Add 50MB file size limit and extension whitelist to evidence uploads
- Configure CORS origins via environment variable instead of hardcoded
- Migrate JWT storage from localStorage to HttpOnly cookies (frontend+backend)
- Add rate limiting (5/min) on login endpoint via slowapi
- Replace generic dict payloads with Pydantic schemas (mass assignment)
Medium (10-17):
- Check is_active on login to prevent disabled users from authenticating
- Sanitize exception messages in API responses (system, data_sources)
- Escape LIKE wildcards in all ilike search filters across 8 routers
- Run Docker container as non-root user (appuser)
- Make MINIO_SECURE configurable via environment variable
- Add password complexity policy (12+ chars, upper/lower/digit/special)
- Implement JWT token revocation via in-memory blacklist + reduce TTL to 15min
- Replace xml.etree with defusedxml to prevent Billion Laughs attacks
Low (18-20):
- Add security headers to Nginx (CSP, X-Frame-Options, HSTS-ready, etc.)
- Disable Swagger UI/ReDoc/OpenAPI in production
- Restrict /health endpoint to internal networks via Nginx ACL
Also: rewrite install.sh as interactive wizard for guided deployment,
fix test-from-template validation error (technique_id UUID vs MITRE ID)
- Auto-detect project root from script location so it works from any dir
- Fail properly if docker-compose build fails (was hidden by pipe)
- Use docker exec for backend health checks (port 8000 not exposed in prod)
- Add fallback API access via docker exec if nginx not reachable yet
- Show backend logs during wait for better debugging
- Increase timeouts for MITRE sync and data source sync
- Fix hardcoded localhost:8000 URLs in frontend to use relative /api/v1
path (works with Nginx proxy in prod and VITE_API_URL in dev)
- Create production entrypoint (entrypoint.prod.sh) that runs migrations,
seeds, and starts uvicorn with 4 workers (no --reload)
- Create comprehensive install.sh script for production deployment that
generates secure .env, builds containers, waits for health, and
optionally triggers initial MITRE sync
- Update docker-compose.prod.yml to use production entrypoint
- Update Dockerfile to make both entrypoints executable
- Remove init.ps1 (production will always be Linux)
- Update README with production deployment instructions
- Rewrite D3FEND import to use tactic-level APIs for reliable technique
fetching with proper ontology IRIs, descriptions, and tactic assignments
- Fix D3FEND technique URLs to use canonical IRI (no more 404s)
- All 255 D3FEND techniques now have descriptions from the official API
- Change Deactivate button color to red in template management table
- Add custom Aegis logo and favicon replacing default Vite assets
- Remove unused old API parsing code and clean up fallback list
- Make D3FEND defense cards clickable with expandable details and external link
- Fix D3FEND URLs to use PascalCase technique names matching the ontology
- Remove duplicate Import Atomic Red Team from System page (use Data Sources)
- Add bulk Activate All / Deactivate All buttons with confirmation modal
- Fix template admin list to show both active and inactive templates
- Add PATCH /test-templates/bulk-activate backend endpoint
- Auto-seed data sources on container startup via entrypoint.sh
- Fix SigmaHQ, CALDERA, GTFOBins import issues
- Register D3FEND sync handler in data sources router
- Add CIS Controls v8 compliance framework import
- Expand Test Catalog source filters (CALDERA, LOLBAS, GTFOBins)
- Campaign Generate from Threat Actor now opens actor selector modal
- Add coverage snapshot creation button to Comparison page
- Update README with accurate data source and feature documentation
