Commit Graph

16 Commits

Author SHA1 Message Date
kitos 8fea0c1ada feat(refactor): PEP8, type annotations, docstrings and PyJWT security fix 2026-06-11 11:09:41 +02:00
kitos d2a46feba8 refactor(docs+comments): add Google-style docstrings and inline comments across backend
Task D — Google-style docstrings (Args/Returns) on every public function,
method, and class across all 158 Python files in the backend. Zero ruff D
violations (pydocstyle Google convention).

Task E — Explanatory one-line comment before every code line (~11600 new
comments). ruff check passes clean after isort re-sort.
2026-06-11 11:06:55 +02:00
kitos 9ff0f04ba3 refactor(types): add comprehensive type annotations across backend Python codebase
Enable ANN rules in ruff.toml (flake8-annotations) and resolve all 221 violations:

ANN201/ANN202 — return types on 168 public/private functions:
- All 28 FastAPI routers: endpoints annotated with dict/list/specific schema/
  StreamingResponse/FileResponse/JSONResponse as appropriate
- main.py: lifespan→AsyncGenerator[None,None], exception handlers→JSONResponse
- database.py: get_db→Generator[Session,None,None], proxy methods→correct types
- middleware/request_context.py: dispatch→Response with Callable call_next type

ANN001/ANN002/ANN003 — 32 missing argument types:
- seed_demo.py: all db parameters typed as Session
- domain/unit_of_work.py: __aexit__ exc_type/exc_val/exc_tb typed with TracebackType
- services: audit_service user_id→UUID|None, heatmap_service query/model/builder,
  notification_service test→Test, tempo_service test→Test/user→User,
  test_workflow_service test_id→UUID, campaign_crud **fields→object,
  test_crud **fields→object (4 sites)

ANN401 — 16 Any usages resolved:
- Domain entities (campaign/technique/threat_actor/test_entity): replaced Any with
  actual ORM types via TYPE_CHECKING guards to avoid circular imports
- detection_rule_service: test_id/detection_rule_id/evaluator_id→UUID
- score_cache: kept Any with # noqa: ANN401 (genuinely generic cache)
- jira_service/tempo_service: kept Any with # noqa: ANN401 (lazy optional deps)
- d3fend_import_service: _to_str(v: Any) kept with # noqa: ANN401

ANN204/ANN205/ANN206 — special/static/class methods:
- database.py proxy __call__/__getattr__: *args: object/**kwargs: object
- schemas/test.py model_validate: obj→object, **kwargs→object
- sa_technique_repository._int_type→type

All 439 unit tests pass. ruff check app/ → All checks passed!
2026-06-11 11:06:54 +02:00
kitos 8f98bdd273 refactor(pep8): enforce full PEP8 compliance across backend Python codebase
- ruff.toml: select E/W/F/I/N rules, line-length=120, drop legacy ignores
- Auto-fix: sort 82 import blocks (isort), remove 29 unused imports,
  strip 6 trailing-whitespace blank lines in docstrings
- main.py: move setup_logging and settings imports to top (E402)
- errors.py: noqa N818 on DDD exception names (96 call sites, safe)
- intel_service.py: noqa N817 for universal ET alias
- atomic/elastic/sigma import services: move _MAX_UNCOMPRESSED_SIZE and
  _MAX_ENTRIES to module level (N806)
- compliance_import_service.py: move SAMPLE_CONTROLS / CIS_CONTROLS to
  module level; wrap long description strings (N806 + E501)
- snapshot_service.py: move STATUS_ORDER dict to module level (N806)
- sigma_import_service.py: remove dead dedup_key expression (F841)
- threat_actor_import_service.py: remove dead stix_to_actor expression (F841)
- data_source.py, seed_demo.py, campaign_scheduler_service.py,
  lolbas_import_service.py: wrap lines exceeding 120 chars (E501)
- d3fend_import_service.py: per-file E501 ignore (data file with long strings)

All 439 unit tests pass. ruff check app/ → All checks passed!
2026-06-11 11:06:54 +02:00
kitos cd718512ad fix(evidence): use @model_validator(mode='before') so evidences appear in API responses
FastAPI 0.136.1 + Pydantic 2.13.4 serialises responses via TypeAdapter which
calls the compiled Rust validator directly, bypassing any Python-level
`model_validate` classmethod override. The @model_validator(mode='before')
decorator IS invoked by the Rust pipeline, so the evidence red/blue split and
technique field population now run on every serialisation path.

Also eager-load technique in get_test_detail to avoid lazy-load surprises.
2026-05-28 13:37:18 +02:00
kitos c0a0e1aa00 fix(schemas): avoid lazy-load in TestOut.model_validate
Accessing obj.evidences on a session-expired ORM object (mutation endpoints
do commit+refresh without joinload) triggers a lazy query that fails or
returns stale data. Use obj.__dict__.get('evidences') instead — SQLAlchemy
stores joinloaded relationships in __dict__; absent means not loaded.

Mutation endpoints (submit-red, submit-blue, etc.) return empty evidence
lists, which is fine: the frontend invalidates and refetches GET /tests/{id},
which uses joinedload and correctly populates red_evidences / blue_evidences.
2026-05-28 12:06:34 +02:00
kitos e6c188c782 fix(tempo,evidence): fix SystemExit crash + evidence not shown in frontend
tempo: tempoapiclient raises SystemExit (BaseException) on API errors like
'User is invalid' 400 responses; except Exception never catches it, killing
the uvicorn worker and causing a 500. Wrap create_worklog() to intercept
BaseException and re-raise as RuntimeError so callers can catch it safely.

evidence: TestOut schema was missing red_evidences / blue_evidences fields.
The ORM model has evidences loaded via joinedload but they were never
serialized into the API response. Add both fields to TestOut and override
model_validate to split Test.evidences by team, injecting the backend-proxy
download_url for each one (/api/v1/evidence/{id}/file).
2026-05-28 11:57:52 +02:00
kitos 3b20911c93 feat(tempo): blue team Tempo time from pick-up, not queue entry
Previously blue_started_at was set when the RED team submitted evidence
(= queue open time), so Tempo was getting total queue wait time instead
of actual work time.

Changes:
- DB: add blue_work_started_at column (migration b045), set when a blue
  tech explicitly picks up the test (mirrors red_started_at for red team)
- Workflow: new start_blue_work() function + POST /tests/{id}/start-blue-work
  endpoint (blue_tech / blue_lead roles). Cannot be called twice.
- submit_blue_evidence: uses blue_work_started_at (when available) as the
  phase start for the Tempo worklog, falls back to blue_started_at
- reopen_test: clears blue_work_started_at alongside other timing fields
- Tempo: both red_team_execution and blue_team_evaluation now synced;
  correct work_date and description per activity type
- Frontend: "Start Evaluation" button shown in blue_evaluating state when
  blue_work_started_at is null; live timer shows from pick-up time

What each timestamp tracks:
  blue_started_at      = queue entry (SLA / internal tracking)
  blue_work_started_at = pick-up by blue tech (Tempo start)
2026-05-27 11:50:15 +02:00
kitos bdeeed54e1 feat(compliance): data classification fields and retention policies job [FASE-3.5]
Aegis CI / lint-and-test (push) Has been cancelled
2026-05-18 14:17:29 +02:00
kitos 31e116b4ba feat(phase-37): timer pause/resume + professional reporting engine
Aegis CI / lint-and-test (push) Has been cancelled
Pause/Resume timer:
- Add paused_at, red_paused_seconds, blue_paused_seconds fields to Test model
- Add pause_timer/resume_timer workflow functions with accumulated pause tracking
- Auto-resume on phase submit; subtract paused time from worklog duration
- Add POST /tests/{id}/pause-timer and resume-timer endpoints
- Update LiveTimer component with pause/resume button and paused visual state
- Wire pause/resume mutations through TestDetailPage and TestDetailHeader

Professional Reporting Engine - Fase 2:
- Add ReportEngine service with Jinja2 HTML rendering, WeasyPrint PDF, and docxtpl DOCX
- Add corporate CSS stylesheet with cover page, data tables, stats grid, findings
- Create purple_campaign, coverage_report, and executive_summary HTML templates
- Add report_generation_service collecting domain data for each report type
- Add professional_reports router: GET /reports/generate/purple-campaign/{id}, coverage-summary, executive-summary
- Add analytics router with flat JSON endpoints for PowerBI: /coverage, /tests, /trends, /operators
- Add advanced_metrics router: /coverage-by-tactic, /never-tested, /avg-validation-time, /detection-rate-trend
- Add weasyprint and docxtpl to requirements.txt
- Add REPORT_TEMPLATES_DIR, REPORT_OUTPUT_DIR, COMPANY_NAME, COMPANY_LOGO_PATH to config
2026-02-17 17:20:45 +01:00
kitos febf460580 feat(phase-36): automatic Tempo time tracking via workflow buttons + fix campaign test management
- Add red_started_at/blue_started_at timing fields to Test model with Alembic migration

- Modify workflow transitions to auto-create integrity-hashed worklogs: Start Execution records red_started_at, Submit to Blue Team stops Red timer and creates worklog then starts Blue timer, Submit for Review stops Blue timer and creates worklog

- Auto-sync worklogs to Tempo when test has a Jira link

- Add LiveTimer component showing real-time elapsed counter during active phases

- Clear timing fields on test reopen

- Fix campaign test management: replace broken navigate-to-tests flow with AddTestToCampaignModal that lets users search and add existing tests directly from the campaign detail page
2026-02-17 16:59:19 +01:00
kitos 4d124b42dd feat(phase-30): add coverage snapshots, temporal comparison and auto re-testing (T-230 to T-232) 2026-02-10 08:34:29 +01:00
kitos 9ea6ce1326 feat(phase-19): add remediation fields and reports system (T-130, T-131) 2026-02-09 13:58:35 +01:00
kitos a95defcee4 feat(phase-16): enhanced Tests view, Red/Blue dashboard metrics, and Template admin panel (T-122, T-123, T-124) 2026-02-09 13:00:07 +01:00
kitos 7af6be10be feat(phase-11): implement Red/Blue business logic services (T-106, T-107, T-108)
T-106: Create test_workflow_service.py with state-machine transitions for the complete test lifecycle (draft -> red_executing -> blue_evaluating -> in_review -> validated/rejected), dual validation by Red/Blue leads, and reopen capability with field cleanup.

T-107: Update status_service.py to use detection_result from Blue Team instead of legacy result field, and differentiate between partial progress (some validated) vs all-in-progress states.

T-108: Create atomic_import_service.py that downloads the Atomic Red Team repo as a ZIP (avoiding API rate limits), parses all atomics YAML files, and creates idempotent TestTemplate records mapped to MITRE techniques.

Includes validation tests for all three tasks (19 checks total).
2026-02-09 09:58:54 +01:00
kitos 4f6dd838fd feat: Phase 3 - CRUD core for Techniques, Tests and Evidence (T-014 to T-017)
- Add Pydantic schemas for Technique, Test and Evidence
- Add CRUD endpoints for Techniques (list with filters, detail, create, update, review)
- Add CRUD endpoints for Tests (create, detail, update, validate, reject)
- Add evidence upload with SHA-256 integrity and presigned download URLs
- Add MinIO/S3 storage client with bucket auto-creation on startup
- Add status_service to recalculate technique coverage from test results
- Add require_any_role RBAC dependency for multi-role authorization
- Update README with API endpoints reference and project structure
2026-02-06 13:52:27 +01:00