Commit Graph

230 Commits

Author SHA1 Message Date
kitos e9a3985a1f fix(jira): create test tickets under campaign on activation
When a campaign is activated (Start), iterate all its tests and create
Jira tickets nested under the campaign ticket for any test that doesn't
already have one. Mirrors the pattern used in generate_campaign_from_actor.
2026-05-27 10:53:39 +02:00
kitos 3d8f445d1b feat(tempo): per-user Tempo API token — same pattern as Jira token
Each user can now store their own personal Tempo API token in their
profile settings. Time is logged using each user's own credentials.

Backend:
- Migration b044: adds tempo_api_token column to users table
- User model: adds tempo_api_token column
- UserPreferencesUpdate: adds tempo_api_token field (write-only)
- UserOut: adds tempo_api_token (excluded) + tempo_token_set bool;
  @model_validator derives both jira_token_set and tempo_token_set
- users router: handles tempo_api_token same as jira_api_token
  (empty string clears it, never returned in responses)
- tempo_service: refactored to per-user token; has_tempo_configured(),
  get_user_tempo_client(user) use user.tempo_api_token; global
  TEMPO_ENABLED still acts as kill-switch
- system router: /system/tempo-test now uses current user's personal
  token (any role); removed global TEMPO_API_TOKEN dependency

Frontend:
- settings.ts: UserPreferencesUpdate.tempo_api_token, UserMeOut.tempo_token_set
- SettingsPage ProfileSection: Tempo Integration section with password
  field, show/hide toggle, configured badge, and Test Tempo button —
  mirrors the Jira token UX exactly
- JiraConfigSection: removed stale global Tempo test block
2026-05-27 10:46:38 +02:00
kitos 8577588a21 fix(jira): correct browse URL, rename Procedure to Proof of Concept; feat(tempo): debug endpoint + UI
Jira URL fix:
- JiraLinkPanel now fetches the configured Jira base URL via getJiraConfig()
  instead of hardcoding https://jira.atlassian.com; falls back to the old
  value if config is not yet loaded

Description fix:
- _build_test_description: renamed 'h3. Procedure' -> 'h3. Proof of Concept'
  so the procedure/tool block maps to the correct Jira field label

Tempo debug:
- New POST /system/tempo-test endpoint: checks TEMPO_ENABLED, token,
  user jira_account_id, and makes a real API call; always returns HTTP 200
  with status field (Cloudflare-safe)
- docker-compose.prod.yml: added TEMPO_ENABLED, TEMPO_API_TOKEN,
  TEMPO_DEFAULT_WORK_TYPE env vars (default off, ready to enable)
- SettingsPage: added 'Test Tempo Connection' button in Jira admin tab
  with clear feedback showing what's missing
2026-05-27 10:33:57 +02:00
kitos 001cefb882 fix(jira): remove priority field from issue creation — OFS project has non-standard priorities
The OFS Jira project does not have the default Jira priority scheme
(Highest/High/Medium/Low/Lowest), causing a 'priority selected is invalid'
error on every ticket creation. Removing the priority field lets Jira use
the project default.
2026-05-27 10:18:16 +02:00
kitos 1ce427db88 feat(jira): implement full ticket hierarchy for campaigns and tests
Jira tickets now follow the correct hierarchy:
  OFS-9107 (system parent)
  ├── Standalone test ticket  (unchanged — was already working)
  └── Campaign ticket         (NEW — created on campaign creation)
      ├── Test 1 ticket       (NEW — created per test)
      └── Test 2 ticket       (NEW — created per test)

Changes:
- jira_service: add auto_create_campaign_issue() — creates campaign
  ticket as child of OFS-9107; stores JiraLink(entity_type=campaign)
- jira_service: add get_campaign_jira_key() / get_test_jira_key()
  helpers to look up existing Jira links by entity
- jira_service: auto_create_test_issue() gains parent_ticket_override
  param — when set, uses it as parent instead of OFS-9107
- campaigns router/create_campaign: triggers auto_create_campaign_issue
  after commit
- campaigns router/from-threat-actor: triggers campaign ticket then
  iterates campaign_tests and creates each test ticket under it
- campaigns router/add_test_to_campaign: if campaign has a Jira ticket
  and the test has none yet, creates test ticket under campaign ticket
2026-05-27 10:13:09 +02:00
kitos 95b4c4ea65 fix(jira): fallback connected_as to auth email, improve 401 error detail
- jira-test: when myself() returns empty displayName/emailAddress/name,
  fall back to the configured Atlassian auth email so 'Connected as:' is
  never empty
- jira-test: 401 error message now includes which email was used, making
  misconfigured Jira email easier to diagnose
- jira-test: missing jira_url now returns HTTP 200 {status: error} instead
  of HTTP 400, consistent with Cloudflare-safe pattern
2026-05-26 18:04:51 +02:00
kitos 174b7e8d24 fix(jira): always return HTTP 200 from jira-test + strip trailing slash
- jira-test now returns {status: "ok"|"error", message: ...} with
  HTTP 200 so Cloudflare never intercepts the response
- jira_service strips trailing slash from URL before creating Jira
  client (avoids double-slash in REST paths)
- Frontend reads data.status field instead of HTTP status code
2026-05-26 17:42:12 +02:00
kitos d307039a41 fix(jira): use model_validator(after) for jira_token_set + timeout on test
FastAPI uses __pydantic_validator__.validate_python() which bypasses
model_validate() overrides. Switch to @model_validator(mode='after')
which the Pydantic Rust core always calls, so jira_token_set is now
correctly derived from the excluded jira_api_token field.

Also add a 10s timeout to the jira-test endpoint and better error
messages (the Atlassian library's "Expecting value" JSON error was
ambiguous).
2026-05-26 17:36:35 +02:00
kitos aaa344ab79 fix(settings): update cache immediately on save instead of invalidating
Using setQueryData with the PATCH response means jira_token_set is
reflected in the UI instantly — no extra GET round-trip that could
leave the badge stale.
2026-05-26 17:20:40 +02:00
kitos 1f08eb014b fix(settings): use useEffect for jira field init, fix token save UX
Replace render-body setState with useEffect so field initialisation
is idiomatic React and never races with user input. Also clarifies
placeholder text: empty token field = keep current, not clear it.
2026-05-26 17:04:22 +02:00
kitos e08d8a9beb feat(jira): add editable jira_email field per user
Users can now set a separate Atlassian email for Jira authentication
in Settings → Profile → Jira Integration. Falls back to the Aegis
account email when not set, so existing setups are unaffected.

- Migration b043: adds jira_email column to users table
- User model/schema: expose jira_email read/write
- jira_service: _effective_jira_email() uses jira_email ?? email
- Frontend: replaces read-only email display with editable input
2026-05-26 16:40:46 +02:00
kitos e4342e1c3f feat(settings): Jira config UI — admin config tab + per-user token in Profile
- backend: add parent_ticket field to JiraConfigOut/JiraConfigUpdate/_JIRA_KEYS
- backend: add get_jira_parent_ticket() helper in jira_service; use it in auto_create_test_issue() to set issue parent
- frontend/api: add jira_token_set to UserMeOut, jira_api_token to UserPreferencesUpdate, and full JiraConfigOut/Update types with getJiraConfig/updateJiraConfig/testJiraConnection functions
- frontend: expand ProfileSection with Jira API token password field (show/hide), token status badge, and account-id field
- frontend: add JiraConfigSection component (admin): enabled toggle, URL, project key, parent ticket, save + test connection
- frontend: add Jira tab (admin-only) with Link2 icon in SettingsPage sidebar
2026-05-26 16:23:24 +02:00
kitos d3831b8ed9 fix(jira): correct down_revision id in b042 migration 2026-05-26 15:59:23 +02:00
kitos 87af1735ce feat(jira): per-user auth, lifecycle hooks, admin config endpoints
- Add jira_api_token field to User model + migration b042
- Per-user Jira client: user's corporate email + personal Atlassian token
- Admin-configurable Jira URL/project via system_configs (GET/PATCH /system/jira-config + POST /system/jira-test)
- Auto-create Jira ticket when a test is created (non-fatal)
- Push lifecycle comments on every state transition: draft→red_executing→blue_evaluating→in_review→validated/rejected→draft
- Rich ticket descriptions with technique, MITRE ID, priority from severity, labels
- UserOut.jira_token_set (bool) instead of exposing raw token
- PATCH /users/me/preferences now accepts jira_api_token
2026-05-26 15:56:28 +02:00
kitos f3109644cb docs(wiki): add wiki creation script for Gitea
Creates 14 comprehensive wiki pages covering architecture, roles,
test lifecycle, API reference, security, deployment, and QA guide.
Run from a machine with access to internal Gitea (192.168.1.107:3000).
2026-05-22 14:30:21 +02:00
kitos ee1c773073 test(qa): fix all test failures - 77/77 passing
- Accept 409 for playbook creation (unique per technique+type is correct behavior)
- Space logins 13s apart to avoid 5/min rate limit on login endpoint
- Reuse admin session from initial login to avoid duplicate login call
2026-05-22 11:05:24 +02:00
kitos a294300052 security(webhooks): restrict all webhook endpoints to admin-only
fix(qa): pass technique_id and test_id context between test suites
fix(qa): playbook creation requires technique_id field
fix(qa): lesson creation requires what_happened and root_cause fields
fix(qa): campaign complete test now activates with test before completing
fix(qa): rate limit test notes loopback exemption instead of failing
2026-05-22 10:56:15 +02:00
kitos f72287984c test(qa): add automated QA runner for all roles and access control 2026-05-22 10:30:54 +02:00
kitos 8a51f98631 security: fix 6 vulnerabilities identified in SDLC audit
- fix(auth): enforce API key scopes in require_role/require_any_role;
  attach _api_key_scopes to user on API key auth; add require_scope()
  dependency — scopes were stored but never enforced (CWE-285)

- fix(sso): read SECURE_COOKIES env var for SSO cookie instead of
  hardcoded secure=False — SAML sessions now respect HTTPS config (CWE-614)

- fix(webhooks): SSRF prevention — validate webhook URLs against private
  and reserved CIDRs at creation/update time (CWE-918)

- fix(knowledge): restrict playbook/lesson create, update and restore
  to admin/red_lead/blue_lead roles — was open to any authenticated user (CWE-284)

- fix(alerts): restrict alert acknowledge/resolve/dismiss to admin/lead
  roles — any user could silence security alerts (CWE-284)

- security: delete get_admin_creds.py, check_auth.py, deploy.py scripts
  containing hardcoded root SSH credentials and production DB access;
  add scripts/.gitignore to prevent reintroduction (CWE-798)
2026-05-22 09:46:29 +02:00
kitos 8e7ee1494e fix(scripts): fix verify_gaps.py Gap 1 check — call start_scheduler() before checking registered jobs 2026-05-21 17:28:34 +02:00
kitos 6f24d340d2 fix(alerts): import User model in operational_alert_service to fix NameError in _dispatch_inapp_notifications 2026-05-21 17:11:35 +02:00
kitos da89a9ae51 test: gap verification script for Phase 13 gaps 2026-05-21 16:08:45 +02:00
kitos 6665efd276 feat(alerts): close Phase 13 gaps — hourly job + webhook + in-app notifications
- Add dispatch_webhook_targeted() to webhook_service for rule-specific delivery
- evaluate_all_rules() now dispatches in-app notifications (admins/leads) and
  webhooks after each alert fires (targeted + global alert.fired broadcast)
- APScheduler: _run_alert_evaluation() job registered hourly alongside existing jobs
2026-05-21 15:57:41 +02:00
kitos 37f2d6daa6 fix(dashboard): make KpiBlock.snapshot_id Optional to handle missing today snapshot 2026-05-21 15:27:26 +02:00
kitos f2787bf860 feat(alerts): Phase 13 — Operational Alert Engine
AlertRule + AlertInstance models (b041alerts migration), 8 pre-seeded system
rules (high_risk x2, stale_technique, coverage_regression, low_coverage,
expiry_wave, new_technique, orphan_spike), evaluation engine with per-rule
cooldown, full alert lifecycle (acknowledge/resolve/dismiss), custom rule CRUD,
and summary endpoint. Rules seeded at app startup.
2026-05-21 15:25:55 +02:00
kitos 21ed939569 feat(enterprise): Phase 14 — API Key Management + SSO/SAML 2.0
- ApiKey model (SHA-256 hash, prefix, scopes, expiry) + Alembic migration (b040ent)
- SsoConfig model for SAML 2.0 IdP settings (attribute mapping, auto-provision)
- API key auth integrated into get_current_user (aegis_ prefix detection)
- Routers: /api/v1/api-keys (full CRUD + revoke) and /api/v1/sso (metadata, login, callback, config)
- python3-saml added to requirements; Dockerfile adds libxmlsec1-dev for SAML XML signing
- QA script: 52 assertions covering key lifecycle, API key auth, SSO config
2026-05-20 16:43:57 +02:00
kitos 3c077f971e feat(dashboard): Phase 13 — Executive Dashboard
PostureSnapshot model, Alembic migration (b039exec), schemas, service
aggregating all phases (coverage/risk/operations/knowledge/MTTD), and
router at /api/v1/dashboard with executive view, KPIs, coverage-by-tactic,
posture-history, posture-snapshot, and activity-feed endpoints.
2026-05-20 16:20:21 +02:00
kitos d9292fb3ff fix(risk): fix remaining t.technique_id → t.mitre_id in get_recommendations 2026-05-20 16:11:48 +02:00
kitos 6fad769c13 fix(risk): Technique uses status_global and mitre_id (not status/technique_id) 2026-05-20 15:59:26 +02:00
kitos d1443d1ffa fix(risk): correct TechniqueConfidenceScore fields, TechniqueStatus values, Test.result usage 2026-05-20 15:58:03 +02:00
kitos 9d0cb6d67d feat(risk): Phase 12 — Risk Intelligence [FASE-12]
- TechniqueRiskProfile model: per-technique risk scoring (0-100)
- 4-factor weighted scoring: detection_gap(35%) + threat_actors(30%) + osint(20%) + test_failures(15%)
- Risk levels: critical(≥75) / high(≥50) / medium(≥25) / low(≥10) / info
- Detailed scoring_breakdown (JSONB) + actionable recommendations per technique
- Router /api/v1/risk: compute-all, compute-one, list, matrix, summary, recommendations, top
- Alembic migration b038risk (raw SQL, idempotent)
- QA script: 60+ tests across all endpoints
2026-05-20 15:31:38 +02:00
kitos 3f174e7d89 fix(qa11): use relative version checks for idempotent runs 2026-05-20 15:26:38 +02:00
kitos 6c3f00f6e6 fix(qa11): make QA idempotent with cleanup step + robust error handling 2026-05-20 15:25:46 +02:00
kitos ed579fb8f7 fix(knowledge): use EntityNotFoundError/DuplicateEntityError instead of DomainError(status_code=) 2026-05-20 15:21:36 +02:00
kitos 612dec7a93 fix(qa11): use correct production credentials 2026-05-20 15:14:58 +02:00
kitos a138c7a8ed fix(qa11): use production admin credentials 2026-05-20 14:31:46 +02:00
kitos 6c4517c7f3 fix(qa11): fix get_token to use form data + fix check() bug 2026-05-20 14:27:41 +02:00
kitos dd1f0e472f feat(knowledge): Phase 11 — Knowledge Management (Playbooks + Lessons Learned) [FASE-11]
- Playbooks: versioned Markdown runbooks per technique × type (attack/detect/investigate/respond/hunt)
- PlaybookVersion: immutable snapshots on every update; restore to any previous version
- LessonLearned: post-mortem records linked to tests/campaigns/attack-paths or manual
- Alembic migration b037know (raw SQL, idempotent, no PostgreSQL enums)
- Router /api/v1/knowledge: 14 endpoints for playbooks + lessons + stats
- Pydantic validators for playbook_type, severity, entity_type (422 on invalid)
- Knowledge stats endpoint: totals + breakdown by severity and playbook type
- Soft-delete on both resources; include_inactive filter for admin recovery
- QA script: 70+ tests across CRUD, versioning, filtering, auth, soft-delete, regression
2026-05-20 13:39:05 +02:00
kitos 8c73377571 feat(attack-paths): Phase 10 — Attack Paths & Advanced Purple Team [FASE-10]
Models (5 tables):
  - AttackPath: named reusable attack scenario with template flag
  - AttackPathStep: ordered kill-chain step (technique + test link)
  - AttackPathExecution: a run with Red/Blue leads, timing, stored metrics
  - AttackPathStepResult: per-step detected/not_detected/skipped result
  - TimelineEntry: timestamped Red/Blue/system actions for MTTD/MTTR

Migration b036atk: raw SQL to avoid SQLAlchemy DDL hook issues

Service (attack_path_service.py):
  - Full CRUD for paths + steps (add, update, delete, reorder)
  - Execution lifecycle: create → start → execute steps → complete/abort
  - Pre-creates pending step results on execution creation
  - Auto-adds system timeline entries on key state transitions
  - complete_execution() computes: detection_rate, mttd_seconds,
    furthest_undetected_step, detected/not_detected/skipped counts
  - get_kill_chain_metrics(): per-step breakdown + phase summary

Router /api/v1/attack-paths (20 endpoints):
  POST/GET/PATCH/DELETE attack paths
  GET/POST/PATCH/DELETE steps + reorder
  POST/GET executions per path
  GET/POST/start/complete/abort executions
  POST/GET step results
  POST/GET timeline entries
  GET kill-chain metrics
2026-05-20 13:11:01 +02:00
kitos ab50bcd90e fix(ownership): validate reason+priority in QueueItemCreate to return 422 not 500
POST /ownership/queue with an invalid reason or priority was silently
passing Pydantic and crashing at the DB layer (PostgreSQL enum type
mismatch → 500). Added @field_validator for both fields, matching the
existing validators in QueueItemPatch.
2026-05-19 17:57:34 +02:00
kitos b78593ca10 fix(migration): rewrite b035 with raw SQL to avoid SQLAlchemy DDL hook
SQLAlchemy fires before_create for ALL known enum types when any table
is created via op.create_table, causing DuplicateObject even with
create_type=False. Rewrite both CREATE TABLE statements as raw SQL via
conn.execute(sa.text(...)) and use CREATE TABLE IF NOT EXISTS / CREATE
INDEX IF NOT EXISTS for full idempotency.
2026-05-19 16:54:32 +02:00
kitos 0b81580b44 fix(migration): use DO/EXCEPTION for idempotent enum creation in b035
Replace _enum_exists() helper (which had connection context issues in
Alembic) with PostgreSQL DO $$ BEGIN ... EXCEPTION WHEN duplicate_object
THEN NULL; END $$ blocks, which are truly idempotent regardless of
transaction state.
2026-05-19 16:51:22 +02:00
kitos 95b46a95a8 feat(ownership): Phase 9 — Ownership & Daily Operations [FASE-9]
Backend:
- TechniqueOwnership model: per-technique owner, backup owner, team
- RevalidationQueueItem model: prioritised analyst work queue
  (critical/high/medium/low, reasons: validation_expired/infra_change/
   osint_alert/mitre_update/rule_modified/low_confidence/manual)
- Migration b035ownerq: creates technique_ownerships and
  revalidation_queue_items tables with full indexes

Services:
- ownership_service: set/get technique ownership, bulk assign by tactic
  or platform, orphan reports for techniques and assets
- revalidation_queue_service: smart queue generation (scans expired
  validations, low-confidence techniques, recent infra changes),
  list/create/update queue items, analyst dashboard

Router /api/v1/ownership:
  GET/PUT /ownership/techniques/{id}   — technique ownership
  PATCH   /ownership/assets/{id}       — asset ownership
  GET     /ownership/orphans/techniques — orphan report
  GET     /ownership/orphans/assets     — orphan report
  POST    /ownership/bulk-assign        — bulk by tactic/platform
  GET/POST /ownership/queue             — revalidation queue CRUD
  PATCH   /ownership/queue/{id}         — update item status/assignee
  POST    /ownership/queue/generate     — scan & generate items
  GET     /ownership/analyst-dashboard  — personalised daily view

Scheduler: queue_generation job daily at 02:30 (after decay engine)
2026-05-19 16:48:47 +02:00
kitos b493f92f75 fix(decay-engine): strip tzinfo from validated_at before datetime arithmetic
The previous fix changed _now() to return naive UTC, but the code still
called .replace(tzinfo=utc) on most_recent (from DB) before subtracting.
This caused "can't subtract offset-naive and offset-aware datetimes".
Now we strip tzinfo if present, keeping everything naive UTC consistently.
2026-05-19 16:35:02 +02:00
kitos 61c26ddd0f fix(detection-lifecycle): fix timezone naive/aware mismatch and duplicate technique mapping
- Replace datetime.now(timezone.utc) with datetime.utcnow() in _now() across
  all three Phase 8 files to match DB DateTime column type (naive UTC)
- Guard POST /assets/{id}/techniques/{tid} against duplicate mappings:
  if mapping already exists, update coverage_type/confidence_level instead
  of inserting a duplicate row
2026-05-19 16:29:04 +02:00
kitos 634abc289b feat(dlm): Phase 8 — Detection Lifecycle Management [FASE-8]
Tasks 8.1-8.5:

Models (8.1):
- DetectionAsset: SIEM/EDR/Sigma rule assets with auto-hash
- DetectionTechniqueMapping: N:M asset ↔ technique coverage
- DetectionValidation: immutable validation records with expiry
- TechniqueConfidenceScore: computed multi-factor confidence
- InfrastructureChangeLog: infra changes that invalidate detections
- DecayPolicy: configurable freshness thresholds per platform/tactic

Services (8.2, 8.3):
- detection_asset_service: CRUD + SHA-256 rule hashing + auto-
  invalidation on rule/infra changes
- decay_engine_service: daily decay engine — expires stale validations,
  recalculates confidence (recency/coverage/health/diversity factors),
  processes infrastructure change propagation

Router (8.4): 15 endpoints under /api/v1/detection-lifecycle:
  assets CRUD, technique mappings, validations, confidence scores,
  infrastructure changes, decay trigger, executive dashboard

Scheduler (8.3): decay engine runs daily at 02:00
Seed (8.5): default policy (90/180/365d) + strict initial-access policy
Migration: b034dlm (6 tables, 11 indexes)
2026-05-19 15:45:16 +02:00
kitos 519ddfb7a0 feat(settings): Settings page with email, webhooks, notifications, profile [FASE-8]
- SystemConfig model + migration b033 for runtime key-value config
- GET/PATCH /system/email-config + POST /system/email-test (admin only)
- email_service reads SMTP config from DB (overrides .env)
- Webhooks now accessible to red_lead/blue_lead + admin
- GET /users/me already existed; /users/me/preferences already working
- SettingsPage with 4 role-aware tabs:
  * Profile & Jira: jira_account_id, user info
  * Notifications: role-specific email/in-app toggles (12 prefs)
  * Webhooks: full CRUD + test ping (leads + admin)
  * Email/SMTP: enable toggle, server config, test email (admin only)
- Added /settings route (all authenticated users)
- Settings link added to Sidebar
2026-05-19 15:10:31 +02:00
kitos 7009fcabbf fix(users): add GET /users/me endpoint for current user profile 2026-05-19 14:04:42 +02:00
kitos b714b466c8 feat(phases): implement webhooks (6.1), email (7.1), user preferences (7.2)
- Phase 6.1: WebhookConfig model, CRUD router (/api/v1/webhooks, admin-only),
  dispatch_webhook() with HMAC signing; integrated into test validation,
  campaign completion, and MITRE sync job
- Phase 7.1: SMTP email service with send_test_validated_email,
  send_campaign_completed_email, send_new_mitre_techniques_email;
  notify_role_with_email() added to notification_service
- Phase 7.2: notification_preferences and jira_account_id on User model;
  PATCH /users/me/preferences endpoint; Alembic migrations b031phase6 and b032phase7
2026-05-19 13:40:45 +02:00
kitos ca17675253 fix(audit): show UTC suffix on timestamp display 2026-05-19 13:05:08 +02:00