feat: Phase 6 - Automated intel scanning (T-021, T-022)
- Add intel_service.py: RSS feed scanner for threat intelligence Searches CISA, NIST NVD, SANS ISC, BleepingComputer, The Hacker News, Krebs on Security for mentions of MITRE technique IDs and names - New intel items stored in intel_items table with URL deduplication - Techniques with new intel flagged with review_required=True - Add POST /system/run-intel-scan endpoint (admin only) - Register weekly intel scan job in APScheduler (every 7 days) - Audit log records each scan execution with summary stats - Update README with new endpoint and project structure
This commit is contained in:
@@ -1,9 +1,13 @@
|
||||
"""Scheduled job for periodic MITRE ATT&CK synchronisation.
|
||||
"""Scheduled background jobs.
|
||||
|
||||
Uses APScheduler's ``BackgroundScheduler`` to run :func:`sync_mitre` every
|
||||
24 hours. The job manages its own database session (created on entry,
|
||||
closed in ``finally``) so it is fully independent from FastAPI's
|
||||
request-scoped sessions.
|
||||
Registers periodic tasks on an APScheduler ``BackgroundScheduler``:
|
||||
|
||||
* **MITRE sync** — every 24 hours (see :func:`sync_mitre`)
|
||||
* **Intel scan** — every 7 days (see :func:`scan_intel`)
|
||||
|
||||
Each job manages its own database session (created on entry, closed in
|
||||
``finally``) so it is fully independent from FastAPI's request-scoped
|
||||
sessions.
|
||||
"""
|
||||
|
||||
import logging
|
||||
@@ -12,6 +16,7 @@ from apscheduler.schedulers.background import BackgroundScheduler
|
||||
|
||||
from app.database import SessionLocal
|
||||
from app.services.mitre_sync_service import sync_mitre
|
||||
from app.services.intel_service import scan_intel
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -22,6 +27,11 @@ logger = logging.getLogger(__name__)
|
||||
scheduler = BackgroundScheduler()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Job functions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _run_mitre_sync() -> None:
|
||||
"""Execute a MITRE sync inside its own DB session."""
|
||||
logger.info("Scheduled MITRE sync job starting...")
|
||||
@@ -35,11 +45,33 @@ def _run_mitre_sync() -> None:
|
||||
db.close()
|
||||
|
||||
|
||||
def start_scheduler() -> None:
|
||||
"""Register the MITRE sync job and start the background scheduler.
|
||||
def _run_intel_scan() -> None:
|
||||
"""Execute an intel scan inside its own DB session."""
|
||||
logger.info("Scheduled intel scan job starting...")
|
||||
db = SessionLocal()
|
||||
try:
|
||||
summary = scan_intel(db)
|
||||
logger.info("Scheduled intel scan job finished — %s", summary)
|
||||
except Exception:
|
||||
logger.exception("Scheduled intel scan job failed")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
The job runs every **24 hours**. It does **not** fire immediately on
|
||||
startup — the first execution happens 24 h after the application boots.
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Scheduler bootstrap
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def start_scheduler() -> None:
|
||||
"""Register all periodic jobs and start the background scheduler.
|
||||
|
||||
Jobs registered:
|
||||
|
||||
* ``mitre_sync`` — every **24 hours**
|
||||
* ``intel_scan`` — every **7 days**
|
||||
|
||||
Neither job fires immediately on startup.
|
||||
"""
|
||||
scheduler.add_job(
|
||||
_run_mitre_sync,
|
||||
@@ -49,5 +81,13 @@ def start_scheduler() -> None:
|
||||
name="MITRE ATT&CK sync (every 24h)",
|
||||
replace_existing=True,
|
||||
)
|
||||
scheduler.add_job(
|
||||
_run_intel_scan,
|
||||
trigger="interval",
|
||||
weeks=1,
|
||||
id="intel_scan",
|
||||
name="Intel scan (every 7d)",
|
||||
replace_existing=True,
|
||||
)
|
||||
scheduler.start()
|
||||
logger.info("MITRE sync scheduler started (interval=24h)")
|
||||
logger.info("Background scheduler started — mitre_sync (24h), intel_scan (7d)")
|
||||
|
||||
Reference in New Issue
Block a user