kitos/Aegis
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
Files
0c526c48f9489dd4caeed0b23033909af15cab43
Aegis/# Aegis — Plan de Implementación Consolidado v3.0

4440 lines
150 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Aegis — Plan de Implementación Consolidado v3.0
**Documento de Referencia para Ralph y Claude Code**
**Fecha:** 17 de febrero de 2026
**Alcance:** Deuda técnica + Features nuevas + Transformación a Detection Assurance Platform
**Estimación total:** ~42-52 semanas
---
## Visión
Aegis evoluciona en tres etapas:
1. **Fundamentos (Fase 0):** Resolver deuda técnica bloqueante
2. **Features Operativas (Fases 1-7):** Integrations, reporting, compliance, inteligencia
3. **Detection Assurance Platform (Fases 8-14):** Transformar Aegis de tracker MITRE a sistema de garantía continua de detección — donde cada detección tiene ciclo de vida, ownership, salud medible, y el sistema orquesta proactivamente la revalidación
---
## Índice de Fases
| Fase | Nombre | Duración | Prioridad/Dependencias |
|------|--------|----------|----------------------|
| 0 | Fundamentos Técnicos (deuda técnica bloqueante) | 2 semanas | CRÍTICA — sin dependencias |
| 1 | Integración Jira + Tempo | 3 semanas | Fase 0 (Redis) |
| 2 | Motor de Reporting Profesional | 3-4 semanas | Fase 0 (excepciones de dominio) |
| 3 | Compliance & Hardening de Seguridad | 2-3 semanas | Fase 0 (audit mejorado) |
| 4 | Inteligencia Automática (OSINT + Stale Detection) | 2-3 semanas | Fase 0 (índices BD) |
| 5 | Gestión Operativa Avanzada | 2-3 semanas | Fase 0 (scoring en BD) |
| 6 | Analytics para BI + Webhooks | 2 semanas | Fase 2 |
| 7 | Notificaciones Multi-Canal | 1-2 semanas | Fase 3 |
| 8 | Detection Lifecycle Management (DLM) | 3-4 semanas | Fase 0 (Redis, índices, excepciones) |
| 9 | Ownership & Operativa Diaria | 2-3 semanas | Fase 8 |
| 10 | Attack Paths & Purple Team Avanzado | 3-4 semanas | Fases 8, 9 |
| 11 | Knowledge Management | 2-3 semanas | Fase 8 |
| 12 | Risk Intelligence & Recomendaciones | 2-3 semanas | Fases 4, 8, 9 |
| 13 | Alertas Inteligentes & Integraciones Avanzadas | 2-3 semanas | Fases 6, 8, 9, 12 |
| 14 | SSO/SAML & API Keys (Enterprise Readiness) | 2 semanas | Fase 0 |
> **Nota sobre paralelismo:** Las Fases 1-7 y la Fase 8 pueden ejecutarse en paralelo tras completar Fase 0, ya que no tienen dependencias cruzadas directas. Sin embargo, se recomienda completar al menos Fases 0-5 antes de iniciar Fase 8 para tener una base operativa sólida.
---
## Mapa de Dependencias
```
Fase 0 (Fundamentos) ─────────────────────────────────────────────────
│ │
├──► Fase 1 (Jira + Tempo) ─── necesita Redis │
│ │
├──► Fase 2 (Reporting) ────── necesita excepciones de dominio │
│ │ │
│ └──► Fase 6 (Analytics/Webhooks) │
│ │ │
│ └──► Fase 13 (Alertas) ◄── usa webhooks │
│ ▲ como canal │
│ │ │
├──► Fase 3 (Compliance) ───── necesita audit mejorado │
│ │ │
│ └──► Fase 7 (Notificaciones) │
│ │
├──► Fase 4 (Intel Auto) ───── necesita índices BD │
│ │ │
│ └──► Fase 12 (Risk) ◄── usa OsintItem │
│ ▲ │
│ │ │
├──► Fase 5 (Operativa) ────── necesita scoring en BD │
│ │
├──► Fase 8 (Detection Lifecycle) ───────────────────────────── │
│ │ │
│ ├──► Fase 9 (Ownership) ── necesita decay + confidence │
│ │ │ │
│ │ ├──► Fase 10 (Attack Paths) │
│ │ │ necesita ownership + revalidation │
│ │ │ │
│ │ └──► Fase 12 (Risk & Recomendaciones) │
│ │ necesita confidence + ownership │
│ │ │
│ ├──► Fase 11 (Knowledge Management) ── independiente │
│ │ │
│ └──► Fase 13 (Alertas Inteligentes) │
│ necesita todas las fuentes anteriores │
│ │
└──► Fase 14 (Enterprise/SSO) ── independiente │
```
> **Nota:** Fase 4 (Stale Detection) implementa una versión simple de detección de obsolescencia. Fase 8 (Decay Engine) la reemplaza con un motor completo y configurable. Fase 4 sirve como stepping stone y puede coexistir — el decay engine de Fase 8 es la versión definitiva.
---
## FASE 0 — Fundamentos Técnicos
**Por qué primero:** Las nuevas features (Jira, Tempo, reports, DLM, etc.) necesitan Redis, excepciones de dominio limpias, e índices en BD para no colapsar el sistema. Esta fase resuelve la deuda técnica que bloquea todo lo demás.
**Duración estimada:** 2 semanas
**Dependencias:** Ninguna
---
### Tarea 0.1: Redis como servicio de infraestructura
**Qué:** Añadir un contenedor Redis al stack Docker Compose para token blacklist, cache de scores y futuras colas.
**Implementación:**
1. **docker-compose.yml y docker-compose.prod.yml** — Agregar servicio Redis:
```yaml
redis:
image: redis:7-alpine
command: redis-server --appendonly yes --maxmemory 256mb --maxmemory-policy allkeys-lru
ports:
- "6379:6379" # solo en dev
volumes:
- aegis_redis_data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 3s
retries: 5
restart: always
```
2. **requirements.txt** — Añadir `redis>=5.0.0`
3. **app/config.py** — Nuevas variables:
```python
REDIS_URL: str = "redis://redis:6379/0"
REDIS_TOKEN_BLACKLIST_DB: int = 1
REDIS_CACHE_DB: int = 2
```
4. **app/infrastructure/redis_client.py** — Cliente singleton:
```python
import redis
from app.config import settings
_redis_client: redis.Redis | None = None
def get_redis() -> redis.Redis:
global _redis_client
if _redis_client is None:
_redis_client = redis.from_url(
settings.REDIS_URL,
decode_responses=True
)
return _redis_client
```
5. **Backend Depends en docker-compose** — Añadir `redis: condition: service_healthy`
**Verificación:**
- `docker compose up redis` arranca y responde a `redis-cli ping` con `PONG`
- Test unitario: conectar, set/get una clave, verificar TTL funciona
- Health check Docker pasa en `docker compose ps`
---
### Tarea 0.2: Mover Token Blacklist a Redis (SEC-001)
**Qué:** Reemplazar el `set()` en memoria de auth.py por Redis con TTL automático.
**Implementación:**
1. **app/auth.py** — Reemplazar:
```python
# ANTES (eliminar):
_blacklisted_tokens: set[str] = set()
# DESPUÉS:
from app.infrastructure.redis_client import get_redis
def blacklist_token(token: str, expires_in_seconds: int):
"""Añade token al blacklist con TTL igual a su tiempo restante de expiración."""
r = get_redis()
r.setex(f"blacklist:{token}", expires_in_seconds, "1")
def is_token_blacklisted(token: str) -> bool:
r = get_redis()
return r.exists(f"blacklist:{token}") > 0
```
2. **Endpoint de logout** — Calcular TTL restante del JWT antes de blacklistear:
```python
from jose import jwt
import time
payload = jwt.decode(token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM])
exp = payload.get("exp", 0)
ttl = max(int(exp - time.time()), 0)
blacklist_token(token, ttl)
```
3. **Middleware de verificación** — En `get_current_user`, añadir check:
```python
if is_token_blacklisted(token):
raise HTTPException(status_code=401, detail="Token has been revoked")
```
**Verificación:**
- Login → obtener token → Logout → usar mismo token → debe dar 401
- Reiniciar backend container → token sigue blacklisteado en Redis
- Verificar en Redis CLI: `keys blacklist:*` muestra tokens activos
- TTL correcto: `ttl blacklist:<token>` debe ser menor al token expiration
---
### Tarea 0.3: Índices de Base de Datos (SR-006)
**Qué:** Crear migración Alembic con los índices compuestos faltantes.
**Implementación:**
1. Crear nueva migración Alembic:
```bash
alembic revision --autogenerate -m "add_composite_indexes"
```
2. En el archivo de migración generado, agregar los índices manualmente:
```python
def upgrade():
# Tests - usado por scoring, heatmap, metrics, reports
op.create_index('ix_tests_technique_id_state', 'tests', ['technique_id', 'state'])
op.create_index('ix_tests_state_red_validated_at', 'tests', ['state', 'red_validated_at'])
op.create_index('ix_tests_remediation_status', 'tests', ['remediation_status'])
op.create_index('ix_tests_campaign_id', 'tests', ['campaign_id'])
# Techniques
op.create_index('ix_techniques_tactic', 'techniques', ['tactic'])
op.create_index('ix_techniques_status_global', 'techniques', ['status_global'])
# Audit logs - para MTTD/MTTR
op.create_index('ix_audit_logs_entity_type_entity_id', 'audit_logs', ['entity_type', 'entity_id', 'action'])
op.create_index('ix_audit_logs_timestamp', 'audit_logs', ['timestamp'])
op.create_index('ix_audit_logs_user_id', 'audit_logs', ['user_id'])
# Detection rules
op.create_index('ix_detection_rules_mitre_technique_id', 'detection_rules', ['mitre_technique_id'])
# Notifications
op.create_index('ix_notifications_user_id_is_read', 'notifications', ['user_id', 'is_read'])
def downgrade():
op.drop_index('ix_tests_technique_id_state')
op.drop_index('ix_tests_state_red_validated_at')
# ... drop todos
```
**Verificación:**
- `alembic upgrade head` ejecuta sin error
- `\di` en psql muestra todos los nuevos índices
- Query `EXPLAIN ANALYZE SELECT * FROM tests WHERE technique_id = '...' AND state = 'validated'` usa el índice (Index Scan, no Seq Scan)
---
### Tarea 0.4: Excepciones de Dominio + Error Handler Middleware (TD-003)
**Qué:** Crear excepciones de dominio y un middleware que las mapee a HTTP, para eliminar HTTPException de los servicios.
**Implementación:**
1. **app/domain/exceptions.py** (nuevo archivo):
```python
class DomainException(Exception):
"""Base para todas las excepciones de dominio."""
def __init__(self, message: str, code: str = "DOMAIN_ERROR"):
self.message = message
self.code = code
super().__init__(message)
class EntityNotFoundError(DomainException):
def __init__(self, entity: str, identifier: str):
super().__init__(f"{entity} not found: {identifier}", "NOT_FOUND")
self.entity = entity
self.identifier = identifier
class DuplicateEntityError(DomainException):
def __init__(self, entity: str, field: str, value: str):
super().__init__(f"{entity} with {field}='{value}' already exists", "DUPLICATE")
class InvalidTransitionError(DomainException):
def __init__(self, current_state: str, target_state: str):
super().__init__(
f"Cannot transition from '{current_state}' to '{target_state}'",
"INVALID_TRANSITION"
)
class InvalidOperationError(DomainException):
def __init__(self, message: str):
super().__init__(message, "INVALID_OPERATION")
class AuthorizationError(DomainException):
def __init__(self, message: str = "Insufficient permissions"):
super().__init__(message, "FORBIDDEN")
```
2. **app/middleware/error_handler.py** (nuevo):
```python
from fastapi import Request
from fastapi.responses import JSONResponse
from app.domain.exceptions import (
EntityNotFoundError, DuplicateEntityError,
InvalidTransitionError, InvalidOperationError,
AuthorizationError, DomainException
)
EXCEPTION_STATUS_MAP = {
EntityNotFoundError: 404,
DuplicateEntityError: 409,
InvalidTransitionError: 400,
InvalidOperationError: 400,
AuthorizationError: 403,
}
async def domain_exception_handler(request: Request, exc: DomainException):
status = EXCEPTION_STATUS_MAP.get(type(exc), 400)
return JSONResponse(
status_code=status,
content={"detail": exc.message, "code": exc.code}
)
```
3. **app/main.py** — Registrar handler:
```python
from app.domain.exceptions import DomainException
from app.middleware.error_handler import domain_exception_handler
app.add_exception_handler(DomainException, domain_exception_handler)
```
4. **test_workflow_service.py** — Reemplazar HTTPException por excepciones de dominio:
```python
# ANTES:
from fastapi import HTTPException
raise HTTPException(status_code=400, detail={...})
# DESPUÉS:
from app.domain.exceptions import InvalidTransitionError
raise InvalidTransitionError(current_state=test.state, target_state="red_executing")
```
5. Repetir para `campaign_service.py` (el otro servicio que importa HTTPException).
**Verificación:**
- `grep -r "from fastapi import HTTPException" app/services/` devuelve 0 resultados
- Tests existentes de workflow siguen pasando (errores 400 se mantienen)
- Nuevo test: llamar endpoint con transición inválida → respuesta JSON con `{code: "INVALID_TRANSITION"}`
---
### Tarea 0.5: Arreglar Excepciones Silenciadas (TD-007)
**Qué:** Reemplazar `except Exception: pass` por logging en workflow service.
**Implementación:**
En `test_workflow_service.py`, localizar los 4 bloques `except Exception: pass` y reemplazar:
```python
# ANTES:
try:
notify_test_state_change(...)
except Exception:
pass
# DESPUÉS:
import logging
logger = logging.getLogger(__name__)
try:
notify_test_state_change(...)
except Exception as e:
logger.warning(f"Notification failed for test {test.id}: {e}", exc_info=True)
```
**Verificación:**
- `grep -r "except Exception: pass" app/` devuelve 0 resultados
- Simular fallo de notificación → ver warning en logs del backend
- Workflow sigue funcionando aunque notificación falle
---
### Tarea 0.6: CI/CD Básico con GitHub Actions (TD-009)
**Qué:** Pipeline mínimo: lint + tests.
**Implementación:**
**.github/workflows/ci.yml:**
```yaml
name: Aegis CI
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
lint-and-test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15-alpine
env:
POSTGRES_DB: testdb
POSTGRES_USER: test
POSTGRES_PASSWORD: test
ports: ["5432:5432"]
options: --health-cmd pg_isready --health-interval 10s
redis:
image: redis:7-alpine
ports: ["6379:6379"]
options: --health-cmd "redis-cli ping" --health-interval 10s
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install deps
run: |
cd backend
pip install -r requirements.txt
pip install ruff pytest
- name: Lint
run: cd backend && ruff check app/
- name: Test
env:
DATABASE_URL: postgresql://test:test@localhost:5432/testdb
REDIS_URL: redis://localhost:6379/0
SECRET_KEY: test-secret-key-for-ci
run: cd backend && pytest tests/ -v --tb=short
```
**Verificación:**
- Push a branch → GitHub Actions ejecuta sin error
- Badge verde en README
- Lint falla si introduces código mal formateado
---
## FASE 1 — Integración Jira + Tempo
**Duración estimada:** 3 semanas
**Dependencias:** Fase 0 (Redis)
---
### Tarea 1.1: Modelo de Datos — Tabla de Asociaciones Jira
**Qué:** Crear modelo y migración para vincular entidades Aegis con tickets Jira.
**Implementación:**
1. **app/models/jira_link.py** (nuevo):
```python
import uuid
from datetime import datetime
from sqlalchemy import Column, String, DateTime, ForeignKey, Enum as SQLEnum, Text
from sqlalchemy.dialects.postgresql import UUID, JSONB
from sqlalchemy.orm import relationship
from app.database import Base
import enum
class JiraLinkEntityType(str, enum.Enum):
test = "test"
technique = "technique"
campaign = "campaign"
evidence = "evidence"
class JiraSyncDirection(str, enum.Enum):
aegis_to_jira = "aegis_to_jira"
jira_to_aegis = "jira_to_aegis"
bidirectional = "bidirectional"
class JiraLink(Base):
__tablename__ = "jira_links"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
entity_type = Column(SQLEnum(JiraLinkEntityType), nullable=False)
entity_id = Column(UUID(as_uuid=True), nullable=False, index=True)
jira_issue_key = Column(String(50), nullable=False, index=True) # e.g. "SEC-1234"
jira_issue_id = Column(String(50)) # Jira internal numeric ID
jira_project_key = Column(String(20))
jira_status = Column(String(100))
jira_priority = Column(String(50))
jira_assignee = Column(String(255))
jira_story_points = Column(String(10))
sync_direction = Column(SQLEnum(JiraSyncDirection), default=JiraSyncDirection.bidirectional)
last_synced_at = Column(DateTime)
sync_metadata = Column(JSONB, default={})
created_by = Column(UUID(as_uuid=True), ForeignKey("users.id"))
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
```
2. **Alembic migration:**
```bash
alembic revision --autogenerate -m "add_jira_links_table"
alembic upgrade head
```
3. **app/schemas/jira_schema.py** (nuevo):
```python
from pydantic import BaseModel, Field
from typing import Optional
from uuid import UUID
from datetime import datetime
from app.models.jira_link import JiraLinkEntityType, JiraSyncDirection
class JiraLinkCreate(BaseModel):
entity_type: JiraLinkEntityType
entity_id: UUID
jira_issue_key: str = Field(..., pattern=r'^[A-Z]+-\d+$')
sync_direction: JiraSyncDirection = JiraSyncDirection.bidirectional
class JiraLinkOut(BaseModel):
id: UUID
entity_type: JiraLinkEntityType
entity_id: UUID
jira_issue_key: str
jira_status: Optional[str]
jira_priority: Optional[str]
jira_assignee: Optional[str]
last_synced_at: Optional[datetime]
created_at: datetime
class Config:
from_attributes = True
class JiraIssueSearch(BaseModel):
query: str
class JiraIssueResult(BaseModel):
issue_key: str
summary: str
status: str
assignee: Optional[str]
priority: Optional[str]
```
**Verificación:**
- Migración ejecuta sin error
- Tabla `jira_links` existe con todas las columnas e índices
- Schema valida que `jira_issue_key` tenga formato `PROJ-123`
---
### Tarea 1.2: Servicio de Integración Jira
**Qué:** Servicio que encapsula toda la comunicación con Jira REST API usando `atlassian-python-api`.
**Implementación:**
1. **requirements.txt** — Añadir: `atlassian-python-api>=4.0.0`
2. **app/config.py** — Nuevas settings:
```python
# Jira Integration
JIRA_ENABLED: bool = False
JIRA_URL: str = ""
JIRA_USERNAME: str = ""
JIRA_API_TOKEN: str = ""
JIRA_IS_CLOUD: bool = True
JIRA_DEFAULT_PROJECT: str = ""
JIRA_ISSUE_TYPE_TEST: str = "Task"
JIRA_ISSUE_TYPE_CAMPAIGN: str = "Epic"
```
3. **app/services/jira_service.py** (nuevo):
```python
import logging
from typing import Optional
from atlassian import Jira
from app.config import settings
from sqlalchemy.orm import Session
from app.models.jira_link import JiraLink, JiraLinkEntityType
from datetime import datetime
logger = logging.getLogger(__name__)
_jira_client: Optional[Jira] = None
def get_jira_client() -> Jira:
global _jira_client
if not settings.JIRA_ENABLED:
raise InvalidOperationError("Jira integration is not enabled")
if _jira_client is None:
_jira_client = Jira(
url=settings.JIRA_URL,
username=settings.JIRA_USERNAME,
password=settings.JIRA_API_TOKEN,
cloud=settings.JIRA_IS_CLOUD,
)
return _jira_client
def search_jira_issues(query: str, max_results: int = 10) -> list[dict]:
"""Busca issues en Jira por JQL o texto."""
jira = get_jira_client()
jql = query if " " not in query or "=" in query else f'summary ~ "{query}"'
results = jira.jql(jql, limit=max_results)
return [
{
"issue_key": issue["key"],
"summary": issue["fields"]["summary"],
"status": issue["fields"]["status"]["name"],
"assignee": (issue["fields"].get("assignee") or {}).get("displayName"),
"priority": (issue["fields"].get("priority") or {}).get("name"),
}
for issue in results.get("issues", [])
]
def create_jira_issue(
project_key: str,
summary: str,
description: str,
issue_type: str = "Task",
labels: list[str] = None,
custom_fields: dict = None,
) -> dict:
"""Crea un issue en Jira y retorna key + id."""
jira = get_jira_client()
fields = {
"project": {"key": project_key},
"summary": summary,
"description": description,
"issuetype": {"name": issue_type},
}
if labels:
fields["labels"] = labels
if custom_fields:
fields.update(custom_fields)
result = jira.issue_create(fields=fields)
return {"issue_key": result["key"], "issue_id": result["id"]}
def sync_aegis_to_jira(db: Session, link: JiraLink, entity_data: dict):
"""Sincroniza datos de Aegis hacia Jira (añade comentario con resultados)."""
jira = get_jira_client()
comment_body = _build_sync_comment(entity_data)
jira.issue_add_comment(link.jira_issue_key, comment_body)
link.last_synced_at = datetime.utcnow()
db.commit()
def sync_jira_to_aegis(db: Session, link: JiraLink):
"""Sincroniza estado de Jira hacia Aegis."""
jira = get_jira_client()
issue = jira.issue(link.jira_issue_key)
link.jira_status = issue["fields"]["status"]["name"]
link.jira_priority = (issue["fields"].get("priority") or {}).get("name")
link.jira_assignee = (issue["fields"].get("assignee") or {}).get("displayName")
link.jira_story_points = issue["fields"].get("customfield_10016")
link.last_synced_at = datetime.utcnow()
db.commit()
def _build_sync_comment(data: dict) -> str:
"""Genera un comentario formateado para Jira."""
lines = ["h3. Aegis Sync Update", ""]
for key, value in data.items():
lines.append(f"*{key}:* {value}")
lines.append(f"\n_Synced at {datetime.utcnow().isoformat()}_")
return "\n".join(lines)
```
**Verificación:**
- Con `JIRA_ENABLED=False`, llamar a `get_jira_client()` lanza `InvalidOperationError`
- Con credenciales válidas de test: `search_jira_issues("project = TEST")` retorna issues
- `create_jira_issue(...)` crea ticket y retorna key válida
- Mock test: verificar que `sync_aegis_to_jira` llama a `issue_add_comment`
---
### Tarea 1.3: Router de Jira Links
**Qué:** Endpoints para crear/listar/sincronizar asociaciones Jira.
**Implementación:**
**app/routers/jira.py** (nuevo):
```python
from fastapi import APIRouter, Depends, Query
from sqlalchemy.orm import Session
from uuid import UUID
from typing import Optional
from app.database import get_db
from app.dependencies.auth import get_current_user, require_role
from app.models.jira_link import JiraLink, JiraLinkEntityType
from app.schemas.jira_schema import (
JiraLinkCreate, JiraLinkOut, JiraIssueSearch, JiraIssueResult
)
from app.services import jira_service, audit_service
router = APIRouter(prefix="/jira", tags=["jira"])
@router.get("/search", response_model=list[JiraIssueResult])
def search_issues(
q: str = Query(..., min_length=2),
max_results: int = Query(10, le=50),
user=Depends(get_current_user),
):
"""Buscar issues en Jira por JQL o texto."""
return jira_service.search_jira_issues(q, max_results)
@router.post("/links", response_model=JiraLinkOut, status_code=201)
def create_link(
body: JiraLinkCreate,
db: Session = Depends(get_db),
user=Depends(get_current_user),
):
"""Asociar una entidad Aegis con un ticket Jira."""
link = JiraLink(
entity_type=body.entity_type,
entity_id=body.entity_id,
jira_issue_key=body.jira_issue_key,
sync_direction=body.sync_direction,
created_by=user.id,
)
jira_service.sync_jira_to_aegis(db, link)
db.add(link)
db.commit()
db.refresh(link)
audit_service.log_action(db, user.id, "JIRA_LINK_CREATED", "jira_link", str(link.id),
details={"entity_type": body.entity_type, "issue_key": body.jira_issue_key})
return link
@router.get("/links", response_model=list[JiraLinkOut])
def list_links(
entity_type: Optional[JiraLinkEntityType] = None,
entity_id: Optional[UUID] = None,
db: Session = Depends(get_db),
user=Depends(get_current_user),
):
"""Listar asociaciones Jira, filtrando opcionalmente por entidad."""
query = db.query(JiraLink)
if entity_type:
query = query.filter(JiraLink.entity_type == entity_type)
if entity_id:
query = query.filter(JiraLink.entity_id == entity_id)
return query.order_by(JiraLink.created_at.desc()).all()
@router.post("/links/{link_id}/sync")
def sync_link(
link_id: UUID,
db: Session = Depends(get_db),
user=Depends(require_role("admin")),
):
"""Forzar sincronización bidireccional de un link."""
link = db.query(JiraLink).filter(JiraLink.id == link_id).first()
if not link:
raise EntityNotFoundError("JiraLink", str(link_id))
jira_service.sync_jira_to_aegis(db, link)
return {"message": "Sync completed", "jira_status": link.jira_status}
@router.post("/create-issue")
def create_issue_from_entity(
entity_type: JiraLinkEntityType,
entity_id: UUID,
db: Session = Depends(get_db),
user=Depends(get_current_user),
):
"""Auto-crear un ticket Jira desde una entidad Aegis."""
summary, description = _build_issue_data(db, entity_type, entity_id)
result = jira_service.create_jira_issue(
project_key=settings.JIRA_DEFAULT_PROJECT,
summary=summary,
description=description,
labels=["aegis", entity_type.value],
)
link = JiraLink(
entity_type=entity_type,
entity_id=entity_id,
jira_issue_key=result["issue_key"],
jira_issue_id=result["issue_id"],
jira_project_key=settings.JIRA_DEFAULT_PROJECT,
created_by=user.id,
)
db.add(link)
db.commit()
return {"issue_key": result["issue_key"], "link_id": str(link.id)}
```
Registrar en main.py: `app.include_router(jira_router, prefix="/api/v1")`
**Verificación:**
- `POST /api/v1/jira/links` con issue_key inválido (sin formato) → 422
- `POST /api/v1/jira/links` con issue_key válido → 201 + datos de Jira populados
- `GET /api/v1/jira/links?entity_type=test&entity_id=...` → lista filtrada
- Audit log registra la acción `JIRA_LINK_CREATED`
---
### Tarea 1.4: Servicio de Integración Tempo
**Qué:** Servicio para registrar worklogs automáticos en Tempo desde tests completados.
**Implementación:**
1. **requirements.txt** — Añadir: `tempo-api-python-client>=0.8.0`
2. **app/config.py** — Nuevas settings:
```python
TEMPO_ENABLED: bool = False
TEMPO_API_TOKEN: str = ""
TEMPO_API_VERSION: int = 4
TEMPO_DEFAULT_WORK_TYPE: str = "Red Team"
```
3. **app/services/tempo_service.py** (nuevo):
```python
import logging
from typing import Optional
from tempoapiclient import client_v4 as tempo_client
from app.config import settings
from sqlalchemy.orm import Session
from app.models.jira_link import JiraLink
logger = logging.getLogger(__name__)
def get_tempo_client():
if not settings.TEMPO_ENABLED:
raise InvalidOperationError("Tempo integration is not enabled")
return tempo_client.Tempo(auth_token=settings.TEMPO_API_TOKEN)
def log_worklog(
jira_issue_id: int,
author_account_id: str,
date: str,
time_spent_seconds: int,
description: str,
work_type: str = None,
) -> dict:
"""Registra un worklog en Tempo."""
tempo = get_tempo_client()
worklog = tempo.create_worklog(
accountId=author_account_id,
issueId=jira_issue_id,
dateFrom=date,
timeSpentSeconds=time_spent_seconds,
description=description,
)
return worklog
def auto_log_test_worklog(db: Session, test, user, activity_type: str):
"""Si el test tiene un Jira link, registrar tiempo automáticamente en Tempo."""
if not settings.TEMPO_ENABLED:
return
link = db.query(JiraLink).filter(
JiraLink.entity_id == test.id,
JiraLink.entity_type == "test"
).first()
if not link or not link.jira_issue_id:
logger.debug(f"No Jira link for test {test.id}, skipping Tempo worklog")
return
duration = _calculate_duration(test, activity_type)
if duration <= 0:
return
try:
log_worklog(
jira_issue_id=int(link.jira_issue_id),
author_account_id=user.jira_account_id or "",
date=test.updated_at.strftime("%Y-%m-%d"),
time_spent_seconds=duration,
description=f"[Aegis] {activity_type}: {test.name}",
)
logger.info(f"Tempo worklog created for test {test.id}, {duration}s")
except Exception as e:
logger.warning(f"Tempo worklog failed for test {test.id}: {e}")
```
**Verificación:**
- Con `TEMPO_ENABLED=False`, la función retorna silenciosamente
- Mock test: verificar que `create_worklog` se llama con parámetros correctos
- Integración: crear worklog en Tempo sandbox y verificar que aparece
---
### Tarea 1.5: Worklog Interno Auditado
**Qué:** Tabla de registro de tiempo interno e inmutable, previo a envío a Tempo.
**Implementación:**
1. **app/models/worklog.py** (nuevo):
```python
import uuid
from datetime import datetime
from sqlalchemy import Column, String, Integer, DateTime, ForeignKey, Text
from sqlalchemy.dialects.postgresql import UUID, JSONB
from app.database import Base
class Worklog(Base):
__tablename__ = "worklogs"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
entity_type = Column(String(50), nullable=False)
entity_id = Column(UUID(as_uuid=True), nullable=False, index=True)
user_id = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
activity_type = Column(String(100), nullable=False)
started_at = Column(DateTime, nullable=False)
ended_at = Column(DateTime)
duration_seconds = Column(Integer, nullable=False)
description = Column(Text)
tempo_synced = Column(DateTime)
tempo_worklog_id = Column(String(100))
integrity_hash = Column(String(64))
created_at = Column(DateTime, default=datetime.utcnow)
metadata = Column(JSONB, default={})
```
2. **app/services/worklog_service.py** — funciones CRUD + cálculo de hash de integridad:
```python
import hashlib
from datetime import datetime
from sqlalchemy.orm import Session
from app.models.worklog import Worklog
def create_worklog(db: Session, **kwargs) -> Worklog:
wl = Worklog(**kwargs)
wl.integrity_hash = _compute_hash(wl)
db.add(wl)
db.commit()
db.refresh(wl)
return wl
def _compute_hash(wl: Worklog) -> str:
data = f"{wl.entity_type}:{wl.entity_id}:{wl.user_id}:{wl.activity_type}:{wl.started_at}:{wl.duration_seconds}"
return hashlib.sha256(data.encode()).hexdigest()
def verify_worklog_integrity(wl: Worklog) -> bool:
return wl.integrity_hash == _compute_hash(wl)
```
3. **app/routers/worklogs.py** — Endpoints para consultar y crear worklogs manuales.
**Verificación:**
- Crear worklog → `integrity_hash` se genera automáticamente
- Modificar un campo en DB directo → `verify_worklog_integrity()` retorna `False`
- Listar worklogs por entidad funciona con filtros
---
### Tarea 1.6: Frontend — Componente JiraLink + Tempo
**Qué:** Componente React para vincular entidades con Jira y ver worklogs.
**Implementación:**
- `src/api/jira.ts` — funciones API (search, createLink, listLinks, syncLink)
- `src/components/JiraLinkPanel.tsx` — Panel con buscar, vincular y ver estado del ticket
- `src/components/WorklogTimeline.tsx` — Timeline de worklogs con duración y tipo
- Integrar JiraLinkPanel en páginas de Test Detail, Campaign Detail y Technique Detail
- `src/types/models.ts` — Añadir tipos JiraLink, Worklog, etc.
**Verificación:**
- En Test Detail, el panel de Jira aparece y permite buscar issues
- Vincular un issue muestra su estado actualizado
- Timeline de worklogs muestra registro cronológico
---
### Tarea 1.7: Job de Sincronización Jira Automática
**Qué:** Job APScheduler que sincroniza el estado de Jira links cada hora.
**Implementación:**
En `jobs/jira_sync_job.py`:
```python
from app.database import SessionLocal
from app.models.jira_link import JiraLink
from app.services import jira_service
from app.config import settings
import logging
logger = logging.getLogger(__name__)
def sync_all_jira_links():
if not settings.JIRA_ENABLED:
return
db = SessionLocal()
try:
links = db.query(JiraLink).all()
synced = 0
for link in links:
try:
jira_service.sync_jira_to_aegis(db, link)
synced += 1
except Exception as e:
logger.warning(f"Jira sync failed for link {link.id}: {e}")
logger.info(f"Jira sync completed: {synced}/{len(links)} links updated")
finally:
db.close()
```
Registrar en scheduler: `scheduler.add_job(sync_all_jira_links, "interval", hours=1, replace_existing=True)`
**Verificación:**
- Job aparece en logs al arrancar el backend
- Tras 1 hora (o trigger manual), los links muestran `last_synced_at` actualizado
- Links con issues borrados en Jira se logean como warning sin crashear
---
## FASE 2 — Motor de Reporting Profesional
**Duración estimada:** 3-4 semanas
**Dependencias:** Fase 0 (excepciones de dominio)
---
### Tarea 2.1: Motor de Plantillas — Backend
**Qué:** Sistema de plantillas Jinja2 que genera HTML, que luego se convierte a PDF con WeasyPrint y a DOCX con docxtpl.
**Implementación:**
1. **requirements.txt** — Añadir:
```
weasyprint>=62.0
docxtpl>=0.18.0
Jinja2>=3.1.0 # (ya instalado por FastAPI)
```
2. **app/config.py:**
```python
REPORT_TEMPLATES_DIR: str = "app/templates/reports"
REPORT_OUTPUT_DIR: str = "/tmp/aegis_reports"
COMPANY_NAME: str = "Organization"
COMPANY_LOGO_PATH: str = "app/templates/reports/assets/logo.png"
```
3. **app/services/report_engine.py** (nuevo):
```python
import os
import uuid
from datetime import datetime
from jinja2 import Environment, FileSystemLoader
from weasyprint import HTML, CSS
from docxtpl import DocxTemplate
from app.config import settings
class ReportEngine:
def __init__(self):
self.jinja_env = Environment(
loader=FileSystemLoader(settings.REPORT_TEMPLATES_DIR),
autoescape=True
)
os.makedirs(settings.REPORT_OUTPUT_DIR, exist_ok=True)
def render_html(self, template_name: str, context: dict) -> str:
template = self.jinja_env.get_template(f"{template_name}.html")
context["company_name"] = settings.COMPANY_NAME
context["generated_at"] = datetime.utcnow().isoformat()
return template.render(context)
def generate_pdf(self, template_name: str, context: dict) -> str:
html_content = self.render_html(template_name, context)
css_path = os.path.join(settings.REPORT_TEMPLATES_DIR, "styles", "report.css")
output_path = os.path.join(
settings.REPORT_OUTPUT_DIR,
f"{template_name}_{uuid.uuid4().hex[:8]}.pdf"
)
css = CSS(filename=css_path) if os.path.exists(css_path) else None
stylesheets = [css] if css else []
HTML(string=html_content, base_url=settings.REPORT_TEMPLATES_DIR).write_pdf(
output_path, stylesheets=stylesheets
)
return output_path
def generate_docx(self, template_name: str, context: dict) -> str:
template_path = os.path.join(settings.REPORT_TEMPLATES_DIR, f"{template_name}.docx")
output_path = os.path.join(
settings.REPORT_OUTPUT_DIR,
f"{template_name}_{uuid.uuid4().hex[:8]}.docx"
)
doc = DocxTemplate(template_path)
context["company_name"] = settings.COMPANY_NAME
context["generated_at"] = datetime.utcnow().strftime("%B %d, %Y")
doc.render(context)
doc.save(output_path)
return output_path
def generate_html(self, template_name: str, context: dict) -> str:
html_content = self.render_html(template_name, context)
output_path = os.path.join(
settings.REPORT_OUTPUT_DIR,
f"{template_name}_{uuid.uuid4().hex[:8]}.html"
)
with open(output_path, "w") as f:
f.write(html_content)
return output_path
report_engine = ReportEngine()
```
4. **Estructura de plantillas:**
```
app/templates/reports/
├── styles/
│ └── report.css
├── assets/
│ └── logo.png
├── purple_campaign.html
├── purple_campaign.docx
├── coverage_report.html
├── technique_detail.html
├── quarterly_summary.html
└── executive_summary.html
```
**Verificación:**
- `report_engine.generate_pdf("coverage_report", sample_context)` genera PDF legible
- `report_engine.generate_docx("purple_campaign", sample_context)` genera DOCX válido
- CSS se aplica correctamente (colores corporativos, headers, page breaks)
---
### Tarea 2.2: Plantilla de Informe Purple Team
**Qué:** La plantilla HTML Jinja2 para informes de campaña Purple Team.
**Implementación:**
**app/templates/reports/purple_campaign.html:**
```html
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<link rel="stylesheet" href="styles/report.css">
</head>
<body>
<section class="cover-page">
<img src="assets/logo.png" class="logo">
<h1>Purple Team Assessment Report</h1>
<h2>{{ campaign.name }}</h2>
<p class="date">{{ generated_at }}</p>
<p class="classification">{{ classification | default('INTERNAL') }}</p>
</section>
<section class="toc">
<h2>Table of Contents</h2>
</section>
<section>
<h2>1. Executive Summary</h2>
<p>Campaign <strong>{{ campaign.name }}</strong> tested
{{ tests | length }} techniques across {{ tactics | length }} tactics.
Overall coverage score: <strong>{{ org_score }}%</strong>.</p>
<div class="stats-grid">
<div class="stat">
<span class="number">{{ tests_validated }}</span>
<span class="label">Validated</span>
</div>
<div class="stat">
<span class="number">{{ tests_detected }}</span>
<span class="label">Detected</span>
</div>
<div class="stat">
<span class="number">{{ tests_not_detected }}</span>
<span class="label">Not Detected</span>
</div>
</div>
</section>
<section>
<h2>2. Scope & Methodology</h2>
<p>{{ campaign.description }}</p>
<p>Period: {{ campaign.start_date }} – {{ campaign.end_date }}</p>
<p>Threat actors modeled: {% for actor in threat_actors %}{{ actor.name }}{% if not loop.last %}, {% endif %}{% endfor %}</p>
</section>
<section>
<h2>3. Techniques Tested</h2>
<table class="data-table">
<thead>
<tr>
<th>MITRE ID</th><th>Name</th><th>Tactic</th>
<th>Result</th><th>Detection</th>
</tr>
</thead>
<tbody>
{% for test in tests %}
<tr class="result-{{ test.detection_result }}">
<td>{{ test.technique_mitre_id }}</td>
<td>{{ test.name }}</td>
<td>{{ test.tactic }}</td>
<td>{{ test.state }}</td>
<td>{{ test.detection_result }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</section>
<section>
<h2>4. Critical Findings</h2>
{% for finding in critical_findings %}
<div class="finding {{ finding.severity }}">
<h3>{{ finding.technique_id }}: {{ finding.name }}</h3>
<p>{{ finding.description }}</p>
<p><strong>Recommendation:</strong> {{ finding.recommendation }}</p>
</div>
{% endfor %}
</section>
<section>
<h2>5. Coverage Evolution</h2>
{% if previous_campaign %}
<p>Compared to previous campaign ({{ previous_campaign.name }}):
Coverage changed from {{ previous_score }}% to {{ org_score }}%.</p>
{% endif %}
</section>
<footer>
<p>{{ company_name }} – Confidential – Page <span class="page-number"></span></p>
</footer>
</body>
</html>
```
**Verificación:**
- Generar PDF con datos de campaña real → todas las secciones se renderizan
- Tabla de técnicas muestra colores según resultado (rojo/verde/amarillo)
- Page breaks funcionan entre secciones
---
### Tarea 2.3: Servicio de Generación de Reportes
**Qué:** Servicio que recopila datos del dominio y llama al ReportEngine.
**Implementación:**
**app/services/report_generation_service.py** (nuevo):
```python
from sqlalchemy.orm import Session
from app.services.report_engine import report_engine
from app.services import scoring_service
from app.models import Campaign, CampaignTest, Test, Technique, ThreatActor
def generate_purple_campaign_report(
db: Session, campaign_id: str, output_format: str = "pdf"
) -> str:
"""Genera el informe completo de una campaña Purple Team."""
campaign = db.query(Campaign).filter(Campaign.id == campaign_id).first()
if not campaign:
raise EntityNotFoundError("Campaign", campaign_id)
campaign_tests = (
db.query(Test)
.join(CampaignTest)
.filter(CampaignTest.campaign_id == campaign_id)
.all()
)
tests_data = []
for test in campaign_tests:
technique = db.query(Technique).filter(Technique.id == test.technique_id).first()
tests_data.append({
"technique_mitre_id": technique.mitre_id if technique else "N/A",
"name": test.name,
"tactic": technique.tactic if technique else "N/A",
"state": test.state.value if test.state else "draft",
"detection_result": test.detection_result.value if test.detection_result else "pending",
})
validated = [t for t in campaign_tests if t.state and t.state.value == "validated"]
detected = [t for t in validated if t.detection_result and t.detection_result.value == "detected"]
not_detected = [t for t in validated if t.detection_result and t.detection_result.value == "not_detected"]
critical_findings = [
{
"technique_id": t.get("technique_mitre_id"),
"name": t.get("name"),
"severity": "critical",
"description": f"Technique not detected during campaign execution.",
"recommendation": "Implement detection rule or review existing SIEM rules.",
}
for t in tests_data if t["detection_result"] == "not_detected"
]
context = {
"campaign": campaign,
"tests": tests_data,
"tests_validated": len(validated),
"tests_detected": len(detected),
"tests_not_detected": len(not_detected),
"critical_findings": critical_findings,
"org_score": scoring_service.calculate_organization_score(db).get("overall", 0),
"tactics": list(set(t.get("tactic") for t in tests_data)),
"threat_actors": [], # TODO: poblar
}
if output_format == "pdf":
return report_engine.generate_pdf("purple_campaign", context)
elif output_format == "docx":
return report_engine.generate_docx("purple_campaign", context)
else:
return report_engine.generate_html("purple_campaign", context)
```
---
### Tarea 2.4: Router de Reportes Profesionales
**Implementación:**
**app/routers/professional_reports.py** (nuevo):
```python
from fastapi import APIRouter, Depends, Query
from fastapi.responses import FileResponse
from sqlalchemy.orm import Session
from uuid import UUID
from app.database import get_db
from app.dependencies.auth import get_current_user, require_role
from app.services import report_generation_service
router = APIRouter(prefix="/reports/generate", tags=["professional-reports"])
@router.get("/purple-campaign/{campaign_id}")
def generate_purple_report(
campaign_id: UUID,
format: str = Query("pdf", regex="^(pdf|docx|html)$"),
db: Session = Depends(get_db),
user=Depends(require_role("red_lead", "blue_lead", "admin")),
):
filepath = report_generation_service.generate_purple_campaign_report(
db, str(campaign_id), output_format=format
)
media_types = {
"pdf": "application/pdf",
"docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"html": "text/html"
}
return FileResponse(filepath, media_type=media_types[format],
filename=f"purple_report.{format}")
@router.get("/coverage-summary")
def generate_coverage_report(
format: str = Query("pdf", regex="^(pdf|docx|html)$"),
db: Session = Depends(get_db),
user=Depends(get_current_user),
):
filepath = report_generation_service.generate_coverage_report(db, output_format=format)
media_types = {
"pdf": "application/pdf",
"docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"html": "text/html"
}
return FileResponse(filepath, media_type=media_types[format],
filename=f"coverage_report.{format}")
# Similares para: quarterly_summary, technique_detail, executive_summary
```
**Verificación:**
- `GET /api/v1/reports/generate/purple-campaign/{id}?format=pdf` → descarga PDF
- `GET /api/v1/reports/generate/purple-campaign/{id}?format=docx` → descarga DOCX
- PDF tiene portada, tabla de contenidos, datos de tests, hallazgos
- DOCX se abre correctamente en Word/LibreOffice
---
### Tarea 2.5: Endpoints de Analytics para PowerBI
**Qué:** Endpoints JSON limpios, sin lógica de presentación, optimizados para consumo BI.
**Implementación:**
**app/routers/analytics.py** (nuevo):
```python
router = APIRouter(prefix="/analytics", tags=["analytics"])
@router.get("/coverage")
def analytics_coverage(db: Session = Depends(get_db), user=Depends(get_current_user)):
"""Cobertura por táctica, plataforma y estado — formato plano para BI."""
techniques = db.query(Technique).all()
return [
{
"mitre_id": t.mitre_id,
"name": t.name,
"tactic": t.tactic,
"status": t.status_global.value if t.status_global else "not_evaluated",
"test_count": len(t.tests) if t.tests else 0,
"review_required": t.review_required,
"last_review_date": t.last_review_date.isoformat() if t.last_review_date else None,
}
for t in techniques
]
@router.get("/tests")
def analytics_tests(
date_from: str = Query(None), date_to: str = Query(None),
db: Session = Depends(get_db), user=Depends(get_current_user)
):
"""Todos los tests con timestamps — formato plano para BI."""
query = db.query(Test)
if date_from:
query = query.filter(Test.created_at >= date_from)
if date_to:
query = query.filter(Test.created_at <= date_to)
tests = query.all()
return [
{
"id": str(t.id),
"technique_id": str(t.technique_id),
"name": t.name,
"state": t.state.value if t.state else None,
"detection_result": t.detection_result.value if t.detection_result else None,
"created_at": t.created_at.isoformat() if t.created_at else None,
"platform": t.platform,
"tool_used": t.tool_used,
}
for t in tests
]
@router.get("/trends")
def analytics_trends(db: Session = Depends(get_db), user=Depends(get_current_user)):
"""Snapshots históricos de cobertura para visualización de tendencias."""
snapshots = db.query(CoverageSnapshot).order_by(CoverageSnapshot.created_at).all()
return [
{
"date": s.created_at.isoformat(),
"total_techniques": s.total_techniques,
"validated_count": s.validated_count,
"coverage_percentage": s.coverage_percentage,
"org_score": s.org_score,
}
for s in snapshots
]
@router.get("/operators")
def analytics_operators(db: Session = Depends(get_db), user=Depends(require_role("admin"))):
"""Métricas por operador — para gestión de carga."""
from sqlalchemy import func
results = (
db.query(
User.username,
User.role,
func.count(Test.id).label("test_count"),
)
.outerjoin(Test, Test.created_by == User.id)
.group_by(User.id)
.all()
)
return [{"username": r[0], "role": r[1], "test_count": r[2]} for r in results]
```
**Verificación:**
- Cada endpoint retorna JSON plano (no anidado más de 1 nivel)
- Filtros de fecha funcionan correctamente
- Datos pueden importarse directamente en PowerBI desde URL
- Sin paginación (datasets completos para BI)
---
### Tarea 2.6: Métricas Avanzadas
**Qué:** Añadir endpoints de métricas avanzadas: % cobertura por táctica, técnicas nunca probadas, tiempo medio de validación, heatmap histórico.
**Implementación:**
Extender `app/routers/metrics.py` o crear `app/routers/advanced_metrics.py`:
```python
@router.get("/coverage-by-tactic")
def coverage_by_tactic(db: Session = Depends(get_db), user=Depends(get_current_user)):
"""% de cobertura desglosado por táctica MITRE."""
from sqlalchemy import func, case
results = (
db.query(
Technique.tactic,
func.count(Technique.id).label("total"),
func.sum(case((Technique.status_global == "validated", 1), else_=0)).label("validated"),
)
.group_by(Technique.tactic)
.all()
)
return [
{
"tactic": r[0],
"total": r[1],
"validated": r[2],
"coverage_pct": round((r[2] / r[1]) * 100, 1) if r[1] > 0 else 0,
}
for r in results
]
@router.get("/never-tested")
def never_tested_techniques(db: Session = Depends(get_db), user=Depends(get_current_user)):
"""Técnicas que nunca han sido probadas."""
techniques = (
db.query(Technique)
.filter(~Technique.id.in_(db.query(Test.technique_id).distinct()))
.order_by(Technique.mitre_id)
.all()
)
return [{"mitre_id": t.mitre_id, "name": t.name, "tactic": t.tactic} for t in techniques]
@router.get("/avg-validation-time")
def avg_validation_time(db: Session = Depends(get_db), user=Depends(get_current_user)):
"""Tiempo medio desde creación del test hasta validación."""
# Implementar con audit log queries optimizadas (batch)
pass
```
**Verificación:**
- `/coverage-by-tactic` retorna una fila por táctica con porcentaje
- `/never-tested` lista técnicas sin ningún test asociado
- Datos son consistentes con lo que muestra el heatmap existente
---
## FASE 3 — Compliance & Hardening de Seguridad
**Duración estimada:** 2-3 semanas
**Dependencias:** Fase 0 (audit mejorado)
---
### Tarea 3.1: Audit Trail Mejorado (IP, Hash de Integridad)
**Qué:** Ampliar el audit log con IP del request, hash de integridad y campos adicionales.
**Implementación:**
1. **Migración Alembic** — Añadir columnas a `audit_logs`:
```python
def upgrade():
op.add_column('audit_logs', Column('ip_address', String(45)))
op.add_column('audit_logs', Column('user_agent', String(500)))
op.add_column('audit_logs', Column('integrity_hash', String(64)))
op.add_column('audit_logs', Column('session_id', String(100)))
```
2. **Middleware de request context** — Capturar IP:
```python
# app/middleware/request_context.py
from starlette.middleware.base import BaseHTTPMiddleware
from contextvars import ContextVar
request_ip: ContextVar[str] = ContextVar("request_ip", default="")
request_user_agent: ContextVar[str] = ContextVar("request_user_agent", default="")
class RequestContextMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request, call_next):
ip = request.client.host if request.client else "unknown"
forwarded = request.headers.get("X-Forwarded-For")
if forwarded:
ip = forwarded.split(",")[0].strip()
request_ip.set(ip)
request_user_agent.set(request.headers.get("User-Agent", ""))
response = await call_next(request)
return response
```
3. **audit_service.py** — Incorporar IP y hash:
```python
import hashlib
from app.middleware.request_context import request_ip, request_user_agent
def log_action(db, user_id, action, entity_type, entity_id, details=None):
ip = request_ip.get("")
ua = request_user_agent.get("")
entry = AuditLog(
user_id=user_id,
action=action,
entity_type=entity_type,
entity_id=entity_id,
details=details or {},
ip_address=ip,
user_agent=ua,
)
data = f"{user_id}:{action}:{entity_type}:{entity_id}:{entry.timestamp.isoformat()}"
entry.integrity_hash = hashlib.sha256(data.encode()).hexdigest()
db.add(entry)
# NO commit aquí — deja que el caller lo haga
```
**Verificación:**
- Crear un test → audit log tiene IP y user agent del request
- `integrity_hash` no es null en nuevos registros
- Modificar un audit log en BD directo → recalcular hash no coincide
---
### Tarea 3.2: Login Attempt Auditing (SEC-009)
**Qué:** Registrar intentos de login exitosos y fallidos.
**Implementación:**
En el router `auth.py`, después de cada intento de login:
```python
@router.post("/login")
def login(body: LoginRequest, request: Request, db: Session = Depends(get_db)):
user = db.query(User).filter(User.username == body.username).first()
# SIEMPRE ejecutar bcrypt para evitar timing attack (SEC-005)
dummy_hash = "$2b$12$LJ3m4ys3Lg7E3cDpBH0pVe8y2V3hZ2n1KX3X5X5X5X5X5X5X5X"
target_hash = user.hashed_password if user else dummy_hash
password_valid = verify_password(body.password, target_hash)
ip = request.client.host if request.client else "unknown"
if not user or not password_valid:
audit_service.log_action(
db, user.id if user else None, "LOGIN_FAILED",
"auth", None,
details={"username": body.username, "ip": ip, "reason": "invalid_credentials"}
)
db.commit()
raise HTTPException(400, detail="Incorrect credentials")
audit_service.log_action(
db, user.id, "LOGIN_SUCCESS", "auth", str(user.id),
details={"ip": ip}
)
token = create_access_token({"sub": user.username})
# ... resto del login
```
**Verificación:**
- Login fallido → registro en `audit_logs` con `action="LOGIN_FAILED"`
- Login exitoso → registro con `action="LOGIN_SUCCESS"`
- Ambos incluyen IP
- Timing de respuesta similar para usuario existente vs no existente
---
### Tarea 3.3: Validación de Password y Username (SEC-004, SEC-007)
**Implementación:**
En `app/schemas/user_schema.py`:
```python
from pydantic import BaseModel, field_validator
import re
RESERVED_USERNAMES = {"admin", "system", "root", "aegis", "api", "null", "undefined"}
class UserCreate(BaseModel):
username: str
password: str
email: str
role: str
@field_validator("username")
@classmethod
def validate_username(cls, v):
if len(v) < 3 or len(v) > 50:
raise ValueError("Username must be 3-50 characters")
if not re.match(r'^[a-zA-Z0-9_-]+$', v):
raise ValueError("Username may only contain letters, numbers, hyphens, underscores")
if v.lower() in RESERVED_USERNAMES:
raise ValueError(f"Username '{v}' is reserved")
return v
@field_validator("password")
@classmethod
def validate_password(cls, v):
if len(v) < 10:
raise ValueError("Password must be at least 10 characters")
if not re.search(r'[A-Z]', v):
raise ValueError("Password must contain at least one uppercase letter")
if not re.search(r'[a-z]', v):
raise ValueError("Password must contain at least one lowercase letter")
if not re.search(r'[0-9]', v):
raise ValueError("Password must contain at least one digit")
return v
```
**Verificación:**
- `POST /users` con password "123" → 422 con mensaje claro
- Username `"../admin"` → 422
- Username `"system"` → 422 (reservado)
- Password `"ValidPass123"` → acepta
---
### Tarea 3.4: Rate Limiting Extendido (SEC-003)
**Implementación:**
En `main.py` y routers relevantes:
```python
from slowapi import Limiter
from slowapi.util import get_remote_address
limiter = Limiter(key_func=get_remote_address)
# En routers de sync/import:
@router.post("/system/sync-mitre")
@limiter.limit("2/hour")
def sync_mitre(request: Request, ...):
...
@router.post("/system/import-atomic-tests")
@limiter.limit("2/hour")
def import_atomic(request: Request, ...):
...
# En routers de escritura:
@router.post("/tests")
@limiter.limit("30/minute")
def create_test(request: Request, ...):
...
# En routers de upload:
@router.post("/tests/{id}/evidence")
@limiter.limit("10/minute")
def upload_evidence(request: Request, ...):
...
# En reports (costosos):
@router.get("/reports/generate/{type}")
@limiter.limit("5/minute")
def generate_report(request: Request, ...):
...
```
**Verificación:**
- Más de 2 syncs/hora → 429 Too Many Requests
- Más de 30 tests/min → 429
- Headers de response incluyen `X-RateLimit-*`
---
### Tarea 3.5: Clasificación de Datos y Retención
**Qué:** Añadir campo de clasificación a entidades y políticas de retención.
**Implementación:**
1. **Migración:**
```python
class DataClassification(str, enum.Enum):
public = "public"
internal = "internal"
sensitive = "sensitive"
restricted = "restricted"
# Añadir a tablas: tests, evidence, campaigns
op.add_column('tests', Column('data_classification', String(20), default='internal'))
op.add_column('evidence', Column('data_classification', String(20), default='internal'))
op.add_column('campaigns', Column('data_classification', String(20), default='internal'))
```
2. **Job de retención** — Limpieza automática según política:
```python
def apply_retention_policies():
"""Job diario que aplica políticas de retención."""
db = SessionLocal()
try:
cutoff = datetime.utcnow() - timedelta(days=730)
deleted = db.query(AuditLog).filter(AuditLog.timestamp < cutoff).delete()
logger.info(f"Retention: deleted {deleted} audit logs older than 2 years")
db.commit()
finally:
db.close()
```
**Verificación:**
- Tests nuevos tienen `data_classification = 'internal'` por defecto
- Admin puede cambiar clasificación de un test
- Job de retención elimina logs antiguos sin error
---
## FASE 4 — Inteligencia Automática
**Duración estimada:** 2-3 semanas
**Dependencias:** Fase 0 (índices BD)
> **Nota:** La detección de stale coverage de esta fase (Tarea 4.2) es una versión simplificada que será reemplazada por el Decay Engine completo de Fase 8. Sirve como stepping stone funcional.
---
### Tarea 4.1: Enriquecimiento OSINT por Técnica
**Qué:** Para cada técnica, buscar automáticamente blogs, PoCs y CVEs relacionados.
**Implementación:**
1. **Modelo — OsintItem:**
```python
class OsintItem(Base):
__tablename__ = "osint_items"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
technique_id = Column(UUID(as_uuid=True), ForeignKey("techniques.id"), index=True)
source_type = Column(String(50)) # "cve", "blog", "poc", "advisory"
source_url = Column(Text)
title = Column(String(500))
description = Column(Text)
severity = Column(String(20))
discovered_at = Column(DateTime, default=datetime.utcnow)
reviewed = Column(Boolean, default=False)
metadata = Column(JSONB, default={})
```
2. **app/services/osint_enrichment_service.py:**
```python
import requests
import logging
from sqlalchemy.orm import Session
from app.models.osint_item import OsintItem
from app.models.technique import Technique
logger = logging.getLogger(__name__)
NVD_API_BASE = "https://services.nvd.nist.gov/rest/json/cves/2.0"
def enrich_technique_with_cves(db: Session, technique: Technique):
"""Buscar CVEs relacionados via NVD API usando CAPEC→CWE→CVE mapping."""
try:
params = {
"keywordSearch": technique.name,
"resultsPerPage": 10,
}
resp = requests.get(NVD_API_BASE, params=params, timeout=30)
if resp.status_code != 200:
logger.warning(f"NVD API error for {technique.mitre_id}: {resp.status_code}")
return 0
data = resp.json()
count = 0
for vuln in data.get("vulnerabilities", []):
cve = vuln.get("cve", {})
cve_id = cve.get("id")
exists = db.query(OsintItem).filter(
OsintItem.technique_id == technique.id,
OsintItem.source_url.contains(cve_id)
).first()
if exists:
continue
descriptions = cve.get("descriptions", [])
desc = next((d["value"] for d in descriptions if d["lang"] == "en"), "")
metrics = cve.get("metrics", {})
cvss = metrics.get("cvssMetricV31", [{}])[0] if metrics.get("cvssMetricV31") else {}
severity = cvss.get("cvssData", {}).get("baseSeverity", "UNKNOWN") if cvss else "UNKNOWN"
item = OsintItem(
technique_id=technique.id,
source_type="cve",
source_url=f"https://nvd.nist.gov/vuln/detail/{cve_id}",
title=cve_id,
description=desc[:500],
severity=severity,
metadata={"cvss_score": cvss.get("cvssData", {}).get("baseScore") if cvss else None},
)
db.add(item)
count += 1
if count > 0:
technique.review_required = True
db.commit()
logger.info(f"Added {count} CVEs for {technique.mitre_id}")
return count
except Exception as e:
logger.error(f"OSINT enrichment failed for {technique.mitre_id}: {e}")
return 0
def enrich_all_techniques(db: Session):
"""Enriquecer todas las técnicas (rate-limited por NVD: 5 req/30s sin API key)."""
import time
techniques = db.query(Technique).all()
total = 0
for i, tech in enumerate(techniques):
total += enrich_technique_with_cves(db, tech)
if i % 5 == 4:
time.sleep(30)
return total
```
3. **Job semanal:**
```python
scheduler.add_job(
lambda: enrich_all_techniques(SessionLocal()),
"interval", days=7, id="osint_enrichment",
replace_existing=True
)
```
**Verificación:**
- Ejecutar `enrich_technique_with_cves(db, technique)` para T1059 → obtiene CVEs
- No se crean duplicados si se ejecuta dos veces
- Técnicas con nuevos CVEs se marcan `review_required = True`
- Rate limiting respeta NVD API limits (5 req/30s)
---
### Tarea 4.2: Detección de Stale Coverage (versión simple)
**Qué:** Marcar técnicas cuya última prueba fue hace más de N meses.
> **Nota:** Esta es la versión simple. Fase 8 (Decay Engine) la reemplaza con un motor completo y configurable con políticas, factores múltiples y confidence scores.
**Implementación:**
```python
# app/services/stale_detection_service.py
from datetime import datetime, timedelta
from sqlalchemy.orm import Session
from sqlalchemy import func
from app.models import Technique, Test
import logging
logger = logging.getLogger(__name__)
STALE_THRESHOLD_DAYS = 365
def detect_stale_coverage(db: Session) -> int:
"""Marcar técnicas con cobertura stale."""
cutoff = datetime.utcnow() - timedelta(days=STALE_THRESHOLD_DAYS)
latest_test = (
db.query(
Test.technique_id,
func.max(Test.updated_at).label("last_tested")
)
.filter(Test.state == "validated")
.group_by(Test.technique_id)
.subquery()
)
stale_techniques = (
db.query(Technique)
.outerjoin(latest_test, Technique.id == latest_test.c.technique_id)
.filter(
(latest_test.c.last_tested < cutoff) |
(latest_test.c.last_tested.is_(None))
)
.filter(Technique.status_global != "not_evaluated")
.all()
)
count = 0
for tech in stale_techniques:
if not tech.review_required:
tech.review_required = True
count += 1
logger.info(f"Marked {tech.mitre_id} as stale coverage")
if count > 0:
db.commit()
return count
```
Registrar como job diario.
**Verificación:**
- Técnica con último test hace 13 meses → se marca `review_required = True`
- Técnica con test reciente → no se marca
- Técnica nunca probada pero con status `not_evaluated` → no se marca
---
## FASE 5 — Gestión Operativa Avanzada
**Duración estimada:** 2-3 semanas
**Dependencias:** Fase 0 (scoring en BD)
---
### Tarea 5.1: Scoring Compuesto Maduro
**Qué:** Reformar el scoring para que sea compuesto: % técnicas probadas × peso por criticidad × recencia × severidad.
**Implementación:**
Actualizar `scoring_service.py` para incluir factor de recencia (decay):
```python
def _recency_factor(last_tested: datetime) -> float:
"""Factor de decaimiento: 1.0 si reciente, disminuye con el tiempo."""
if not last_tested:
return 0.0
days_ago = (datetime.utcnow() - last_tested).days
if days_ago <= 90:
return 1.0
elif days_ago <= 180:
return 0.8
elif days_ago <= 365:
return 0.5
else:
return 0.2
```
Y persistir pesos en BD en vez de settings:
```python
class ScoringConfig(Base):
__tablename__ = "scoring_config"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
weight_tests = Column(Float, default=40.0)
weight_detection = Column(Float, default=25.0)
weight_d3fend = Column(Float, default=15.0)
weight_recency = Column(Float, default=10.0)
weight_severity = Column(Float, default=10.0)
updated_by = Column(UUID(as_uuid=True), ForeignKey("users.id"))
updated_at = Column(DateTime, default=datetime.utcnow)
```
**Verificación:**
- Score de técnica probada hace 1 año es menor que una probada ayer (mismo resultado)
- Cambiar pesos en BD se refleja inmediatamente en scores
- Pesos persisten tras reinicio del servidor
- Score compuesto total suma 100 siempre
---
### Tarea 5.2: Histórico y Evolución de Cobertura
**Qué:** Expandir snapshots para comparar entre meses, equipos y tácticas.
**Implementación:**
Añadir campos al modelo CoverageSnapshot:
```python
op.add_column('coverage_snapshots', Column('by_tactic', JSONB, default={}))
op.add_column('coverage_snapshots', Column('by_status', JSONB, default={}))
op.add_column('coverage_snapshots', Column('stale_count', Integer, default=0))
op.add_column('coverage_snapshots', Column('never_tested_count', Integer, default=0))
```
Y endpoint para consumir tendencias:
```python
@router.get("/evolution")
def coverage_evolution(months: int = 12, db: Session = Depends(get_db), user=Depends(get_current_user)):
cutoff = datetime.utcnow() - timedelta(days=months * 30)
snapshots = db.query(CoverageSnapshot).filter(
CoverageSnapshot.created_at >= cutoff
).order_by(CoverageSnapshot.created_at).all()
return [
{
"date": s.created_at.isoformat(),
"org_score": s.org_score,
"coverage_pct": s.coverage_percentage,
"by_tactic": s.by_tactic,
"stale_count": s.stale_count,
}
for s in snapshots
]
```
**Verificación:**
- Snapshot semanal ahora incluye desglose por táctica
- `/evolution?months=6` retorna snapshots de los últimos 6 meses
- Gráfico de tendencias en frontend muestra línea temporal
---
## FASE 6 — Analytics para BI + Webhooks (Feature Adicional)
**Duración estimada:** 2 semanas
**Dependencias:** Fase 2
---
### Tarea 6.1: Sistema de Webhooks
**Qué:** Permitir que Aegis envíe notificaciones HTTP a sistemas externos cuando ocurren eventos.
**Implementación:**
1. **Modelo — WebhookConfig:**
```python
class WebhookConfig(Base):
__tablename__ = "webhook_configs"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name = Column(String(200), nullable=False)
url = Column(Text, nullable=False)
secret = Column(String(256)) # Para HMAC signature
events = Column(JSONB, default=[]) # ["test.validated", "campaign.completed", ...]
is_active = Column(Boolean, default=True)
created_by = Column(UUID(as_uuid=True), ForeignKey("users.id"))
last_triggered_at = Column(DateTime)
failure_count = Column(Integer, default=0)
```
2. **Servicio de dispatch:**
```python
import requests
import hashlib
import hmac
import json
def dispatch_webhook(event_type: str, payload: dict):
db = SessionLocal()
try:
configs = db.query(WebhookConfig).filter(
WebhookConfig.is_active == True,
WebhookConfig.events.contains([event_type])
).all()
for config in configs:
try:
body = json.dumps({
"event": event_type,
"data": payload,
"timestamp": datetime.utcnow().isoformat()
})
headers = {"Content-Type": "application/json"}
if config.secret:
sig = hmac.new(config.secret.encode(), body.encode(), hashlib.sha256).hexdigest()
headers["X-Aegis-Signature"] = sig
resp = requests.post(config.url, data=body, headers=headers, timeout=10)
config.last_triggered_at = datetime.utcnow()
if resp.status_code >= 400:
config.failure_count += 1
except Exception as e:
config.failure_count += 1
logger.warning(f"Webhook {config.name} failed: {e}")
db.commit()
finally:
db.close()
```
3. Llamar `dispatch_webhook` en puntos clave: post-validación de test, completar campaña, sync MITRE, etc.
**Verificación:**
- Crear webhook para `test.validated` → validar un test → webhook recibe POST
- Signature HMAC es verificable en el receptor
- Webhook con URL inválida incrementa `failure_count` sin crashear
- Desactivar webhook → no se dispara
---
## FASE 7 — Notificaciones Multi-Canal (Feature Adicional)
**Duración estimada:** 1-2 semanas
**Dependencias:** Fase 3
---
### Tarea 7.1: Notificaciones por Email
**Qué:** Añadir capacidad de enviar notificaciones por email además de in-app.
**Implementación:**
1. **Config:**
```python
SMTP_ENABLED: bool = False
SMTP_HOST: str = ""
SMTP_PORT: int = 587
SMTP_USERNAME: str = ""
SMTP_PASSWORD: str = ""
SMTP_FROM_EMAIL: str = "aegis@company.com"
SMTP_USE_TLS: bool = True
```
2. **app/services/email_service.py:**
```python
import smtplib
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from app.config import settings
def send_email(to: str, subject: str, html_body: str):
if not settings.SMTP_ENABLED:
return
msg = MIMEMultipart("alternative")
msg["Subject"] = f"[Aegis] {subject}"
msg["From"] = settings.SMTP_FROM_EMAIL
msg["To"] = to
msg.attach(MIMEText(html_body, "html"))
with smtplib.SMTP(settings.SMTP_HOST, settings.SMTP_PORT) as server:
if settings.SMTP_USE_TLS:
server.starttls()
if settings.SMTP_USERNAME:
server.login(settings.SMTP_USERNAME, settings.SMTP_PASSWORD)
server.send_message(msg)
```
3. Integrar en `notification_service.py` — para eventos críticos (test validado, campaña completada, nueva técnica MITRE).
**Verificación:**
- Con SMTP configurado, validar test → email al lead correspondiente
- Email incluye link directo al test en la plataforma
- Con SMTP deshabilitado, no falla — solo skip
---
### Tarea 7.2: Preferencias de Notificación por Usuario
**Migración:**
```python
op.add_column('users', Column('notification_preferences', JSONB, default={
"email_on_test_validated": True,
"email_on_campaign_completed": True,
"email_on_new_mitre_techniques": False,
"in_app_all": True,
}))
op.add_column('users', Column('jira_account_id', String(100))) # para Tempo
```
---
## FASE 8 — Detection Lifecycle Management (DLM)
**Duración estimada:** 3-4 semanas
**Dependencias:** Fase 0 (Redis, índices, excepciones de dominio)
> **Por qué aquí:** Todo el resto de las fases avanzadas (9-14) depende de que las detecciones tengan ciclo de vida. Sin decay, confidence score y revalidación, el resto opera sobre datos estáticos que pierden valor con el tiempo.
>
> **Nota sobre paralelismo:** Esta fase puede ejecutarse en paralelo con Fases 1-7, ya que su única dependencia real es Fase 0. Sin embargo, se recomienda tener al menos Fases 1-5 funcionando para que haya datos reales sobre los que el DLM opere.
---
### Tarea 8.1: Modelo de Datos — Infraestructura de Detección
**Qué:** Crear las tablas que soportan el ciclo de vida de detecciones: metadatos de versión, asociaciones SIEM/EDR, y estado de salud.
**Implementación:**
**app/models/detection_lifecycle.py** (nuevo):
```python
import uuid
import enum
from datetime import datetime
from sqlalchemy import (
Column, String, Integer, Float, Boolean, DateTime,
ForeignKey, Text, Enum as SQLEnum
)
from sqlalchemy.dialects.postgresql import UUID, JSONB
from sqlalchemy.orm import relationship
from app.database import Base
class DetectionConfidence(str, enum.Enum):
fresh = "fresh" # Validado recientemente, todo OK
aging = "aging" # Acercándose a caducidad
stale = "stale" # Caducado, necesita revalidación
broken = "broken" # Se detectó cambio que invalida
unknown = "unknown" # Sin datos suficientes
class DetectionHealthStatus(str, enum.Enum):
healthy = "healthy" # Regla activa, disparando normalmente
silent = "silent" # Regla no ha disparado en período esperado
noisy = "noisy" # Regla disparando excesivamente (false positives)
orphan = "orphan" # Regla sin owner asignado
deprecated = "deprecated" # Regla marcada como obsoleta
untested = "untested" # Regla nunca validada
class InvalidationReason(str, enum.Enum):
time_decay = "time_decay"
mitre_update = "mitre_update"
log_source_change = "log_source_change"
siem_update = "siem_update"
edr_update = "edr_update"
infrastructure_change = "infrastructure_change"
parser_change = "parser_change"
manual = "manual"
rule_modified = "rule_modified"
class DetectionAsset(Base):
"""Representa un activo de detección concreto: una regla SIEM,
query SPL, regla YARA, regla EDR, etc."""
__tablename__ = "detection_assets"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name = Column(String(500), nullable=False)
description = Column(Text)
# Tipo y plataforma
asset_type = Column(String(50), nullable=False)
platform = Column(String(100))
# Contenido de la regla
rule_content = Column(Text)
rule_language = Column(String(50))
rule_repository_url = Column(Text)
rule_file_path = Column(String(500))
# Versionado
rule_version = Column(String(50))
rule_hash = Column(String(64))
last_rule_change_at = Column(DateTime)
# Log source tracking
log_source_name = Column(String(200))
log_source_version = Column(String(50))
log_source_config = Column(JSONB, default={})
# Infraestructura asociada
infrastructure_hash = Column(String(64))
infrastructure_details = Column(JSONB, default={})
# Estado de salud
health_status = Column(
SQLEnum(DetectionHealthStatus),
default=DetectionHealthStatus.untested
)
last_alert_at = Column(DateTime)
alert_count_30d = Column(Integer, default=0)
false_positive_rate = Column(Float)
expected_alert_frequency = Column(String(50))
# Ownership (se completa en Fase 9)
owner_id = Column(UUID(as_uuid=True), ForeignKey("users.id"))
backup_owner_id = Column(UUID(as_uuid=True), ForeignKey("users.id"))
team = Column(String(100))
# Metadata
is_active = Column(Boolean, default=True)
tags = Column(JSONB, default=[])
metadata = Column(JSONB, default={})
created_by = Column(UUID(as_uuid=True), ForeignKey("users.id"))
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
# Relationships
technique_mappings = relationship(
"DetectionTechniqueMapping", back_populates="detection_asset"
)
validations = relationship(
"DetectionValidation", back_populates="detection_asset"
)
class DetectionTechniqueMapping(Base):
"""Mapeo N:M entre activos de detección y técnicas MITRE."""
__tablename__ = "detection_technique_mappings"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
detection_asset_id = Column(
UUID(as_uuid=True),
ForeignKey("detection_assets.id", ondelete="CASCADE"),
nullable=False
)
technique_id = Column(
UUID(as_uuid=True),
ForeignKey("techniques.id", ondelete="CASCADE"),
nullable=False
)
coverage_type = Column(String(50), default="detect")
confidence_level = Column(String(20), default="medium")
notes = Column(Text)
created_at = Column(DateTime, default=datetime.utcnow)
detection_asset = relationship(
"DetectionAsset", back_populates="technique_mappings"
)
class DetectionValidation(Base):
"""Registro inmutable de cada validación de una detección.
Es el 'sello de calidad' con fecha de caducidad."""
__tablename__ = "detection_validations"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
detection_asset_id = Column(
UUID(as_uuid=True),
ForeignKey("detection_assets.id", ondelete="CASCADE"),
nullable=False
)
technique_id = Column(UUID(as_uuid=True), ForeignKey("techniques.id"), nullable=True)
test_id = Column(UUID(as_uuid=True), ForeignKey("tests.id"), nullable=True)
# Resultado
validated_at = Column(DateTime, default=datetime.utcnow)
expires_at = Column(DateTime, nullable=False)
is_valid = Column(Boolean, default=True)
validation_result = Column(String(50))
validation_method = Column(String(100))
# Snapshot del estado en el momento de validación
rule_hash_at_validation = Column(String(64))
log_source_version_at_validation = Column(String(50))
infrastructure_hash_at_validation = Column(String(64))
environment_snapshot = Column(JSONB, default={})
# Invalidación
invalidated_at = Column(DateTime)
invalidation_reason = Column(SQLEnum(InvalidationReason))
invalidation_details = Column(Text)
invalidated_by = Column(UUID(as_uuid=True), ForeignKey("users.id"))
# Quién validó
validated_by = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
# Integridad
integrity_hash = Column(String(64))
notes = Column(Text)
evidence_ids = Column(JSONB, default=[])
detection_asset = relationship("DetectionAsset", back_populates="validations")
class TechniqueConfidenceScore(Base):
"""Score calculado de confianza por técnica.
Se recalcula periódicamente por el decay engine."""
__tablename__ = "technique_confidence_scores"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
technique_id = Column(
UUID(as_uuid=True),
ForeignKey("techniques.id", ondelete="CASCADE"),
nullable=False, unique=True
)
confidence_level = Column(
SQLEnum(DetectionConfidence), default=DetectionConfidence.unknown
)
confidence_score = Column(Float, default=0.0)
detection_count = Column(Integer, default=0)
valid_detection_count = Column(Integer, default=0)
last_validated_at = Column(DateTime)
next_validation_due = Column(DateTime)
last_recalculated_at = Column(DateTime, default=datetime.utcnow)
recency_factor = Column(Float, default=0.0)
coverage_factor = Column(Float, default=0.0)
health_factor = Column(Float, default=0.0)
diversity_factor = Column(Float, default=0.0)
score_breakdown = Column(JSONB, default={})
risk_factors = Column(JSONB, default=[])
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class InfrastructureChangeLog(Base):
"""Registro de cambios en infraestructura que pueden invalidar detecciones."""
__tablename__ = "infrastructure_change_logs"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
change_type = Column(String(100), nullable=False)
description = Column(Text, nullable=False)
affected_platforms = Column(JSONB, default=[])
affected_log_sources = Column(JSONB, default=[])
change_date = Column(DateTime, default=datetime.utcnow)
reported_by = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
auto_invalidate = Column(Boolean, default=True)
invalidated_count = Column(Integer, default=0)
metadata = Column(JSONB, default={})
created_at = Column(DateTime, default=datetime.utcnow)
```
**Migración Alembic:**
```bash
alembic revision --autogenerate -m "add_detection_lifecycle_tables"
alembic upgrade head
```
**Índices adicionales** (agregar manualmente en la migración):
```python
def upgrade():
# ... tablas auto-generadas ...
op.create_index('ix_detection_assets_platform', 'detection_assets', ['platform'])
op.create_index('ix_detection_assets_health_status', 'detection_assets', ['health_status'])
op.create_index('ix_detection_assets_owner_id', 'detection_assets', ['owner_id'])
op.create_index('ix_detection_technique_mappings_technique_id', 'detection_technique_mappings', ['technique_id'])
op.create_index('ix_detection_technique_mappings_asset_id', 'detection_technique_mappings', ['detection_asset_id'])
op.create_index('ix_detection_validations_asset_id_valid', 'detection_validations', ['detection_asset_id', 'is_valid'])
op.create_index('ix_detection_validations_expires_at', 'detection_validations', ['expires_at'])
op.create_index('ix_technique_confidence_scores_technique_id', 'technique_confidence_scores', ['technique_id'])
op.create_index('ix_technique_confidence_scores_confidence_level', 'technique_confidence_scores', ['confidence_level'])
op.create_index('ix_infrastructure_change_logs_change_date', 'infrastructure_change_logs', ['change_date'])
```
**app/schemas/detection_lifecycle_schema.py** (nuevo):
```python
from pydantic import BaseModel, Field
from typing import Optional
from uuid import UUID
from datetime import datetime
from app.models.detection_lifecycle import (
DetectionConfidence, DetectionHealthStatus, InvalidationReason
)
class DetectionAssetCreate(BaseModel):
name: str = Field(..., min_length=3, max_length=500)
description: Optional[str] = None
asset_type: str = Field(
...,
pattern=r'^(siem_rule|edr_rule|sigma_rule|yara_rule|spl_query|kql_query|custom_script)$'
)
platform: Optional[str] = None
rule_content: Optional[str] = None
rule_language: Optional[str] = None
rule_repository_url: Optional[str] = None
rule_file_path: Optional[str] = None
rule_version: Optional[str] = None
log_source_name: Optional[str] = None
log_source_version: Optional[str] = None
log_source_config: Optional[dict] = {}
infrastructure_details: Optional[dict] = {}
expected_alert_frequency: Optional[str] = None
tags: Optional[list[str]] = []
technique_ids: Optional[list[UUID]] = []
class DetectionAssetUpdate(BaseModel):
name: Optional[str] = None
description: Optional[str] = None
rule_content: Optional[str] = None
rule_version: Optional[str] = None
log_source_version: Optional[str] = None
infrastructure_details: Optional[dict] = None
expected_alert_frequency: Optional[str] = None
health_status: Optional[DetectionHealthStatus] = None
last_alert_at: Optional[datetime] = None
alert_count_30d: Optional[int] = None
false_positive_rate: Optional[float] = None
owner_id: Optional[UUID] = None
backup_owner_id: Optional[UUID] = None
team: Optional[str] = None
tags: Optional[list[str]] = None
is_active: Optional[bool] = None
class DetectionAssetOut(BaseModel):
id: UUID
name: str
description: Optional[str]
asset_type: str
platform: Optional[str]
rule_language: Optional[str]
rule_version: Optional[str]
rule_hash: Optional[str]
health_status: DetectionHealthStatus
last_alert_at: Optional[datetime]
alert_count_30d: int
false_positive_rate: Optional[float]
expected_alert_frequency: Optional[str]
owner_id: Optional[UUID]
team: Optional[str]
is_active: bool
tags: list
created_at: datetime
updated_at: datetime
class Config:
from_attributes = True
class DetectionValidationCreate(BaseModel):
detection_asset_id: UUID
technique_id: Optional[UUID] = None
test_id: Optional[UUID] = None
validation_result: str = Field(
..., pattern=r'^(detected|not_detected|partial|error)$'
)
validation_method: str
notes: Optional[str] = None
evidence_ids: Optional[list[UUID]] = []
validity_days: int = Field(default=180, ge=30, le=730)
class DetectionValidationOut(BaseModel):
id: UUID
detection_asset_id: UUID
technique_id: Optional[UUID]
validated_at: datetime
expires_at: datetime
is_valid: bool
validation_result: str
validation_method: str
invalidated_at: Optional[datetime]
invalidation_reason: Optional[InvalidationReason]
validated_by: UUID
notes: Optional[str]
class Config:
from_attributes = True
class TechniqueConfidenceOut(BaseModel):
technique_id: UUID
confidence_level: DetectionConfidence
confidence_score: float
detection_count: int
valid_detection_count: int
last_validated_at: Optional[datetime]
next_validation_due: Optional[datetime]
recency_factor: float
coverage_factor: float
health_factor: float
diversity_factor: float
risk_factors: list
class Config:
from_attributes = True
class InfrastructureChangeCreate(BaseModel):
change_type: str
description: str = Field(..., min_length=10)
affected_platforms: list[str] = []
affected_log_sources: list[str] = []
change_date: Optional[datetime] = None
auto_invalidate: bool = True
class InfrastructureChangeOut(BaseModel):
id: UUID
change_type: str
description: str
affected_platforms: list
affected_log_sources: list
change_date: datetime
auto_invalidate: bool
invalidated_count: int
reported_by: UUID
created_at: datetime
class Config:
from_attributes = True
```
**Verificación:**
- `alembic upgrade head` ejecuta sin error
- `\dt` en psql muestra las 5 nuevas tablas
- `\di` muestra todos los índices creados
- Schemas validan correctamente: `DetectionAssetCreate(name="x", asset_type="invalid")` → error
- Los JSONB defaults funcionan: crear un DetectionAsset sin tags → `tags = []`
---
### Tarea 8.2: Servicio de Detection Assets — CRUD + Versionado
**Qué:** Servicio que encapsula la gestión de activos de detección con auto-hash de contenido y detección de cambios.
**Implementación:**
**app/services/detection_asset_service.py** (nuevo):
```python
import hashlib
import logging
from datetime import datetime
from typing import Optional
from uuid import UUID
from sqlalchemy.orm import Session, joinedload
from sqlalchemy import func
from app.models.detection_lifecycle import (
DetectionAsset, DetectionTechniqueMapping,
DetectionValidation, DetectionHealthStatus
)
from app.models.technique import Technique
from app.domain.exceptions import (
EntityNotFoundError, DuplicateEntityError, InvalidOperationError
)
from app.services import audit_service
logger = logging.getLogger(__name__)
def _compute_rule_hash(content: str) -> str:
"""SHA256 del contenido de la regla, normalizado."""
normalized = content.strip().replace('\r\n', '\n')
return hashlib.sha256(normalized.encode()).hexdigest()
def create_detection_asset(
db: Session, data: dict, user_id: UUID
) -> DetectionAsset:
"""Crea un nuevo activo de detección con hash automático."""
technique_ids = data.pop("technique_ids", [])
asset = DetectionAsset(**data, created_by=user_id)
if asset.rule_content:
asset.rule_hash = _compute_rule_hash(asset.rule_content)
asset.last_rule_change_at = datetime.utcnow()
if asset.infrastructure_details:
infra_str = str(sorted(asset.infrastructure_details.items()))
asset.infrastructure_hash = hashlib.sha256(infra_str.encode()).hexdigest()
db.add(asset)
db.flush()
for tech_id in technique_ids:
technique = db.query(Technique).filter(Technique.id == tech_id).first()
if technique:
mapping = DetectionTechniqueMapping(
detection_asset_id=asset.id,
technique_id=tech_id,
)
db.add(mapping)
db.commit()
db.refresh(asset)
audit_service.log_action(
db, user_id, "DETECTION_ASSET_CREATED",
"detection_asset", str(asset.id),
details={
"name": asset.name,
"type": asset.asset_type,
"platform": asset.platform,
"technique_count": len(technique_ids),
}
)
return asset
def update_detection_asset(
db: Session, asset_id: UUID, data: dict, user_id: UUID
) -> DetectionAsset:
"""Actualiza un activo, detectando cambios en contenido de regla."""
asset = db.query(DetectionAsset).filter(DetectionAsset.id == asset_id).first()
if not asset:
raise EntityNotFoundError("DetectionAsset", str(asset_id))
changes = {}
rule_changed = False
for key, value in data.items():
if value is not None and hasattr(asset, key):
old_value = getattr(asset, key)
if old_value != value:
changes[key] = {"old": str(old_value), "new": str(value)}
setattr(asset, key, value)
if "rule_content" in data and data["rule_content"]:
new_hash = _compute_rule_hash(data["rule_content"])
if new_hash != asset.rule_hash:
rule_changed = True
asset.rule_hash = new_hash
asset.last_rule_change_at = datetime.utcnow()
changes["rule_hash"] = {"old": asset.rule_hash, "new": new_hash}
if "infrastructure_details" in data and data["infrastructure_details"]:
infra_str = str(sorted(data["infrastructure_details"].items()))
new_hash = hashlib.sha256(infra_str.encode()).hexdigest()
if new_hash != asset.infrastructure_hash:
asset.infrastructure_hash = new_hash
changes["infrastructure_hash_changed"] = True
db.commit()
db.refresh(asset)
if changes:
audit_service.log_action(
db, user_id, "DETECTION_ASSET_UPDATED",
"detection_asset", str(asset.id),
details={"changes": changes, "rule_changed": rule_changed}
)
if rule_changed:
_invalidate_validations_for_asset(db, asset.id, user_id, "rule_modified")
return asset
def _invalidate_validations_for_asset(
db: Session, asset_id: UUID, user_id: UUID, reason: str
):
"""Invalida todas las validaciones vigentes de un asset."""
validations = db.query(DetectionValidation).filter(
DetectionValidation.detection_asset_id == asset_id,
DetectionValidation.is_valid == True,
).all()
count = 0
for v in validations:
v.is_valid = False
v.invalidated_at = datetime.utcnow()
v.invalidation_reason = reason
v.invalidated_by = user_id
count += 1
if count > 0:
db.commit()
logger.info(f"Invalidated {count} validations for asset {asset_id} due to {reason}")
return count
def get_asset_with_details(db: Session, asset_id: UUID) -> DetectionAsset:
"""Obtiene un asset con sus mapeos y validaciones."""
asset = (
db.query(DetectionAsset)
.options(
joinedload(DetectionAsset.technique_mappings),
joinedload(DetectionAsset.validations),
)
.filter(DetectionAsset.id == asset_id)
.first()
)
if not asset:
raise EntityNotFoundError("DetectionAsset", str(asset_id))
return asset
def list_assets(
db: Session,
platform: Optional[str] = None,
asset_type: Optional[str] = None,
health_status: Optional[str] = None,
technique_id: Optional[UUID] = None,
is_active: Optional[bool] = True,
) -> list[DetectionAsset]:
"""Lista assets con filtros opcionales."""
query = db.query(DetectionAsset)
if platform:
query = query.filter(DetectionAsset.platform == platform)
if asset_type:
query = query.filter(DetectionAsset.asset_type == asset_type)
if health_status:
query = query.filter(DetectionAsset.health_status == health_status)
if is_active is not None:
query = query.filter(DetectionAsset.is_active == is_active)
if technique_id:
query = query.join(DetectionTechniqueMapping).filter(
DetectionTechniqueMapping.technique_id == technique_id
)
return query.order_by(DetectionAsset.name).all()
def get_technique_detection_summary(db: Session, technique_id: UUID) -> dict:
"""Resumen de detecciones para una técnica específica."""
mappings = (
db.query(DetectionTechniqueMapping)
.options(joinedload(DetectionTechniqueMapping.detection_asset))
.filter(DetectionTechniqueMapping.technique_id == technique_id)
.all()
)
assets = [m.detection_asset for m in mappings if m.detection_asset]
active_assets = [a for a in assets if a.is_active]
valid_count = 0
for asset in active_assets:
has_valid = db.query(DetectionValidation).filter(
DetectionValidation.detection_asset_id == asset.id,
DetectionValidation.is_valid == True,
DetectionValidation.expires_at > datetime.utcnow(),
).first()
if has_valid:
valid_count += 1
health_distribution = {}
for asset in active_assets:
status = asset.health_status.value if asset.health_status else "unknown"
health_distribution[status] = health_distribution.get(status, 0) + 1
platforms = list(set(a.platform for a in active_assets if a.platform))
return {
"technique_id": str(technique_id),
"total_assets": len(active_assets),
"validated_assets": valid_count,
"health_distribution": health_distribution,
"platforms": platforms,
"coverage_types": list(set(m.coverage_type for m in mappings if m.coverage_type)),
}
```
**Verificación:**
- Crear un DetectionAsset con `rule_content` → `rule_hash` se genera automáticamente
- Actualizar `rule_content` con contenido diferente → hash cambia + `last_rule_change_at` se actualiza + validaciones vigentes se invalidan
- Actualizar con mismo contenido → hash no cambia, validaciones intactas
- `list_assets(platform="Splunk")` filtra correctamente
- `get_technique_detection_summary` retorna conteos correctos
- Audit log registra creación y actualización con detalles de cambios
---
### Tarea 8.3: Decay Engine — Motor de Obsolescencia
**Qué:** El cerebro del sistema de lifecycle. Evalúa periódicamente todas las detecciones y degrada su estado según políticas configurables.
> **Nota:** Este motor reemplaza funcionalmente la detección simple de Fase 4 (Tarea 4.2). La Tarea 4.2 puede seguir activa como fallback simple, pero el Decay Engine es el sistema definitivo.
**Implementación:**
**app/models/decay_policy.py** (nuevo):
```python
import uuid
from datetime import datetime
from sqlalchemy import Column, String, Integer, Float, Boolean, DateTime
from sqlalchemy.dialects.postgresql import UUID, JSONB
from app.database import Base
class DecayPolicy(Base):
"""Política de caducidad configurable por plataforma/tipo."""
__tablename__ = "decay_policies"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name = Column(String(200), nullable=False)
description = Column(String(500))
# Alcance
applies_to_platform = Column(String(100)) # null = todas
applies_to_asset_type = Column(String(50)) # null = todos
applies_to_tactic = Column(String(100)) # null = todas
# Umbrales de tiempo (días)
fresh_days = Column(Integer, default=90) # 0-90 = Fresh
aging_days = Column(Integer, default=180) # 91-180 = Aging
stale_days = Column(Integer, default=365) # 181-365 = Stale
# >365 = Broken (implícito)
# Validación por defecto
default_validity_days = Column(Integer, default=180)
# Health: umbrales
silent_threshold_days = Column(Integer, default=30)
noisy_threshold_daily = Column(Integer, default=100)
# Pesos de factores
recency_weight = Column(Float, default=0.3)
coverage_weight = Column(Float, default=0.3)
health_weight = Column(Float, default=0.25)
diversity_weight = Column(Float, default=0.15)
is_default = Column(Boolean, default=False)
is_active = Column(Boolean, default=True)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
```
**app/services/decay_engine_service.py** (nuevo):
```python
import logging
from datetime import datetime, timedelta
from typing import Optional
from uuid import UUID
from sqlalchemy.orm import Session
from sqlalchemy import func, and_
from app.models.detection_lifecycle import (
DetectionAsset, DetectionValidation,
DetectionTechniqueMapping, TechniqueConfidenceScore,
DetectionConfidence, DetectionHealthStatus,
InfrastructureChangeLog
)
from app.models.decay_policy import DecayPolicy
from app.models.technique import Technique
logger = logging.getLogger(__name__)
def get_applicable_policy(
db: Session,
platform: Optional[str] = None,
asset_type: Optional[str] = None,
tactic: Optional[str] = None,
) -> DecayPolicy:
"""Obtiene la política más específica aplicable."""
query = db.query(DecayPolicy).filter(DecayPolicy.is_active == True)
if platform:
specific = query.filter(DecayPolicy.applies_to_platform == platform).first()
if specific:
return specific
if asset_type:
specific = query.filter(DecayPolicy.applies_to_asset_type == asset_type).first()
if specific:
return specific
default_policy = query.filter(DecayPolicy.is_default == True).first()
if default_policy:
return default_policy
return DecayPolicy()
def calculate_confidence_for_technique(
db: Session, technique_id: UUID
) -> TechniqueConfidenceScore:
"""Calcula el Confidence Score completo para una técnica."""
technique = db.query(Technique).filter(Technique.id == technique_id).first()
if not technique:
return None
policy = get_applicable_policy(db, tactic=technique.tactic)
mappings = (
db.query(DetectionTechniqueMapping)
.filter(DetectionTechniqueMapping.technique_id == technique_id)
.all()
)
asset_ids = [m.detection_asset_id for m in mappings]
if not asset_ids:
return _create_or_update_score(
db, technique_id,
confidence_level=DetectionConfidence.unknown,
confidence_score=0.0,
factors={"recency": 0.0, "coverage": 0.0, "health": 0.0, "diversity": 0.0},
risk_factors=["no_detection_assets"],
detection_count=0, valid_count=0,
)
assets = (
db.query(DetectionAsset)
.filter(DetectionAsset.id.in_(asset_ids), DetectionAsset.is_active == True)
.all()
)
now = datetime.utcnow()
# 1. RECENCY FACTOR
valid_validations = (
db.query(DetectionValidation)
.filter(
DetectionValidation.detection_asset_id.in_(asset_ids),
DetectionValidation.is_valid == True,
DetectionValidation.expires_at > now,
)
.all()
)
recency_factor = 0.0
last_validated = None
if valid_validations:
most_recent = max(v.validated_at for v in valid_validations)
last_validated = most_recent
days_since = (now - most_recent).days
if days_since <= policy.fresh_days:
recency_factor = 1.0
elif days_since <= policy.aging_days:
range_days = policy.aging_days - policy.fresh_days
elapsed = days_since - policy.fresh_days
recency_factor = 1.0 - (elapsed / range_days) * 0.4
elif days_since <= policy.stale_days:
range_days = policy.stale_days - policy.aging_days
elapsed = days_since - policy.aging_days
recency_factor = 0.6 - (elapsed / range_days) * 0.4
else:
recency_factor = max(0.1, 0.2 - ((days_since - policy.stale_days) / 365) * 0.1)
# 2. COVERAGE FACTOR
active_count = len(assets)
valid_count = len(set(v.detection_asset_id for v in valid_validations))
if active_count == 0:
coverage_factor = 0.0
elif valid_count >= 3:
coverage_factor = 1.0
elif valid_count >= 2:
coverage_factor = 0.8
elif valid_count >= 1:
coverage_factor = 0.5
else:
coverage_factor = 0.1
# 3. HEALTH FACTOR
health_scores = {
DetectionHealthStatus.healthy: 1.0,
DetectionHealthStatus.silent: 0.4,
DetectionHealthStatus.noisy: 0.6,
DetectionHealthStatus.orphan: 0.3,
DetectionHealthStatus.deprecated: 0.0,
DetectionHealthStatus.untested: 0.2,
}
if assets:
health_values = [health_scores.get(a.health_status, 0.2) for a in assets]
health_factor = sum(health_values) / len(health_values)
else:
health_factor = 0.0
# 4. DIVERSITY FACTOR
platforms = set(a.platform for a in assets if a.platform)
asset_types = set(a.asset_type for a in assets)
diversity_factor = min(1.0, (len(platforms) * 0.3 + len(asset_types) * 0.2))
# SCORE COMPUESTO
confidence_score = (
recency_factor * policy.recency_weight +
coverage_factor * policy.coverage_weight +
health_factor * policy.health_weight +
diversity_factor * policy.diversity_weight
) * 100
# CONFIDENCE LEVEL
if confidence_score >= 75:
confidence_level = DetectionConfidence.fresh
elif confidence_score >= 50:
confidence_level = DetectionConfidence.aging
elif confidence_score >= 25:
confidence_level = DetectionConfidence.stale
elif confidence_score > 0:
confidence_level = DetectionConfidence.broken
else:
confidence_level = DetectionConfidence.unknown
# RISK FACTORS
risk_factors = []
if len(platforms) <= 1:
risk_factors.append("single_platform")
if valid_count == 0:
risk_factors.append("no_valid_detections")
if any(a.health_status == DetectionHealthStatus.silent for a in assets):
risk_factors.append("silent_rules_present")
if any(a.health_status == DetectionHealthStatus.orphan for a in assets):
risk_factors.append("orphan_rules_present")
if recency_factor < 0.5:
risk_factors.append("stale_validation")
if len(assets) < 2:
risk_factors.append("low_detection_diversity")
next_due = None
if valid_validations:
earliest_expiry = min(v.expires_at for v in valid_validations)
next_due = earliest_expiry
return _create_or_update_score(
db, technique_id,
confidence_level=confidence_level,
confidence_score=round(confidence_score, 1),
factors={
"recency": round(recency_factor, 3),
"coverage": round(coverage_factor, 3),
"health": round(health_factor, 3),
"diversity": round(diversity_factor, 3),
},
risk_factors=risk_factors,
detection_count=active_count,
valid_count=valid_count,
last_validated=last_validated,
next_due=next_due,
)
def _create_or_update_score(db: Session, technique_id: UUID, **kwargs) -> TechniqueConfidenceScore:
"""Crea o actualiza el score de confianza."""
score = db.query(TechniqueConfidenceScore).filter(
TechniqueConfidenceScore.technique_id == technique_id
).first()
if not score:
score = TechniqueConfidenceScore(technique_id=technique_id)
db.add(score)
score.confidence_level = kwargs["confidence_level"]
score.confidence_score = kwargs["confidence_score"]
score.detection_count = kwargs["detection_count"]
score.valid_detection_count = kwargs["valid_count"]
score.recency_factor = kwargs["factors"]["recency"]
score.coverage_factor = kwargs["factors"]["coverage"]
score.health_factor = kwargs["factors"]["health"]
score.diversity_factor = kwargs["factors"]["diversity"]
score.risk_factors = kwargs["risk_factors"]
score.score_breakdown = kwargs["factors"]
score.last_validated_at = kwargs.get("last_validated")
score.next_validation_due = kwargs.get("next_due")
score.last_recalculated_at = datetime.utcnow()
db.commit()
db.refresh(score)
return score
def run_decay_engine(db: Session) -> dict:
"""Ejecuta el motor de decay sobre todas las técnicas. Job diario."""
techniques = db.query(Technique).all()
results = {
"total_techniques": len(techniques),
"fresh": 0, "aging": 0, "stale": 0,
"broken": 0, "unknown": 0,
"validations_expired": 0,
}
now = datetime.utcnow()
# 1. Expirar validaciones vencidas
expired = (
db.query(DetectionValidation)
.filter(DetectionValidation.is_valid == True, DetectionValidation.expires_at <= now)
.all()
)
for v in expired:
v.is_valid = False
v.invalidated_at = now
v.invalidation_reason = "time_decay"
results["validations_expired"] = len(expired)
if expired:
db.commit()
# 2. Recalcular confidence scores
for technique in techniques:
score = calculate_confidence_for_technique(db, technique.id)
if score:
level = score.confidence_level.value
results[level] = results.get(level, 0) + 1
logger.info(f"Decay engine completed: {results}")
return results
def process_infrastructure_change(db: Session, change_id: UUID) -> int:
"""Procesa un cambio de infraestructura e invalida detecciones afectadas."""
change = db.query(InfrastructureChangeLog).filter(
InfrastructureChangeLog.id == change_id
).first()
if not change or not change.auto_invalidate:
return 0
query = db.query(DetectionAsset).filter(DetectionAsset.is_active == True)
if change.affected_platforms:
query = query.filter(DetectionAsset.platform.in_(change.affected_platforms))
affected_assets = query.all()
total_invalidated = 0
for asset in affected_assets:
if change.affected_log_sources:
asset_log_source = asset.log_source_name or ""
if not any(ls in asset_log_source for ls in change.affected_log_sources):
continue
count = _invalidate_validations_for_asset(
db, asset.id, change.reported_by, "infrastructure_change"
)
total_invalidated += count
change.invalidated_count = total_invalidated
db.commit()
logger.info(f"Infrastructure change {change_id}: invalidated {total_invalidated} validations")
return total_invalidated
def _invalidate_validations_for_asset(
db: Session, asset_id: UUID, user_id: UUID, reason: str
) -> int:
"""Invalida validaciones vigentes de un asset."""
validations = db.query(DetectionValidation).filter(
DetectionValidation.detection_asset_id == asset_id,
DetectionValidation.is_valid == True,
).all()
for v in validations:
v.is_valid = False
v.invalidated_at = datetime.utcnow()
v.invalidation_reason = reason
v.invalidated_by = user_id
return len(validations)
```
**Registrar job en scheduler:**
```python
scheduler.add_job(
decay_engine_job,
"cron",
hour=2, minute=0,
id="decay_engine",
replace_existing=True,
)
def decay_engine_job():
db = SessionLocal()
try:
from app.services.decay_engine_service import run_decay_engine
results = run_decay_engine(db)
logger.info(f"Decay engine job results: {results}")
except Exception as e:
logger.error(f"Decay engine job failed: {e}", exc_info=True)
finally:
db.close()
```
**Verificación:**
- Crear asset + validación con `expires_at = ayer` → ejecutar `run_decay_engine()` → validación marcada `is_valid=False` con `invalidation_reason="time_decay"`
- Técnica con validación vigente → `confidence_level = "fresh"`
- Técnica con validación de hace 100 días → `confidence_level = "aging"`
- Técnica sin validaciones → `confidence_level = "unknown"`
- Registrar infrastructure change con `affected_platforms=["Splunk"]` → invalida validaciones de assets Splunk
- Log muestra resumen completo del decay engine
---
### Tarea 8.4: Router de Detection Lifecycle
**Qué:** Endpoints API para gestionar todo el ciclo de vida de detecciones.
**Implementación:**
**app/routers/detection_lifecycle.py** (nuevo):
```python
from fastapi import APIRouter, Depends, Query
from sqlalchemy.orm import Session
from uuid import UUID
from typing import Optional
from datetime import datetime, timedelta
from app.database import get_db
from app.dependencies.auth import get_current_user, require_role
from app.schemas.detection_lifecycle_schema import (
DetectionAssetCreate, DetectionAssetUpdate, DetectionAssetOut,
DetectionValidationCreate, DetectionValidationOut,
TechniqueConfidenceOut,
InfrastructureChangeCreate, InfrastructureChangeOut,
)
from app.services import detection_asset_service, decay_engine_service
from app.services import audit_service
from app.models.detection_lifecycle import (
DetectionAsset, DetectionValidation,
DetectionTechniqueMapping, TechniqueConfidenceScore,
InfrastructureChangeLog, DetectionHealthStatus,
)
from app.models.decay_policy import DecayPolicy
from app.domain.exceptions import EntityNotFoundError
import hashlib
router = APIRouter(prefix="/detection-lifecycle", tags=["detection-lifecycle"])
# — Detection Assets —
@router.post("/assets", response_model=DetectionAssetOut, status_code=201)
def create_asset(
body: DetectionAssetCreate,
db: Session = Depends(get_db),
user=Depends(get_current_user),
):
return detection_asset_service.create_detection_asset(db, body.model_dump(), user.id)
@router.get("/assets", response_model=list[DetectionAssetOut])
def list_assets(
platform: Optional[str] = None,
asset_type: Optional[str] = None,
health_status: Optional[str] = None,
technique_id: Optional[UUID] = None,
is_active: Optional[bool] = True,
db: Session = Depends(get_db),
user=Depends(get_current_user),
):
return detection_asset_service.list_assets(
db, platform=platform, asset_type=asset_type,
health_status=health_status, technique_id=technique_id, is_active=is_active,
)
@router.get("/assets/{asset_id}")
def get_asset(
asset_id: UUID, db: Session = Depends(get_db), user=Depends(get_current_user),
):
return detection_asset_service.get_asset_with_details(db, asset_id)
@router.patch("/assets/{asset_id}", response_model=DetectionAssetOut)
def update_asset(
asset_id: UUID, body: DetectionAssetUpdate,
db: Session = Depends(get_db), user=Depends(get_current_user),
):
return detection_asset_service.update_detection_asset(
db, asset_id, body.model_dump(exclude_unset=True), user.id
)
# — Technique Mappings —
@router.post("/assets/{asset_id}/techniques/{technique_id}")
def map_technique(
asset_id: UUID, technique_id: UUID,
coverage_type: str = Query("detect"),
confidence_level: str = Query("medium"),
db: Session = Depends(get_db), user=Depends(get_current_user),
):
mapping = DetectionTechniqueMapping(
detection_asset_id=asset_id, technique_id=technique_id,
coverage_type=coverage_type, confidence_level=confidence_level,
)
db.add(mapping)
db.commit()
return {"message": "Technique mapped", "mapping_id": str(mapping.id)}
@router.get("/techniques/{technique_id}/detections")
def get_technique_detections(
technique_id: UUID, db: Session = Depends(get_db), user=Depends(get_current_user),
):
return detection_asset_service.get_technique_detection_summary(db, technique_id)
# — Validations —
@router.post("/validations", response_model=DetectionValidationOut, status_code=201)
def create_validation(
body: DetectionValidationCreate,
db: Session = Depends(get_db), user=Depends(get_current_user),
):
asset = db.query(DetectionAsset).filter(
DetectionAsset.id == body.detection_asset_id
).first()
if not asset:
raise EntityNotFoundError("DetectionAsset", str(body.detection_asset_id))
validation = DetectionValidation(
detection_asset_id=body.detection_asset_id,
technique_id=body.technique_id,
test_id=body.test_id,
validation_result=body.validation_result,
validation_method=body.validation_method,
notes=body.notes,
evidence_ids=body.evidence_ids or [],
validated_by=user.id,
expires_at=datetime.utcnow() + timedelta(days=body.validity_days),
rule_hash_at_validation=asset.rule_hash,
log_source_version_at_validation=asset.log_source_version,
infrastructure_hash_at_validation=asset.infrastructure_hash,
)
data = f"{validation.detection_asset_id}:{validation.validated_by}:{validation.validation_result}:{validation.validated_at}"
validation.integrity_hash = hashlib.sha256(data.encode()).hexdigest()
db.add(validation)
db.commit()
db.refresh(validation)
if body.technique_id:
decay_engine_service.calculate_confidence_for_technique(db, body.technique_id)
audit_service.log_action(
db, user.id, "DETECTION_VALIDATED",
"detection_validation", str(validation.id),
details={
"asset_id": str(body.detection_asset_id),
"result": body.validation_result,
"validity_days": body.validity_days,
}
)
return validation
@router.get("/validations", response_model=list[DetectionValidationOut])
def list_validations(
asset_id: Optional[UUID] = None,
technique_id: Optional[UUID] = None,
is_valid: Optional[bool] = None,
db: Session = Depends(get_db), user=Depends(get_current_user),
):
query = db.query(DetectionValidation)
if asset_id:
query = query.filter(DetectionValidation.detection_asset_id == asset_id)
if technique_id:
query = query.filter(DetectionValidation.technique_id == technique_id)
if is_valid is not None:
query = query.filter(DetectionValidation.is_valid == is_valid)
return query.order_by(DetectionValidation.validated_at.desc()).all()
@router.post("/validations/{validation_id}/invalidate")
def invalidate_validation(
validation_id: UUID,
reason: str = Query(...),
details: Optional[str] = None,
db: Session = Depends(get_db),
user=Depends(require_role("admin", "blue_lead")),
):
validation = db.query(DetectionValidation).filter(
DetectionValidation.id == validation_id
).first()
if not validation:
raise EntityNotFoundError("DetectionValidation", str(validation_id))
validation.is_valid = False
validation.invalidated_at = datetime.utcnow()
validation.invalidation_reason = reason
validation.invalidation_details = details
validation.invalidated_by = user.id
db.commit()
return {"message": "Validation invalidated"}
# — Confidence Scores —
@router.get("/confidence", response_model=list[TechniqueConfidenceOut])
def list_confidence_scores(
confidence_level: Optional[str] = None,
min_score: Optional[float] = None,
max_score: Optional[float] = None,
db: Session = Depends(get_db), user=Depends(get_current_user),
):
query = db.query(TechniqueConfidenceScore)
if confidence_level:
query = query.filter(TechniqueConfidenceScore.confidence_level == confidence_level)
if min_score is not None:
query = query.filter(TechniqueConfidenceScore.confidence_score >= min_score)
if max_score is not None:
query = query.filter(TechniqueConfidenceScore.confidence_score <= max_score)
return query.order_by(TechniqueConfidenceScore.confidence_score.asc()).all()
@router.get("/confidence/{technique_id}", response_model=TechniqueConfidenceOut)
def get_technique_confidence(
technique_id: UUID, recalculate: bool = Query(False),
db: Session = Depends(get_db), user=Depends(get_current_user),
):
if recalculate:
return decay_engine_service.calculate_confidence_for_technique(db, technique_id)
score = db.query(TechniqueConfidenceScore).filter(
TechniqueConfidenceScore.technique_id == technique_id
).first()
if not score:
return decay_engine_service.calculate_confidence_for_technique(db, technique_id)
return score
# — Infrastructure Changes —
@router.post("/infrastructure-changes", response_model=InfrastructureChangeOut, status_code=201)
def report_infrastructure_change(
body: InfrastructureChangeCreate,
db: Session = Depends(get_db),
user=Depends(require_role("admin", "blue_lead")),
):
change = InfrastructureChangeLog(
change_type=body.change_type,
description=body.description,
affected_platforms=body.affected_platforms,
affected_log_sources=body.affected_log_sources,
change_date=body.change_date or datetime.utcnow(),
auto_invalidate=body.auto_invalidate,
reported_by=user.id,
)
db.add(change)
db.commit()
db.refresh(change)
if change.auto_invalidate:
decay_engine_service.process_infrastructure_change(db, change.id)
db.refresh(change)
audit_service.log_action(
db, user.id, "INFRASTRUCTURE_CHANGE_REPORTED",
"infrastructure_change", str(change.id),
details={"type": body.change_type, "invalidated_count": change.invalidated_count}
)
return change
@router.get("/infrastructure-changes", response_model=list[InfrastructureChangeOut])
def list_infrastructure_changes(
days: int = Query(90, ge=1, le=730),
db: Session = Depends(get_db), user=Depends(get_current_user),
):
cutoff = datetime.utcnow() - timedelta(days=days)
return (
db.query(InfrastructureChangeLog)
.filter(InfrastructureChangeLog.change_date >= cutoff)
.order_by(InfrastructureChangeLog.change_date.desc())
.all()
)
# — Decay Engine Control —
@router.post("/decay-engine/run")
def trigger_decay_engine(
db: Session = Depends(get_db), user=Depends(require_role("admin")),
):
results = decay_engine_service.run_decay_engine(db)
return {"message": "Decay engine completed", "results": results}
# — Dashboard Summary —
@router.get("/dashboard")
def lifecycle_dashboard(
db: Session = Depends(get_db), user=Depends(get_current_user),
):
"""Resumen ejecutivo del estado de detecciones."""
from sqlalchemy import func
health_dist = dict(
db.query(DetectionAsset.health_status, func.count(DetectionAsset.id))
.filter(DetectionAsset.is_active == True)
.group_by(DetectionAsset.health_status)
.all()
)
confidence_dist = dict(
db.query(TechniqueConfidenceScore.confidence_level, func.count(TechniqueConfidenceScore.id))
.group_by(TechniqueConfidenceScore.confidence_level)
.all()
)
expiring_soon = (
db.query(func.count(DetectionValidation.id))
.filter(
DetectionValidation.is_valid == True,
DetectionValidation.expires_at <= (datetime.utcnow() + timedelta(days=7)),
)
.scalar()
)
total_assets = db.query(func.count(DetectionAsset.id)).filter(
DetectionAsset.is_active == True
).scalar()
total_valid = db.query(func.count(DetectionValidation.id)).filter(
DetectionValidation.is_valid == True
).scalar()
recent_changes = db.query(func.count(InfrastructureChangeLog.id)).filter(
InfrastructureChangeLog.change_date >= (datetime.utcnow() - timedelta(days=30))
).scalar()
return {
"total_detection_assets": total_assets,
"total_valid_validations": total_valid,
"health_distribution": {
k.value if hasattr(k, 'value') else str(k): v for k, v in health_dist.items()
},
"confidence_distribution": {
k.value if hasattr(k, 'value') else str(k): v for k, v in confidence_dist.items()
},
"validations_expiring_7d": expiring_soon,
"infrastructure_changes_30d": recent_changes,
}
```
Registrar en main.py:
```python
from app.routers.detection_lifecycle import router as detection_lifecycle_router
app.include_router(detection_lifecycle_router, prefix="/api/v1")
```
**Verificación:**
- `POST /api/v1/detection-lifecycle/assets` con datos válidos → 201, asset creado con hash
- `POST /api/v1/detection-lifecycle/validations` → validación con `expires_at` correcto
- `GET /api/v1/detection-lifecycle/confidence/{technique_id}?recalculate=true` → score calculado
- `POST /api/v1/detection-lifecycle/infrastructure-changes` con `auto_invalidate=true` → validaciones afectadas invalidadas
- `GET /api/v1/detection-lifecycle/dashboard` → resumen completo con distribuciones
- `POST /api/v1/detection-lifecycle/decay-engine/run` → resultados del decay completo
---
### Tarea 8.5: Seed de Decay Policy por Defecto
**Qué:** Crear una política por defecto durante el seed de la aplicación.
**Implementación:**
En `app/seed.py` o el script de seed existente:
```python
from app.models.decay_policy import DecayPolicy
def seed_decay_policies(db: Session):
"""Crear política de decay por defecto si no existe."""
existing = db.query(DecayPolicy).filter(DecayPolicy.is_default == True).first()
if existing:
return
default_policy = DecayPolicy(
name="Default Decay Policy",
description="Standard detection decay: Fresh up to 90 days, Aging 91-180 days, Stale 181-365 days, Broken after 365 days.",
fresh_days=90,
aging_days=180,
stale_days=365,
default_validity_days=180,
silent_threshold_days=30,
noisy_threshold_daily=100,
recency_weight=0.30,
coverage_weight=0.30,
health_weight=0.25,
diversity_weight=0.15,
is_default=True,
is_active=True,
)
db.add(default_policy)
critical_policy = DecayPolicy(
name="Critical Techniques Policy",
description="Stricter policy for high-impact tactics: Fresh 60 days, Aging 90 days, Stale 180 days.",
applies_to_tactic="initial-access",
fresh_days=60,
aging_days=90,
stale_days=180,
default_validity_days=90,
silent_threshold_days=14,
noisy_threshold_daily=50,
recency_weight=0.35,
coverage_weight=0.30,
health_weight=0.25,
diversity_weight=0.10,
is_default=False,
is_active=True,
)
db.add(critical_policy)
db.commit()
```
**Verificación:**
- Ejecutar seed → dos políticas creadas en `decay_policies`
- Ejecutar seed de nuevo → no duplica
- La política de initial-access tiene umbrales más estrictos
---
## FASE 9 — Ownership & Operativa Diaria
**Duración estimada:** 2-3 semanas
**Dependencias:** Fase 8
---
### Tarea 9.1: Sistema de Ownership
**Qué:** Cada técnica y regla de detección tiene un owner responsable, backup owner, y equipo.
**Implementación:**
1. **Migración Alembic** — Añadir columnas a `techniques`:
```python
def upgrade():
op.add_column('techniques', Column('owner_id', UUID(as_uuid=True), ForeignKey("users.id"), nullable=True))
op.add_column('techniques', Column('backup_owner_id', UUID(as_uuid=True), ForeignKey("users.id"), nullable=True))
op.add_column('techniques', Column('team', String(100), nullable=True))
op.add_column('techniques', Column('ownership_assigned_at', DateTime, nullable=True))
op.create_index('ix_techniques_owner_id', 'techniques', ['owner_id'])
```
2. **app/services/ownership_service.py** (nuevo):
```python
import logging
from datetime import datetime
from uuid import UUID
from sqlalchemy.orm import Session
from sqlalchemy import func
from app.models.technique import Technique
from app.models.user import User
logger = logging.getLogger(__name__)
def assign_owner(db: Session, technique_id: UUID, owner_id: UUID,
backup_owner_id: UUID = None, team: str = None):
technique = db.query(Technique).filter(Technique.id == technique_id).first()
if not technique:
from app.domain.exceptions import EntityNotFoundError
raise EntityNotFoundError("Technique", str(technique_id))
technique.owner_id = owner_id
technique.backup_owner_id = backup_owner_id
technique.team = team
technique.ownership_assigned_at = datetime.utcnow()
db.commit()
return technique
def bulk_assign_by_tactic(db: Session, tactic: str, owner_id: UUID, team: str = None):
"""Asignar owner a todas las técnicas de una táctica."""
techniques = db.query(Technique).filter(Technique.tactic == tactic).all()
count = 0
for t in techniques:
if not t.owner_id:
t.owner_id = owner_id
t.team = team
t.ownership_assigned_at = datetime.utcnow()
count += 1
db.commit()
return count
def get_orphan_techniques(db: Session) -> list:
"""Técnicas sin owner asignado."""
return db.query(Technique).filter(Technique.owner_id.is_(None)).order_by(Technique.mitre_id).all()
def get_workload_by_user(db: Session) -> list[dict]:
"""Carga de trabajo por usuario."""
results = (
db.query(User.username, User.role, func.count(Technique.id).label("technique_count"))
.outerjoin(Technique, Technique.owner_id == User.id)
.group_by(User.id)
.all()
)
return [{"username": r[0], "role": r[1], "technique_count": r[2]} for r in results]
```
3. **app/routers/ownership.py** (nuevo) — Endpoints para asignar owners, ver huérfanos, ver carga.
**Verificación:**
- Asignar owner a técnica → `owner_id` y `ownership_assigned_at` se actualizan
- `get_orphan_techniques()` retorna técnicas sin owner
- Bulk assign por táctica asigna solo las que no tienen owner
- Workload muestra conteo correcto por usuario
---
### Tarea 9.2: Cola de Revalidación
**Qué:** Sistema que genera automáticamente una cola priorizada de técnicas/detecciones que necesitan revalidación.
**Implementación:**
**app/models/revalidation_queue.py** (nuevo):
```python
import uuid
import enum
from datetime import datetime
from sqlalchemy import (
Column, String, Integer, DateTime, ForeignKey,
Text, Enum as SQLEnum, Boolean
)
from sqlalchemy.dialects.postgresql import UUID, JSONB
from app.database import Base
class RevalidationPriority(str, enum.Enum):
critical = "critical"
high = "high"
medium = "medium"
low = "low"
class RevalidationStatus(str, enum.Enum):
pending = "pending"
assigned = "assigned"
in_progress = "in_progress"
completed = "completed"
skipped = "skipped"
class RevalidationItem(Base):
__tablename__ = "revalidation_queue"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
technique_id = Column(UUID(as_uuid=True), ForeignKey("techniques.id"), nullable=False)
detection_asset_id = Column(UUID(as_uuid=True), ForeignKey("detection_assets.id"), nullable=True)
reason = Column(String(200), nullable=False)
priority = Column(SQLEnum(RevalidationPriority), default=RevalidationPriority.medium)
status = Column(SQLEnum(RevalidationStatus), default=RevalidationStatus.pending)
assigned_to = Column(UUID(as_uuid=True), ForeignKey("users.id"))
assigned_at = Column(DateTime)
due_date = Column(DateTime)
completed_at = Column(DateTime)
completed_by = Column(UUID(as_uuid=True), ForeignKey("users.id"))
result_validation_id = Column(UUID(as_uuid=True), ForeignKey("detection_validations.id"))
notes = Column(Text)
metadata = Column(JSONB, default={})
created_at = Column(DateTime, default=datetime.utcnow)
```
**app/services/revalidation_service.py** — Genera cola automáticamente basándose en: validaciones expiradas, cambios de infra, coverage stale, updates MITRE. Prioriza por criticidad de la táctica y risk score.
**Verificación:**
- Ejecutar generación de cola → items creados con prioridad correcta
- Asignar item a usuario → status cambia a "assigned"
- Completar item con nueva validación → status "completed" + `result_validation_id` poblado
- Cola respeta priorización (critical > high > medium > low)
---
### Tarea 9.3: Dashboard del Analista — API
**Qué:** Endpoint que retorna la vista personalizada del día para cada analista.
**Implementación:**
**app/routers/analyst_dashboard.py** (nuevo):
```python
@router.get("/my-dashboard")
def get_my_dashboard(
db: Session = Depends(get_db), user=Depends(get_current_user),
):
"""Vista personalizada del día para el analista logueado."""
now = datetime.utcnow()
# 1. Mis revalidaciones pendientes
my_revalidations = (
db.query(RevalidationItem)
.filter(
RevalidationItem.assigned_to == user.id,
RevalidationItem.status.in_(["pending", "assigned", "in_progress"]),
)
.order_by(RevalidationItem.priority.asc(), RevalidationItem.due_date.asc())
.limit(10)
.all()
)
# 2. Mis técnicas con validaciones que caducan esta semana
my_expiring = (
db.query(DetectionValidation)
.join(DetectionTechniqueMapping,
DetectionValidation.detection_asset_id == DetectionTechniqueMapping.detection_asset_id)
.join(Technique, DetectionTechniqueMapping.technique_id == Technique.id)
.filter(
Technique.owner_id == user.id,
DetectionValidation.is_valid == True,
DetectionValidation.expires_at <= (now + timedelta(days=7)),
)
.all()
)
# 3. Mis tests pendientes de workflow
my_tests = (
db.query(Test)
.filter(
Test.created_by == user.id,
Test.state.in_(["draft", "red_executing", "blue_evaluating"])
)
.order_by(Test.updated_at.desc())
.limit(10)
.all()
)
# 4. Notificaciones no leídas
unread_notifications = (
db.query(func.count(Notification.id))
.filter(Notification.user_id == user.id, Notification.is_read == False)
.scalar()
)
# 5. Cambios de infra recientes
recent_changes = (
db.query(InfrastructureChangeLog)
.filter(InfrastructureChangeLog.change_date >= (now - timedelta(days=7)))
.order_by(InfrastructureChangeLog.change_date.desc())
.limit(5)
.all()
)
return {
"user": {"username": user.username, "role": user.role},
"date": now.isoformat(),
"revalidations_pending": len(my_revalidations),
"revalidations": [_serialize_revalidation(r) for r in my_revalidations],
"expiring_validations": len(my_expiring),
"active_tests": len(my_tests),
"unread_notifications": unread_notifications,
"recent_infrastructure_changes": len(recent_changes),
}
```
**Verificación:**
- `GET /api/v1/analyst/my-dashboard` retorna datos personalizados del usuario logueado
- Revalidaciones aparecen ordenadas por prioridad
- Validaciones expirando muestran solo las del owner logueado
- Dashboard funciona para todos los roles
---
## FASE 10 — Attack Paths & Purple Team Avanzado
**Duración estimada:** 3-4 semanas
**Dependencias:** Fases 8, 9
---
### Tarea 10.1: Modelo de Attack Paths
**Qué:** Permitir definir escenarios de ataque encadenados (Initial Access → Execution → Persistence → Lateral Movement → Exfiltration) como entidades de primer nivel.
**Implementación:**
**app/models/attack_path.py** (nuevo):
```python
import uuid
import enum
from datetime import datetime
from sqlalchemy import (
Column, String, Integer, Float, DateTime,
ForeignKey, Text, Boolean, Enum as SQLEnum
)
from sqlalchemy.dialects.postgresql import UUID, JSONB
from sqlalchemy.orm import relationship
from app.database import Base
class AttackPathStatus(str, enum.Enum):
draft = "draft"
ready = "ready"
executing = "executing"
completed = "completed"
archived = "archived"
class AttackPath(Base):
"""Escenario de ataque encadenado multi-fase."""
__tablename__ = "attack_paths"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name = Column(String(300), nullable=False)
description = Column(Text)
threat_scenario = Column(Text)
status = Column(SQLEnum(AttackPathStatus), default=AttackPathStatus.draft)
threat_actor_id = Column(UUID(as_uuid=True), ForeignKey("threat_actors.id"))
campaign_id = Column(UUID(as_uuid=True), ForeignKey("campaigns.id"))
# Resultado
detection_rate = Column(Float)
mean_time_to_detect = Column(Float)
furthest_step_reached = Column(Integer, default=0)
complexity = Column(String(20), default="medium")
target_environment = Column(String(200))
tags = Column(JSONB, default=[])
metadata = Column(JSONB, default={})
created_by = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
started_at = Column(DateTime)
completed_at = Column(DateTime)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
steps = relationship("AttackPathStep", back_populates="attack_path",
order_by="AttackPathStep.step_order")
class AttackPathStep(Base):
"""Un paso individual dentro de un attack path."""
__tablename__ = "attack_path_steps"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
attack_path_id = Column(UUID(as_uuid=True),
ForeignKey("attack_paths.id", ondelete="CASCADE"), nullable=False)
step_order = Column(Integer, nullable=False)
name = Column(String(300), nullable=False)
description = Column(Text)
tactic = Column(String(100))
technique_id = Column(UUID(as_uuid=True), ForeignKey("techniques.id"))
test_id = Column(UUID(as_uuid=True), ForeignKey("tests.id"))
procedure = Column(Text)
tools = Column(JSONB, default=[])
expected_detection = Column(Text)
# Resultado
executed_at = Column(DateTime)
detected = Column(Boolean)
detection_time_seconds = Column(Integer)
detection_details = Column(Text)
red_notes = Column(Text)
blue_notes = Column(Text)
events = Column(JSONB, default=[])
metadata = Column(JSONB, default={})
attack_path = relationship("AttackPath", back_populates="steps")
```
Services y routers: CRUD completo + endpoint de ejecución colaborativa + auto-generación de informe del ejercicio.
**Verificación:**
- Crear attack path con 5 pasos → pasos ordenados correctamente
- Ejecutar paso a paso → `detection_rate` se recalcula automáticamente
- `furthest_step_reached` refleja hasta dónde llegó sin detección
- Completar path → genera timeline completa en `events`
---
### Tarea 10.2: Timeline Colaborativa
**Qué:** Registro en tiempo real de acciones Red/Blue durante un ejercicio, con timestamps para medir tiempos de detección y respuesta.
Este modelo ya existe parcialmente con los `events` JSONB en `AttackPathStep`. El servicio encapsula la lógica de agregar eventos y calcular métricas temporales.
---
## FASE 11 — Knowledge Management
**Duración estimada:** 2-3 semanas
**Dependencias:** Fase 8
---
### Tarea 11.1: Modelo de Playbooks
**Qué:** Cada técnica puede tener playbooks asociados: cómo atacar, cómo detectar, cómo investigar, cómo responder.
**Implementación:**
**app/models/playbook.py** (nuevo):
```python
import uuid
import enum
from datetime import datetime
from sqlalchemy import (
Column, String, Integer, DateTime, ForeignKey,
Text, Boolean, Enum as SQLEnum
)
from sqlalchemy.dialects.postgresql import UUID, JSONB
from app.database import Base
class PlaybookType(str, enum.Enum):
attack = "attack"
detect = "detect"
investigate = "investigate"
respond = "respond"
hunt = "hunt"
class Playbook(Base):
__tablename__ = "playbooks"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
technique_id = Column(UUID(as_uuid=True), ForeignKey("techniques.id"),
nullable=False, index=True)
playbook_type = Column(SQLEnum(PlaybookType), nullable=False)
title = Column(String(300), nullable=False)
content_markdown = Column(Text, nullable=False)
version = Column(Integer, default=1)
is_published = Column(Boolean, default=False)
last_reviewed_at = Column(DateTime)
last_reviewed_by = Column(UUID(as_uuid=True), ForeignKey("users.id"))
difficulty = Column(String(20))
estimated_time_minutes = Column(Integer)
tools_required = Column(JSONB, default=[])
prerequisites = Column(JSONB, default=[])
references = Column(JSONB, default=[])
tags = Column(JSONB, default=[])
created_by = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class LessonLearned(Base):
"""Registro inmutable de lecciones aprendidas."""
__tablename__ = "lessons_learned"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
technique_id = Column(UUID(as_uuid=True), ForeignKey("techniques.id"), index=True)
test_id = Column(UUID(as_uuid=True), ForeignKey("tests.id"))
attack_path_id = Column(UUID(as_uuid=True), ForeignKey("attack_paths.id"))
campaign_id = Column(UUID(as_uuid=True), ForeignKey("campaigns.id"))
title = Column(String(300), nullable=False)
what_happened = Column(Text, nullable=False)
what_failed = Column(Text)
root_cause = Column(Text)
how_fixed = Column(Text)
recommendations = Column(Text)
impact = Column(String(20))
tags = Column(JSONB, default=[])
created_by = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
created_at = Column(DateTime, default=datetime.utcnow)
```
Routers y servicios: CRUD con soporte Markdown, búsqueda fulltext, versionado.
**Verificación:**
- Crear playbook tipo "detect" para T1059 → asociado correctamente
- Editar contenido → `version` incrementa, contenido anterior preservado en audit log
- Buscar playbooks por técnica retorna todos los tipos
- Lessons learned vinculadas a test/campaign correctamente
---
## FASE 12 — Risk Intelligence & Recomendaciones
**Duración estimada:** 2-3 semanas
**Dependencias:** Fases 4 (OsintItem), 8, 9
---
### Tarea 12.1: Risk Score Compuesto por Técnica
**Qué:** Score de riesgo operativo real que combina: explotabilidad, frecuencia en amenazas reales, tiempo sin validar, cobertura de detección, y severidad.
**Implementación:**
**app/services/risk_scoring_service.py** (nuevo):
```python
def calculate_technique_risk(db: Session, technique_id: UUID) -> dict:
"""Calcula un risk score multidimensional por técnica."""
technique = db.query(Technique).filter(Technique.id == technique_id).first()
if not technique:
return None
# 1. EXPLOITABILITY (0-100)
templates = db.query(func.count(TestTemplate.id)).filter(
TestTemplate.technique_id == technique_id
).scalar() or 0
osint_count = db.query(func.count(OsintItem.id)).filter(
OsintItem.technique_id == technique_id
).scalar() or 0
exploitability = min(100, templates * 15 + osint_count * 10)
# 2. THREAT_FREQUENCY (0-100)
actor_count = db.query(func.count(ThreatActorTechnique.id)).filter(
ThreatActorTechnique.technique_id == technique_id
).scalar() or 0
threat_frequency = min(100, actor_count * 20)
# 3. DETECTION_GAP (0-100): inverso del confidence score
confidence = db.query(TechniqueConfidenceScore).filter(
TechniqueConfidenceScore.technique_id == technique_id
).first()
confidence_score = confidence.confidence_score if confidence else 0
detection_gap = 100 - confidence_score
# 4. STALENESS (0-100)
last_validated = confidence.last_validated_at if confidence else None
if not last_validated:
staleness = 100
else:
days = (datetime.utcnow() - last_validated).days
staleness = min(100, days / 3.65)
# 5. SEVERITY (0-100): basado en táctica
tactic_severity = {
"initial-access": 90, "execution": 85,
"persistence": 80, "privilege-escalation": 85,
"defense-evasion": 90, "credential-access": 85,
"discovery": 50, "lateral-movement": 80,
"collection": 60, "command-and-control": 75,
"exfiltration": 90, "impact": 95,
"resource-development": 40, "reconnaissance": 35,
}
severity = tactic_severity.get(technique.tactic, 50)
# RISK SCORE COMPUESTO
risk_score = (
exploitability * 0.15 +
threat_frequency * 0.25 +
detection_gap * 0.25 +
staleness * 0.15 +
severity * 0.20
)
return {
"technique_id": str(technique_id),
"mitre_id": technique.mitre_id,
"risk_score": round(risk_score, 1),
"risk_level": (
"critical" if risk_score >= 80 else
"high" if risk_score >= 60 else
"medium" if risk_score >= 40 else "low"
),
"factors": {
"exploitability": round(exploitability, 1),
"threat_frequency": round(threat_frequency, 1),
"detection_gap": round(detection_gap, 1),
"staleness": round(staleness, 1),
"severity": severity,
},
}
```
---
### Tarea 12.2: Motor de Recomendaciones
**Qué:** Sistema que sugiere automáticamente acciones prioritarias.
**Categorías de recomendaciones:**
- Técnicas de alto riesgo sin cobertura
- Técnicas nunca probadas usadas por actores relevantes
- Detecciones silenciosas que necesitan verificación
- Reglas huérfanas sin owner
- Gaps de cobertura por táctica
- Técnicas con muchos CVEs recientes sin validación
El servicio consulta múltiples fuentes de datos y genera una lista priorizada de recomendaciones con acciones sugeridas.
---
## FASE 13 — Alertas Inteligentes Operativas
**Duración estimada:** 2-3 semanas
**Dependencias:** Fases 6 (webhooks como canal), 8, 9, 12
---
### Tarea 13.1: Sistema de Alertas Operativas
**Qué:** Alertas basadas en reglas configurables que detectan condiciones operativas y notifican proactivamente.
**Modelo:**
```python
class OperationalAlertRule(Base):
__tablename__ = "operational_alert_rules"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name = Column(String(300), nullable=False)
description = Column(Text)
rule_type = Column(String(100), nullable=False)
condition = Column(JSONB, nullable=False)
severity = Column(String(20), default="medium")
is_active = Column(Boolean, default=True)
notification_channels = Column(JSONB, default=["in_app"])
# ["in_app", "email", "webhook"] ← usa Fase 6 webhooks y Fase 7 email
last_triggered_at = Column(DateTime)
trigger_count = Column(Integer, default=0)
created_at = Column(DateTime, default=datetime.utcnow)
```
**Job evaluador** — Ejecuta cada hora, evalúa todas las reglas activas, genera alertas cuando se cumplen condiciones.
**Reglas pre-configuradas (seed):**
| Regla | Condición | Severidad |
|-------|-----------|-----------|
| 3+ técnicas críticas llevan 90 días sin validarse | `stale_critical_techniques: threshold_days=90, min=3` | critical |
| EDR actualizado — revalidaciones pendientes | `edr_update_pending_revalidation` | high |
| Nueva técnica MITRE sin cobertura | `new_mitre_techniques_uncovered` | medium |
| Regresión de cobertura detectada | `coverage_regression: drop_pct=5` | high |
| Ola de validaciones expirando esta semana | `validation_expiry_wave: days=7, min=10` | medium |
---
## FASE 14 — Enterprise Readiness (SSO + API Keys)
**Duración estimada:** 2 semanas
**Dependencias:** Fase 0
---
### Tarea 14.1: API Key Management
**Qué:** Permitir crear API keys para integraciones automatizadas (PowerBI, SOAR, scripts) sin usar JWT de usuario.
**Modelo:**
```python
class ApiKey(Base):
__tablename__ = "api_keys"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name = Column(String(200), nullable=False)
key_hash = Column(String(64), nullable=False, unique=True) # SHA256
key_prefix = Column(String(8)) # Primeros 8 chars para identificación visual
user_id = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
scopes = Column(JSONB, default=["read"])
is_active = Column(Boolean, default=True)
last_used_at = Column(DateTime)
expires_at = Column(DateTime)
created_at = Column(DateTime, default=datetime.utcnow)
```
**Flujo:** Al crear → generar key aleatoria → mostrar UNA vez al usuario → guardar solo hash. Autenticar via header `X-API-Key`.
---
### Tarea 14.2: SSO/SAML Integration
**Qué:** Soporte para Single Sign-On vía SAML 2.0 usando `python3-saml`.
Configuración en `app/config.py` con `SAML_ENABLED`, `SAML_IDP_METADATA_URL`, etc. Endpoint `/auth/saml/login` y `/auth/saml/callback`.
---
## Features Adicionales Recomendadas
Estas no estaban en los requisitos originales pero beneficiarían la plataforma:
| # | Feature | Razón | Fase sugerida |
|---|---------|-------|---------------|
| A1 | Dashboard personalizable por rol | CISO ve métricas ejecutivas, Red Tech ve sus tests pendientes | Fase 5 |
| A2 | Import/Export de ATT&CK Navigator layers | Los equipos ya usan Navigator | Fase 2 |
| A3 | Approval workflow para cambios de scoring weights | Evitar que un admin cambie pesos sin supervisión | Fase 3 |
| A4 | Tags y campos custom | Cada organización tiene taxonomías propias | Fase 5 |
| A5 | Bulk operations | Validar/rechazar múltiples tests a la vez, asignar campaña masiva | Fases 5 y 9 |
| A6 | Markdown support en descripciones | Los técnicos quieren formatear procedimientos | Fase 11 |
| A7 | Detection Rule Git Sync | Sincronizar reglas desde repo Git corporativo | Fase 8 |
| A8 | Coverage Heatmap con Confidence overlay | Heatmap muestra estado + confidence como segunda capa | Fase 8 |
| A9 | Automatic Detection Gap → Ticket pipeline | RT rompe algo → auto-crear item en cola → asignar a Blue | Fase 10 |
| A10 | Export/Import ATT&CK Navigator con Confidence | Exportar layer que incluye confidence level | Fase 8 |
| A11 | Comparative Attack Path Results | Comparar mismo path ejecutado en diferentes fechas | Fase 10 |
| A12 | SLA Tracking para Detection Gaps | Medir tiempo desde gap hasta implementación de regla | Fase 13 |
---
## Checklist de Nuevas Dependencias Python
```
# Fase 0
redis>=5.0.0
ruff # dev
pytest # dev
# Fase 1
atlassian-python-api>=4.0.0
tempo-api-python-client>=0.8.0
# Fase 2
weasyprint>=62.0
docxtpl>=0.18.0
# Fase 7
# smtplib es stdlib — no necesita instalar
# Fase 11
markdown>=3.6
Pygments>=2.18
# Fase 14
python3-saml>=1.16.0
```
---
## Checklist de Verificación Global por Fase
| Fase | Tests Mínimos Requeridos |
|------|-------------------------|
| 0 | Redis ping, token blacklist persistencia, índices en EXPLAIN, excepciones de dominio mapean a HTTP, CI pasa |
| 1 | Jira link CRUD, sync bidireccional, worklog integridad hash, Tempo mock |
| 2 | PDF genera con datos reales, DOCX abre en Word, analytics endpoints retornan JSON plano |
| 3 | Audit log con IP/hash, login attempts registrados, password validation, rate limiting 429 |
| 4 | CVEs importados sin duplicados, stale detection marca técnicas, rate limit NVD respetado |
| 5 | Scoring con recencia funciona, pesos persisten en BD, snapshots con desglose por táctica |
| 6 | Webhook recibe POST con signature HMAC válida, failure_count incrementa |
| 7 | Email enviado con SMTP, skip silencioso sin SMTP, preferencias por usuario |
| 8 | Assets CRUD, hash auto-generation, decay engine run, confidence calculation, infrastructure change invalidation |
| 9 | Ownership assignment, revalidation queue generation, analyst dashboard data correctness |
| 10 | Attack path creation, step execution, timeline recording, detection rate calculation |
| 11 | Playbook CRUD, Markdown rendering, lessons learned linking |
| 12 | Risk score calculation, recommendation generation, ranking correctness |
| 13 | Alert rule evaluation, trigger conditions, notification dispatch via múltiples canales |
| 14 | API key auth flow, SAML login flow |
---
## Instrucciones para Claude Code
Para cada tarea, Claude Code debe seguir este flujo:
1. Leer el contexto de la tarea y los archivos existentes relevantes
2. Crear los archivos nuevos indicados
3. Modificar los archivos existentes según las instrucciones
4. Crear migración Alembic si hay cambios de BD
5. Ejecutar `alembic upgrade head` para verificar migración
6. Ejecutar `pytest tests/` para verificar que tests existentes no se rompen
7. Crear tests nuevos para la funcionalidad añadida
8. Verificar según los criterios de verificación de la tarea
9. Commit con mensaje descriptivo: `feat(detection-lifecycle): add decay engine service [FASE-8.3]`
**Importante:** Cada tarea debe funcionar independientemente antes de pasar a la siguiente. No avanzar si la verificación no pasa.
Reference in New Issue View Git Blame Copy Permalink