67 lines
1.9 KiB
Markdown
67 lines
1.9 KiB
Markdown
# Phase 8: Job Queue System
|
|
|
|
## Tabla jobs (SQLite)
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS jobs (
|
|
id TEXT PRIMARY KEY,
|
|
type TEXT NOT NULL,
|
|
status TEXT NOT NULL DEFAULT 'pending',
|
|
payload TEXT NOT NULL,
|
|
result TEXT,
|
|
error TEXT,
|
|
attempts INTEGER NOT NULL DEFAULT 0,
|
|
max_attempts INTEGER NOT NULL DEFAULT 3,
|
|
priority INTEGER NOT NULL DEFAULT 0,
|
|
run_at TEXT NOT NULL,
|
|
started_at TEXT,
|
|
completed_at TEXT,
|
|
created_at TEXT NOT NULL,
|
|
updated_at TEXT NOT NULL
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_jobs_poll ON jobs(status, run_at, priority DESC);
|
|
```
|
|
|
|
## Interface
|
|
```typescript
|
|
export interface IJobQueue {
|
|
enqueue<T>(type: string, payload: T, opts?: { runAt?: Date; priority?: number; maxAttempts?: number }): Promise<string>;
|
|
start(): void;
|
|
pause(): void;
|
|
waitForActive(timeoutMs: number): Promise<void>;
|
|
}
|
|
```
|
|
|
|
## Polling logic
|
|
```
|
|
loop (cada pollIntervalMs):
|
|
SELECT id, type, payload FROM jobs
|
|
WHERE status = 'pending' AND run_at <= datetime('now')
|
|
ORDER BY priority DESC, created_at ASC
|
|
LIMIT 1
|
|
|
|
if found:
|
|
UPDATE jobs SET status = 'running', started_at = now, attempts = attempts + 1
|
|
WHERE id = ? AND status = 'pending' // optimistic lock
|
|
|
|
if updated 0 rows → skip (otro worker lo tomó)
|
|
|
|
try:
|
|
result = await executeJob(type, payload)
|
|
UPDATE jobs SET status = 'completed', result = ?, completed_at = now
|
|
catch:
|
|
if attempts >= max_attempts:
|
|
UPDATE jobs SET status = 'failed', error = ?
|
|
else:
|
|
backoff = min(1000 * 2^attempts, 60000)
|
|
UPDATE jobs SET status = 'pending', run_at = now + backoff, error = ?
|
|
```
|
|
|
|
## Job types
|
|
- `exploration:run` — payload: { sessionId, config }
|
|
- `report:generate` — payload: { reportId, format, filters }
|
|
- `cleanup:old-data` — payload: { retentionDays }
|
|
|
|
## NO usar Redis
|
|
El job queue es SQLite-based para zero-dependency self-hosted.
|
|
Es simple, funciona para el volumen esperado (decenas de jobs, no miles).
|