docs: enterprise refactor plan with ralph specs
This commit is contained in:
118
.ralph/specs/legacy/cli-cicd.md
Normal file
118
.ralph/specs/legacy/cli-cicd.md
Normal file
@@ -0,0 +1,118 @@
|
||||
# ABE — CLI & CI/CD Integration Specification
|
||||
|
||||
## CLI Entry Point
|
||||
|
||||
File: `src/cli.ts`
|
||||
Script in package.json: `"abe": "ts-node src/cli.ts"`
|
||||
Global after install: `npx abe` or `abe` if installed globally.
|
||||
|
||||
## CLI Usage
|
||||
```bash
|
||||
# Basic run
|
||||
abe run --url http://localhost:3000
|
||||
|
||||
# With auth
|
||||
abe run --url http://app.com \
|
||||
--auth-type login_flow \
|
||||
--login-url http://app.com/login \
|
||||
--username test@app.com \
|
||||
--password secret
|
||||
|
||||
# With scope limits
|
||||
abe run --url http://app.com \
|
||||
--max-states 30 \
|
||||
--max-depth 4 \
|
||||
--allowed-domains app.com
|
||||
|
||||
# CI mode: exit 1 if any anomaly found
|
||||
abe run --url http://localhost:3000 --fail-on-anomaly
|
||||
|
||||
# CI mode: exit 1 only on high/critical anomalies
|
||||
abe run --url http://localhost:3000 --fail-on-severity high
|
||||
|
||||
# Output formats
|
||||
abe run --url http://localhost:3000 --output json # prints JSON summary to stdout
|
||||
abe run --url http://localhost:3000 --output junit # generates junit.xml for CI
|
||||
|
||||
# Connect to a running ABE server instead of running inline
|
||||
abe run --url http://localhost:3000 --server http://abe-server:3001 --api-key mykey
|
||||
```
|
||||
|
||||
## Exit Codes
|
||||
|
||||
- 0 → exploration complete, no anomalies (or no anomalies above threshold)
|
||||
- 1 → anomalies found above threshold
|
||||
- 2 → exploration failed (target unreachable, auth failed, etc.)
|
||||
|
||||
## stdout JSON output (--output json)
|
||||
```json
|
||||
{
|
||||
"sessionId": "sess_abc123",
|
||||
"url": "http://localhost:3000",
|
||||
"duration_ms": 45000,
|
||||
"states_visited": 12,
|
||||
"anomalies": [
|
||||
{
|
||||
"id": "anom_xyz",
|
||||
"type": "http_error",
|
||||
"severity": "high",
|
||||
"description": "Form returns 500 on empty email",
|
||||
"report_path": "reports/anom_xyz/report.json"
|
||||
}
|
||||
],
|
||||
"exit_code": 1
|
||||
}
|
||||
```
|
||||
|
||||
## JUnit XML output (--output junit)
|
||||
|
||||
Generates `abe-results.xml` compatible with Jenkins, GitHub Actions, GitLab CI:
|
||||
- Each anomaly = one failing test case
|
||||
- Each explored state = one passing test case
|
||||
|
||||
## GitHub Actions Example Workflow
|
||||
|
||||
Create file: `.github/workflows/abe-example.yml` in the repo:
|
||||
```yaml
|
||||
name: ABE Exploratory Testing
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
explore:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Start application
|
||||
run: docker-compose up -d app
|
||||
# assumes the project has a docker-compose with the target app
|
||||
|
||||
- name: Wait for app
|
||||
run: npx wait-on http://localhost:3000 --timeout 30000
|
||||
|
||||
- name: Run ABE
|
||||
run: |
|
||||
npm install -g abe-explorer # or: npx abe
|
||||
abe run \
|
||||
--url http://localhost:3000 \
|
||||
--max-states 30 \
|
||||
--fail-on-severity high \
|
||||
--output junit
|
||||
|
||||
- name: Upload results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: abe-reports
|
||||
path: reports/
|
||||
|
||||
- name: Publish test results
|
||||
if: always()
|
||||
uses: EnricoMi/publish-unit-test-result-action@v2
|
||||
with:
|
||||
files: abe-results.xml
|
||||
```
|
||||
Reference in New Issue
Block a user