HomeLab/ForgeBucket
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

added ai prompt to gitignore file

Browse Source
This commit is contained in:
erangel1
2026-05-13 00:56:41 +02:00
parent 77268e2302
commit 994570ca74
4 changed files with 135 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+3 -1
View File
@@ -12,4 +12,6 @@ cache
uploads
# Database
*.db
*.db
ai_agent_master_prompt_for_building_modern_git_platform.md
AGENTS.md
+29 -4
View File
@@ -36,6 +36,20 @@ internal/
dag.go — topological sort, ReadyJobs
parser.go — .forgebucket/workflows/*.yml parser
types.go — WorkflowFile, WorkflowJob, WorkflowStep structs
sbom/ — SBOM generator (fully built — Phase 4)
generator.go — CycloneDX 1.4 generation, auto on pipeline success + on-demand
cyclonedx.go — CycloneDX document model and helpers
parsers.go — Manifest parsers: go.mod, package.json, requirements.txt, Cargo.toml, Gemfile.lock, pom.xml
scanning/ — Secret scanner (fully built — Phase 4)
scanner.go — Push-triggered regex scanning, listing, dismissal
secrets.go — 15 high/medium severity secret patterns (AWS, GitHub, SSH, JWT, etc.)
vulnscan/ — Vulnerability scanner (fully built — Phase 4)
scanner.go — OSV API-backed dependency vulnerability scanning
osv.go — HTTP client for api.osv.dev/v1
signing/ — Artifact signing (fully built — Phase 4)
keystore.go — ECDSA P-256 signing and verification, self-verifying bundles
oci/ — OCI registry (fully built — Phase 4)
registry.go — Content-addressable blob store, upload sessions, OCI Distribution Spec v1.1
gitops/ — GitOps controller (fully built — Phase 3D)
controller.go — NATS subscriptions, startup, periodic ticker
drift.go — CheckDrift, handlePush, periodicCheck
@@ -49,7 +63,7 @@ internal/
observability/ — Prometheus metrics + health (fully built — Phase 3E)
metrics.go — metric definitions, HTTP middleware, NATS watcher
health.go — Check() returning HealthStatus (DB ping + NATS)
models/ — XORM structs + 14 migration files
models/ — XORM structs + 20 migration files
config/ — ENV-driven config, fails fast on missing secrets
events/ — NATS EventBus interface + NATSBus + NoOpBus (Healthy() bool)
web/ — //go:embed target for the built React SPA
@@ -84,7 +98,8 @@ Logger → RealIP → Recoverer → Metrics → CORS → CSRF → SessionAuth
| 3D | GitOps controller + drift detection + auto-sync | **Complete** |
| 3E | Observability (Prometheus `/metrics`, structured `/health`, repo health API) | **Complete** |
| 3F | Federation handlers (ActivityPub WebFinger, actor, inbox/outbox, HTTP signatures, Follow/Accept) | **Complete** |
| 4 | AI diagnostics, signed artifacts, OCI registry, dep/secret scanning | Planned |
| 4 | SBOM generation, secret scanning, vuln scanning, signed artifacts, OCI registry, security page | **Complete** |
| 5 | AI diagnostics, deployment promotions, rollback visualization | Planned |
---
@@ -111,7 +126,7 @@ This rule is non-negotiable. It prevents command injection.
### Database
- XORM for all DB access. Structs in `internal/models/`.
- Migrations are numbered files in `internal/models/migrations/`. Always add a new file; never edit existing ones. Current highest: **014**.
- Migrations are numbered files in `internal/models/migrations/`. Always add a new file; never edit existing ones. Current highest: **020**.
- No raw SQL strings built from user input.
### Events
@@ -218,6 +233,16 @@ make lint # go vet + ESLint
| `internal/domain/federation/remote.go` | FetchActor (cached remote actors), DeliverActivity |
| `internal/api/handlers/secret.go` | Scoped secret management |
| `internal/api/handlers/workspace.go` | Workspace + member management |
| `internal/api/handlers/secret.go` | Scoped secret management |
| `internal/api/handlers/sbom.go` | SBOM generation + download endpoints |
| `internal/api/handlers/scanning.go` | Secret leak list + dismiss endpoints |
| `internal/api/handlers/vulnscan.go` | Vulnerability scan + dismiss endpoints |
| `internal/api/handlers/oci.go` | OCI Distribution Spec v1.1 registry handler |
| `internal/domain/sbom/generator.go` | SBOM generator (CycloneDX 1.4) |
| `internal/domain/scanning/scanner.go` | Push-triggered secret scanner |
| `internal/domain/vulnscan/scanner.go` | OSV API-backed vulnerability scanner |
| `internal/domain/signing/keystore.go` | ECDSA P-256 artifact signing |
| `internal/domain/oci/registry.go` | Content-addressable OCI blob store |
| `internal/api/middleware/audit.go` | Audit log middleware |
| `frontend/src/ui/tokens.ts` | Design token source of truth |
| `frontend/src/components/AppShell.tsx` | Root layout wrapper |
@@ -232,7 +257,7 @@ make lint # go vet + ESLint
```bash
cp .env.example .env # fill SESSION_SECRET and CSRF_SECRET
make docker-up # PostgreSQL + NATS via Docker Compose
make migrate # run XORM migrations (currently 014)
make migrate # run XORM migrations (currently 020)
make dev # Go :8080 + Vite :5173
```
CHANGELOG.md
+73 -32
View File
@@ -9,41 +9,81 @@ Versions follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Planned — Phase 4 ( Artifacts + Git HTTP(S)/SSH Support + Releases Page)
### Planned — Phase 5 (AI Diagnostics + Deployment Promotions + Rollback Visualization)
- AI-powered pipeline failure diagnostics
- Deployment promotion workflows (manual + automated)
- Rollback visualization and timeline
### 4A. Artifacts
- Signed artifacts (Sigstore/Cosign)
- SBOM generation (CycloneDX/SPDX)
- OCI container registry
- Secret scanning (commit-level pattern detection)
- Dependency vulnerability scanning
- Cross-instance pull requests (ForgeFed ActivityPub extension)
---
### 4B. Git HTTP(S)/SSH Support
## [1.0.0] — 2026-05-13
### 4C. Releases Page
- Goal:
- Make releases operationally meaningful.
- Build:
- releases
- release notes
- release assets
- changelog generation
- release timelines
- release channels
- Add:
- signed artifacts
- rollback metadata
- deployment associations
- UI:
- Release page should show:
- version
- artifacts
- deployments
- timeline
- health
Phase 4 complete. SBOM generation, secret scanning, dependency vulnerability scanning, signed artifacts, and OCI registry are operational.
### Added — SBOM Generation (`internal/domain/sbom/`)
- **`Generator`** — subscribes to `pipeline.completed` events and auto-generates CycloneDX 1.4 SBOM documents for every successful pipeline run; also supports on-demand generation via `GenerateOnDemand`
- **6 manifest parsers**: `go.mod`, `package.json`, `requirements.txt`, `Cargo.toml`, `Gemfile.lock`, `pom.xml` — lightweight line-scanning, no external parser dependencies
- **API endpoints** — `GET /sbom`, `GET /sbom/document`, `GET /runs/{runID}/sbom`, `GET /runs/{runID}/sbom/document`, `POST /sbom/generate?ref=&runID=`
- **Database** — migration `016_sbom` adds `SBOMReport` model with CycloneDX document body
- Automatic generation on pipeline completion now also fires directly from the orchestrator (not solely via NATS), ensuring SBOMs are generated even when NATS is unavailable
### Added — Secret Scanning (`internal/domain/scanning/`)
- **`Scanner`** — subscribes to `push.received` events, scans git diffs against 15 regex patterns for high/medium severity secrets
- **Secret patterns**: AWS keys, GitHub/GitLab tokens, generic API keys, Bearer tokens, Slack tokens, Google API keys, Google service accounts, SSH private keys, JWTs, NPM tokens, PostgreSQL/Redis connection strings, generic passwords
- **API endpoints** — `GET /secrets/leaks`, `POST /secrets/leaks/{leakID}/dismiss` (repo-scoped), `GET /api/v1/secrets/leaks` (global admin)
- **Database** — migration `018_scanning` adds `SecretLeak` model
### Added — Vulnerability Scanning (`internal/domain/vulnscan/`)
- **`Scanner`** — triggers on-demand scans against the OSV API (`api.osv.dev/v1`); supports scanning by PURL or by fetching the latest SBOM and scanning all components
- **OSV client** — HTTP client with 30-second timeout, queries OSV database for CVEs by PURL or ecosystem+name, extracts CVSS scores and fixed version ranges
- **API endpoints** — `GET /vulnerabilities`, `POST /vulnerabilities/scan`, `POST /vulnerabilities/{findingID}/dismiss` (repo-scoped), `GET /api/v1/vulnerabilities` (global admin)
- **Database** — migration `019_vulnscan` adds `VulnerabilityFinding` model
- Findings deduplicated by `(vuln_id, purl, repo_id)`
### Added — Artifact Signing (`internal/domain/signing/`)
- **`KeyStore`** — ECDSA P-256 signing and verification; produces self-verifying `Bundle` carrying payload, signature (ASN.1 DER), and public key PEM
- `Sign(artifactID, name, rawContent)` — computes SHA-256 digest, signs, returns signed `Bundle` with key ID fingerprint
- `Verify(bundleJSON)` — extracts public key from bundle, verifies ECDSA signature, returns `VerifyResult` with key-matching check
- `Generate()` — creates ephemeral ECDSA P-256 key when `ARTIFACT_SIGNING_KEY` env var is unset (logs warning; signatures lost on restart unless persisted)
- **API endpoints** — `GET /artifacts/{artifactID}/signature`, `GET /artifacts/{artifactID}/verify`
- **Database** — migration `015_signing` adds `ArtifactSignature` model
### Added — OCI Registry (`internal/domain/oci/`)
- **`Registry`** — content-addressable on-disk blob store implementing OCI Distribution Spec v1.1
- Storage layout: `{root}/blobs/sha256/<hex>` for blobs, `{root}/uploads/<uuid>` for in-progress uploads
- Full upload session lifecycle: start (POST), append chunk (PATCH), finalize with digest verification (PUT), cancel (DELETE), offset query (GET)
- 13 OCI distribution error codes defined (`ErrBlobUnknown`, `ErrDigestInvalid`, `ErrManifestInvalid`, etc.)
- **API handlers** (`internal/api/handlers/oci.go`, 525 lines) — full `/v2/{name}/{kind}/{ref}` routing: manifest push/get/delete, blob HEAD/get/delete, tag listing, chunked upload
- **Database** — migration `017_oci` adds `OCIRepository`, `OCIManifest`, `OCITag`, `OCIBlob`, `OCIUpload` models
- Registry is consumed by standard OCI tools (Docker, Podman, Skopeo, containerd)
### Added — Unified Security Page (`/repos/:owner/:repo/security`)
- **`RepoSecurityPage`** — single-page view combining SBOM status, secret leak detection, and vulnerability findings
- SBOM section: displays existing SBOM metadata with download button, or "Generate SBOM" form with branch/SHA input
- Secret Leaks section: lists leaks with severity badge, pattern name, commit SHA, ref, match sample, dismiss button
- Vulnerabilities section: lists findings with CVSS severity (CRITICAL/HIGH/MEDIUM/LOW), vuln ID, score, summary, PURL, version, fix suggestion, dismiss button; "Scan now" trigger
- **Route** added in `App.tsx`, nav link added in `RepoPage.tsx` tab bar
### Added — Pipeline Run SBOM Integration
- `PipelineRunPage` shows per-run SBOM section: metadata (components, SHA, generation time) + download button
- "Generate SBOM" button for completed/failed runs that lack one
- `useRunSBOM` / `useGenerateSBOM` hooks in `frontend/src/api/queries/sbom.ts`
- 404 from `useRunSBOM` handled gracefully (returns `null` instead of throwing)
### Added — Database Models
- Migration `016_sbom` — `SBOMReport` (repoId, runId, sha, format, componentCount, bomDocument, generatedAt)
- Migration `017_oci` — `OCIRepository`, `OCIManifest`, `OCITag`, `OCIBlob`, `OCIUpload`
- Migration `018_scanning` — `SecretLeak` (repoId, commitSha, ref, patternName, description, severity, matchSample, dismissed, dismissedBy, dismissedAt, detectedAt)
- Migration `019_vulnscan` — `VulnerabilityFinding` (repoId, vulnId, purl, version, summary, details, cvssScore, fixedVersion, dismissed, dismissedBy, dismissedAt, detectedAt)
- Migration `020_forgefed` — repository + pull request column updates
### Fixed
- SBOM per-run download endpoint (`/runs/{runID}/sbom/document`) was registered at the wrong router nesting level, causing a route conflict with the `GetLatestDocument` handler. Moved into the correct `/runs/{runID}` route block.
- `username` context key extraction in scanning and vulnerability handlers changed from raw string `"user"` to typed `middleware.ContextKeyUsername`
- Nil-safe `Needs` marshalling in orchestrator job creation
- Nil-safe findings response in vulnerability scan API
- `GenerateOnDemand` SBOM cache key now includes `runID` to prevent per-run generation from being shadowed by prior on-demand generation
> NOT just markdown notes.
---
## [0.9.0] — 2026-05-12
@@ -361,7 +401,8 @@ Initial development milestone. Core Git hosting, collaboration, and frontend SPA
---
[Unreleased]: https://github.com/forgeo/forgebucket/compare/v0.9.0...HEAD
[Unreleased]: https://github.com/forgeo/forgebucket/compare/v1.0.0...HEAD
[1.0.0]: https://github.com/forgeo/forgebucket/compare/v0.9.0...v1.0.0
[0.9.0]: https://github.com/forgeo/forgebucket/compare/v0.8.0...v0.9.0
[0.8.0]: https://github.com/forgeo/forgebucket/compare/v0.7.0...v0.8.0
[0.7.0]: https://github.com/forgeo/forgebucket/compare/v0.6.0...v0.7.0
README.md
+30 -12
View File
@@ -4,7 +4,7 @@
ForgeBucket is a self-hosted, federated developer operations platform. Where other Git platforms show you a list of files, ForgeBucket surfaces deployments, pipeline health, environment drift, and operational context directly alongside your code. Repositories are runtime systems. The dashboard is a command center.
**Status:** Active development. Phase 3F (ActivityPub federation) complete. Phase 4 (AI diagnostics + signed artifacts) is next.
**Status:** Active development. Phase 4 (signed artifacts, SBOM, secret/dependency scanning, OCI registry) complete. Phase 5 (AI diagnostics) is next.
---
@@ -69,7 +69,9 @@ ForgeBucket is a self-hosted, federated developer operations platform. Where oth
| Pipeline DAG visualization (frontend) | Done |
| Dashboard CI command center | Done |
| Pipeline log viewer (collapsible, per-step) | Done |
| Kubernetes / Firecracker runner backends | Planned (Phase 2D) |
| SBOM auto-generation on pipeline success | Done |
| Per-run SBOM download on pipeline detail page | Done |
| Kubernetes / Firecracker runner backends | Planned |
| Matrix builds + reusable workflow templates | Planned |
| Flaky test detection | Planned |
@@ -81,9 +83,9 @@ ForgeBucket is a self-hosted, federated developer operations platform. Where oth
| Unified operational timeline | Done |
| Secret management (Global → Workspace → Repo → Env) | Done |
| GitOps controller (drift detection + auto-sync) | Done |
| Deployment promotion workflows | Planned (Phase 4) |
| Rollback visualization | Planned (Phase 4) |
| Canary / blue-green support | Planned (Phase 4) |
| Deployment promotion workflows | Planned |
| Rollback visualization | Planned |
| Canary / blue-green support | Planned |
### Observability + Security
| Feature | Status |
@@ -93,10 +95,13 @@ ForgeBucket is a self-hosted, federated developer operations platform. Where oth
| HTTP instrumentation middleware (latency histogram, request counter) | Done |
| Per-repo operational health summary (`GET /repos/.../health`) | Done |
| NATS-driven pipeline + deployment counters | Done |
| Health sparklines in repo/env pages (frontend) | Planned (Phase 4) |
| Secret scanning | Planned (Phase 4) |
| Dependency scanning | Planned (Phase 4) |
| Signed artifacts (Sigstore/Cosign) | Planned (Phase 4) |
| SBOM generation (CycloneDX 1.4, auto on pipeline success) | Done |
| Secret scanning (15 regex patterns, push-triggered) | Done |
| Dependency vulnerability scanning (OSV API backed) | Done |
| Signed artifacts (ECDSA P-256, self-verifying bundles) | Done |
| OCI Distribution Spec v1.1 registry | Done |
| Unified repo Security page (SBOM + leaks + vulns) | Done |
| Health sparklines in repo/env pages (frontend) | Planned |
### Federation
| Feature | Status |
@@ -110,7 +115,7 @@ ForgeBucket is a self-hosted, federated developer operations platform. Where oth
| HTTP signatures (draft-cavage-http-signatures) | Done |
| Follow / Accept auto-accept flow | Done |
| RSA-2048 key pair lazy generation | Done |
| Cross-instance pull requests (ForgeFed) | Planned (Phase 4) |
| Cross-instance pull requests (ForgeFed) | Planned |
---
@@ -153,10 +158,15 @@ ForgeBucket
├── Environment Service (environments, deployments — internal/api/handlers/environment.go)
├── Secret Manager (scoped AES-256-GCM — internal/api/handlers/secret.go)
├── Workspace Service (multi-tenant namespaces — internal/api/handlers/workspace.go)
├── SBOM Generator (CycloneDX 1.4, auto on pipeline success — internal/domain/sbom/)
├── Secret Scanner (15 push-triggered regex patterns — internal/domain/scanning/)
├── Vulnerability Scanner (OSV API-backed dependency scanning — internal/domain/vulnscan/)
├── Artifact Signing (ECDSA P-256 self-verifying bundles — internal/domain/signing/)
├── OCI Registry (Distribution Spec v1.1 blob store — internal/domain/oci/)
├── Event Bus (NATS core, NoOp fallback — internal/events/)
├── Audit Log (every mutating request — internal/api/middleware/audit.go)
├── Federation Layer (ActivityPub inbox/outbox, HTTP signatures — internal/domain/federation/)
├── Database (PostgreSQL + XORM — internal/models/)
├── Database (PostgreSQL + XORM 20 migrations — internal/models/)
└── Web Frontend (React 18 + TypeScript, //go:embed — web/)
```
@@ -184,6 +194,11 @@ Logger → RealIP → Recoverer → Metrics → CORS → CSRF → SessionAuth
| Code editing | CodeMirror |
| Container | Docker Compose (dev) |
| Federation | ActivityPub / ForgeFed (WebFinger, actor, inbox/outbox, HTTP signatures) |
| SBOM format | CycloneDX 1.4 (JSON) |
| Vulnerability data | OSV API (`api.osv.dev`) |
| Secret detection | Regex-based (15 patterns, push-triggered) |
| Artifact signing | ECDSA P-256 (ASN.1 DER, self-verifying bundles) |
| OCI storage | On-disk content-addressable blob store (Distribution Spec v1.1) |
---
@@ -214,6 +229,8 @@ ForgeBucket has its own design language — intentionally distinct from GitHub a
| `ARTIFACT_ROOT` | No | Artifact storage path, defaults to `../artifacts` relative to `REPO_ROOT` |
| `NATS_URL` | No | NATS connection URL (e.g. `nats://localhost:4222`). If unset, event bus is no-op |
| `GITOPS_RECONCILE_INTERVAL` | No | Seconds between periodic drift checks, default `300`. `0` disables the ticker |
| `OCI_ROOT` | No | Root directory for OCI Distribution Spec blob and upload storage, defaults to `../oci` relative to `REPO_ROOT` |
| `ARTIFACT_SIGNING_KEY` | No | Path to ECDSA P-256 PEM for artifact signing; auto-generates ephemeral key if unset (warns on restart) |
| `INSTANCE_URL` | Yes | Public URL of this instance (no trailing slash) |
| `INSTANCE_NAME` | No | Display name, default `ForgeBucket` |
| `OIDC_ISSUER` | No | OIDC provider URL |
@@ -250,7 +267,8 @@ ForgeBucket has its own design language — intentionally distinct from GitHub a
| Phase 3D | GitOps controller + drift detection + auto-sync | Done |
| Phase 3E | Observability (Prometheus `/metrics`, structured `/health`, repo health API) | Done |
| Phase 3F | Federation handlers (ActivityPub WebFinger, actor, inbox/outbox, HTTP signatures) | Done |
| Phase 4 | AI diagnostics, signed artifacts, OCI registry, secret/dep scanning | Next |
| Phase 4 | Signed artifacts, SBOM, OCI registry, secret/dep scanning, security page | Done |
| Phase 5 | AI diagnostics, deployment promotions, rollback visualization | Next |
---