HomeLab/ForgeBucket
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

pipeline dag visualization + Dashboard command center upgrade + command palette wiring. fixed repo pipeline page.

Browse Source
This commit is contained in:
erangel1
2026-05-11 20:49:48 +02:00
parent 3838aa1f53
commit 4f2fb846dd
15 changed files with 1659 additions and 203 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+103 -18
View File
@@ -9,28 +9,108 @@ Versions follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Planned — Phase 2 (CI/CD)
- CI orchestrator with DAG pipeline execution
- Runner manager (Docker, Kubernetes, Firecracker backends)
- Pipeline DAG visualization (visual dependency graph with live execution state)
- Build artifact storage and retention policies
- Forgejo Actions gRPC integration
- Flaky test detection
- Pipeline log viewer (collapsible, filterable, syntax-highlighted)
- Matrix builds and reusable workflow templates
- Concurrency controls and pipeline cancellation
### In Progress — Phase 2C (CI Legibility)
- Pipeline DAG visualization (PipelineRunPage with real job/step graph)
- Dashboard CI command center upgrade (replace placeholder with live recent runs)
- Command palette wiring (pipeline runs in search, Pipelines quick-nav)
- Global cross-repo pipeline runs feed (`/pipelines` page)
- Per-step log viewer (collapsible, streamed from backend)
### Planned — Phase 3 (GitOps + Observability + Federation)
- GitOps controller with reconciliation loops
- Environment management and topology visualization
- Environment model + deployment tracking
- Unified operational timeline (commits + deployments + CI failures merged)
- Drift detection and sync status
- Deployment promotion workflows (dev → staging → production)
- Rollback visualization and one-click rollbacks
- Canary and blue/green deployment support
- Unified operational timeline (commits + deployments + incidents + CI failures merged)
- ActivityPub / ForgeFed federation handlers (inbox, outbox, cross-instance PRs)
- Audit log (all administrative and git-over-HTTP actions)
- Secret scanning and dependency vulnerability scanning
- Secret management hierarchy (Global → Org → Repo → Env)
- Observability (Prometheus endpoint, health sparklines)
### Planned — Phase 4
- AI diagnostics (pipeline failure root-cause analysis)
- Signed artifacts (Sigstore/Cosign)
- OCI package registry
- Secret and dependency vulnerability scanning
---
## [0.3.0] — 2026-05-11
Phase 2B complete. Full CI/CD execution backend operational.
### Added — CI Orchestrator (`internal/domain/ci/`)
- DAG-based pipeline orchestrator (`orchestrator.go`): subscribes to NATS `push.received`,
parses `.forgebucket/workflows/*.yml`, creates `PipelineRun`/`PipelineJob`/`PipelineStep`
records, advances DAG on `job.completed`/`job.failed`, recovers stale runs on startup
- Docker executor (`executor.go`): runs steps in isolated containers (`docker run --rm`),
streams logs to DB and NATS via `pipeline.log` subject, handles `git archive` workspace extraction
- Runner manager (`runner_manager.go`): semaphore-limited concurrent job dispatch (default 4),
subscribes to `job.queued`, calls executor when Docker is available
- DAG engine (`dag.go`): full topological sort (`TopoSort`) and `ReadyJobs` for dependency resolution
- Workflow parser (`parser.go`): reads `.forgebucket/workflows/*.yml` from git ref,
`MatchesPushTrigger` with glob pattern support
- CI types (`types.go`): `WorkflowFile`, `WorkflowJob`, `WorkflowStep`, YAML `StringOrSlice` unmarshaler
### Added — CI API Handlers
- `GET /api/v1/repos/:owner/:repo/pipelines` — list pipeline definitions
- `GET /api/v1/repos/:owner/:repo/runs` — list pipeline runs (most recent first, limit 30)
- `GET /api/v1/repos/:owner/:repo/runs/:runID` — run detail with full job + step tree
- `POST /api/v1/repos/:owner/:repo/runs/:runID/cancel` — cancel queued or running run
- `POST /api/v1/repos/:owner/:repo/runs/:runID/jobs/:jobID/retry` — re-queue failed/cancelled job
- `GET /api/v1/repos/:owner/:repo/runs/:runID/jobs/:jobID/logs` — step-level log chunks
- `GET /api/v1/repos/:owner/:repo/runs/:runID/artifacts` — list artifacts for a run
- `POST /api/v1/repos/:owner/:repo/runs/:runID/artifacts` — upload artifact (multipart, 512 MB max)
- `GET /api/v1/repos/:owner/:repo/artifacts/:artifactID/download` — artifact download with path traversal guard
- `GET /api/v1/admin/runners` — list registered runners (admin-only)
- `POST /api/v1/admin/runners/register` — register a new runner with bcrypt token hashing (admin-only)
### Added — Database Models (migration `009_ci`)
- `Pipeline` — workflow definition record (name, filePath, repoId)
- `PipelineRun` — execution record (triggerRef, triggerSha, triggeredBy, status, startedAt, finishedAt)
- `PipelineJob` — single DAG node (name, image, needs JSON, status, timing)
- `PipelineStep` — single command within a job (seq, runCmd, usesAction, exitCode, timing)
- `PipelineStepLog` — append-only log chunk storage (stepId, chunkIndex, content)
- `Runner` — registered execution backend (name, labels, status, tokenHash, lastSeenAt)
- `Artifact` — build artifact (runId, repoId, name, storagePath, size, contentType)
---
## [0.2.0] — 2026-05-11
Phase 2A complete. Real-time event infrastructure and audit log operational.
### Added — NATS Event Bus (`internal/events/`)
- `EventBus` interface: `Publish`, `Subscribe`, `Close`
- `NATSBus`: NATS-backed implementation with auto-reconnect, max-reconnect disabled
- `NoOpBus`: silent fallback when `NATS_URL` is not configured (app fully functional without NATS)
- `New(url)` factory: returns `NATSBus` if URL is set, `NoOpBus` otherwise
- Event subjects defined in `subjects.go`:
- `repo.*` (created, deleted, pushed)
- `push.received`
- `pr.*` (opened, merged, closed)
- `issue.*` (opened, closed)
- `pipeline.*` (queued, started, succeeded, failed, cancelled)
- `job.*` (queued, started, completed, failed), `pipeline.log`
- `deployment.*`, `environment.*` (Phase 3 stubs)
- `audit.event`
### Added — WebSocket Hub (`internal/api/handlers/ws.go`)
- `GET /ws` — upgrades HTTP to WebSocket (nhooyr.io/websocket)
- Subscribes to all NATS subjects on connect, fans events to the client as JSON
- Optional session auth (`auth.Optional` middleware) — works for guests too
- Phase 2B note: per-user event filtering is a planned upgrade
### Added — Audit Log
- `AuditLog` model (migration `008_audit_log`): actor, method, path, statusCode, requestBody, ipAddr, timestamp
- `AuditLog` middleware: records every authenticated request to the DB and publishes `audit.event`
- `GET /api/v1/audit` — paginated audit log query (admin-only, filterable by actor/method/time range)
### Fixed — Local development environment
- `DATABASE_URL` was using Docker-internal hostname `postgres`; corrected to `localhost` for `make dev`
- Added `NATS_URL=nats://localhost:4222` to `.env` (was missing; CI orchestrator requires it)
- `REPO_ROOT` corrected to `/tmp/forgebucket/repos` (Docker path `/var/lib/forgebucket/repos` requires sudo on macOS)
---
@@ -70,7 +150,9 @@ Initial development milestone. Core Git hosting, collaboration, and frontend SPA
### Added — Frontend SPA
- React 18 + TypeScript + Vite, embedded into Go binary via `//go:embed`
- 20 route-level pages: Login, Register, Dashboard, Repos, CreateRepo, ImportRepo, Repo, RepoSettings, Blob, Commits, Branches, RepoIssues, RepoPRs, CreatePR, PRDetail, Starred, PRs (cross-repo), Pipelines (placeholder), Explore, Profile, Settings
- 20 route-level pages: Login, Register, Dashboard, Repos, CreateRepo, ImportRepo, Repo,
RepoSettings, Blob, Commits, Branches, RepoIssues, RepoPRs, CreatePR, PRDetail, Starred,
PRs (cross-repo), Pipelines (placeholder), Explore, Profile, Settings
- AppShell layout wrapper for all authenticated pages
- Triple-state sidebar: expanded (320px) / collapsed (56px) / mobile bottom bar
- Mobile-first responsive design (375px → 1440px)
@@ -90,13 +172,16 @@ Initial development milestone. Core Git hosting, collaboration, and frontend SPA
- System font stack (Segoe UI, Roboto, sans-serif)
### Added — Infrastructure
- PostgreSQL + XORM with 7 migration files covering: users, repositories, issues, SSH keys, access tokens, deploy keys, workflows, and LFS settings
- PostgreSQL + XORM with 7 migration files covering: users, repositories, issues, SSH keys,
access tokens, deploy keys, workflows, and LFS settings
- ActivityPub actor data model (FederationActor with inbox/outbox URLs and RSA key pairs) — data layer only
- Docker Compose setup for local PostgreSQL
- Docker Compose setup for local PostgreSQL + NATS
- Makefile targets: dev, build, migrate, test, lint, docker-up
- WebSockets foundation for live logs and notifications
---
[Unreleased]: https://github.com/forgeo/forgebucket/compare/v0.1.0...HEAD
[Unreleased]: https://github.com/forgeo/forgebucket/compare/v0.3.0...HEAD
[0.3.0]: https://github.com/forgeo/forgebucket/compare/v0.2.0...v0.3.0
[0.2.0]: https://github.com/forgeo/forgebucket/compare/v0.1.0...v0.2.0
[0.1.0]: https://github.com/forgeo/forgebucket/releases/tag/v0.1.0