HomeLab/ForgeBucket
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

implemented gitops controller + drift detection

Browse Source
This commit is contained in:
erangel1
2026-05-12 19:51:59 +02:00
parent 35afa8d8f1
commit c7df53708c
17 changed files with 1064 additions and 261 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+50 -38
View File
@@ -4,7 +4,7 @@
ForgeBucket is a self-hosted, federated developer operations platform. Where other Git platforms show you a list of files, ForgeBucket surfaces deployments, pipeline health, environment drift, and operational context directly alongside your code. Repositories are runtime systems. The dashboard is a command center.
**Status:** Phase 2C in progress. CI/CD execution backend is fully operational. Pipeline visualization and dashboard integration are being wired up now.
**Status:** Active development. Phase 3D (GitOps controller + drift detection) complete. Phase 3E (observability) is next.
---
@@ -32,7 +32,8 @@ ForgeBucket is a self-hosted, federated developer operations platform. Where oth
| OIDC / OAuth2 (optional) | Done |
| Access tokens (scoped, expiring) | Done |
| Deploy keys | Done |
| Audit log | Done |
| Audit log (admin-only, filterable) | Done |
| Workspaces (multi-tenant namespaces) | Done |
### Git Hosting
| Feature | Status |
@@ -59,34 +60,36 @@ ForgeBucket is a self-hosted, federated developer operations platform. Where oth
### CI/CD
| Feature | Status |
|---------|--------|
| CI orchestrator (DAG pipeline execution) | Done (Phase 2B) |
| Runner manager (Docker backend) | Done (Phase 2B) |
| Build artifact storage | Done (Phase 2B) |
| Pipeline cancellation + job retry | Done (Phase 2B) |
| NATS event bus + WebSocket live push | Done (Phase 2A) |
| Pipeline DAG visualization (frontend) | Done (Phase 2C) |
| Dashboard CI command center | Done (Phase 2C) |
| Pipeline log viewer (per-step, collapsible) | Done (Phase 2C) |
| NATS event bus + WebSocket live push | Done |
| CI orchestrator (DAG pipeline execution) | Done |
| Runner manager (Docker backend) | Done |
| Build artifact storage + download | Done |
| Pipeline cancellation + job retry | Done |
| Pipeline log streaming (per-step, NATS) | Done |
| Pipeline DAG visualization (frontend) | Done |
| Dashboard CI command center | Done |
| Pipeline log viewer (collapsible, per-step) | Done |
| Kubernetes / Firecracker runner backends | Planned (Phase 2D) |
| Forgejo Actions gRPC integration | Planned |
| Matrix builds + reusable workflow templates | Planned |
| Flaky test detection | Planned |
### GitOps + Environments
### Environments + GitOps
| Feature | Status |
|---------|--------|
| Environment model + deployment tracking | **In progress (Phase 3A)** |
| Unified operational timeline | Planned (Phase 3B) |
| Secret management hierarchy | Planned (Phase 3C) |
| GitOps controller + drift detection | Planned (Phase 3D) |
| Deployment promotion workflows | Planned (Phase 3D) |
| Rollback visualization | Planned (Phase 3D) |
| Canary / blue-green support | Planned (Phase 3D) |
| Environment model + deployment tracking | Done |
| Deployment status lifecycle API | Done |
| Unified operational timeline | Done |
| Secret management (Global → Workspace → Repo → Env) | Done |
| GitOps controller (drift detection + auto-sync) | Done |
| Deployment promotion workflows | Planned (Phase 4) |
| Rollback visualization | Planned (Phase 4) |
| Canary / blue-green support | Planned (Phase 4) |
### Observability + Security
| Feature | Status |
|---------|--------|
| Prometheus endpoint + health sparklines | Planned (Phase 3E) |
| Prometheus endpoint + health checks | Planned (Phase 3E) |
| Health sparklines in repo/env pages | Planned (Phase 3E) |
| Secret scanning | Planned (Phase 4) |
| Dependency scanning | Planned (Phase 4) |
| Signed artifacts (Sigstore/Cosign) | Planned (Phase 4) |
@@ -120,7 +123,7 @@ make dev
The Go API runs at `http://localhost:8080`. The Vite dev server runs at `http://localhost:5173` and proxies API requests.
> **Local dev note:** `DATABASE_URL` must use `localhost` (not `postgres`) and `NATS_URL` must be set to `nats://localhost:4222`. The `.env` file ships with correct defaults for local development. See `.env.example` for all variables.
> **Docker note:** CI execution requires the Docker daemon to be running. If Docker is unavailable, the runner manager logs a warning and disables CI; the rest of the platform works normally.
---
@@ -128,22 +131,26 @@ The Go API runs at `http://localhost:8080`. The Vite dev server runs at `http://
```
ForgeBucket
├── API Gateway (Chi router, internal/api/)
├── API Gateway (Chi router — internal/api/router.go)
├── Auth Service (sessions, CSRF, OIDC — internal/api/handlers/)
├── Repository Service (git HTTP, branches, LFS — internal/domain/git/)
├── Pull Request Service (PRs, reviews, merge — internal/api/handlers/)
├── Issue Service (issues, labels — internal/api/handlers/)
├── CI Orchestrator (DAG execution, Docker runner — internal/domain/ci/) ← Phase 2B done
├── Event Bus (NATS core, NoOp fallback — internal/events/) ← Phase 2A done
├── Federation Layer (ActivityPub actors — internal/domain/federation/) ← Phase 3F stub
├── Secret Manager (env-based, scoped tokens — internal/config/)
├── Issue Service (issues — internal/api/handlers/)
├── CI Orchestrator (DAG execution, Docker runner — internal/domain/ci/)
├── GitOps Controller (drift detection, auto-sync — internal/domain/gitops/)
├── Environment Service (environments, deployments — internal/api/handlers/environment.go)
├── Secret Manager (scoped AES-256-GCM — internal/api/handlers/secret.go)
├── Workspace Service (multi-tenant namespaces — internal/api/handlers/workspace.go)
├── Event Bus (NATS core, NoOp fallback — internal/events/)
├── Audit Log (every mutating request — internal/api/middleware/audit.go)
├── Federation Layer (ActivityPub actors — internal/domain/federation/) ← Phase 3F stub
├── Database (PostgreSQL + XORM — internal/models/)
└── Web Frontend (React 18 + TypeScript, embedded via //go:embed — web/)
└── Web Frontend (React 18 + TypeScript, //go:embed — web/)
```
**Middleware chain (every request):**
**Middleware chain (every authenticated request):**
```
Logger → RealIP → Recoverer → CORS → CSRF → SessionAuth → RBAC → AuditLog → Handler
Logger → RealIP → Recoverer → CORS → CSRF → SessionAuth → AuditLog → Handler
```
---
@@ -155,12 +162,13 @@ Logger → RealIP → Recoverer → CORS → CSRF → SessionAuth → RBAC → A
| Language | Go 1.21+ |
| Router | Chi |
| ORM / Migrations | XORM + PostgreSQL |
| Event bus | NATS (core; JetStream planned for Phase 2B durability) |
| Real-time | WebSockets (nhooyr.io/websocket) |
| CI execution | Docker (`docker run --rm`) |
| Event bus | NATS core (`github.com/nats-io/nats.go`) |
| Real-time | WebSockets (`nhooyr.io/websocket`) |
| CI execution | Docker (`docker run --rm` via `exec.Command`) |
| Frontend framework | React 18 + TypeScript |
| Build tool | Vite |
| Styling | Tailwind CSS v4 |
| YAML parsing | `gopkg.in/yaml.v3` (workflow definitions) |
| Code editing | CodeMirror |
| Container | Docker Compose (dev) |
| Federation | ActivityPub / ForgeFed (data layer only) |
@@ -186,12 +194,14 @@ ForgeBucket has its own design language — intentionally distinct from GitHub a
| Variable | Required | Description |
|----------|----------|-------------|
| `DATABASE_URL` | Yes | PostgreSQL connection string — use `localhost` for local dev |
| `DATABASE_URL` | Yes | PostgreSQL connection string |
| `SESSION_SECRET` | Yes | Session signing key, ≥ 32 chars (`openssl rand -hex 32`) |
| `CSRF_SECRET` | Yes | CSRF key, exactly 32 chars (`openssl rand -hex 16`) |
| `PORT` | No | HTTP port, default `8080` |
| `REPO_ROOT` | Yes | Absolute path for bare git repository storage |
| `NATS_URL` | No | NATS connection URL (e.g. `nats://localhost:4222`). If unset, CI runs in no-op mode |
| `ARTIFACT_ROOT` | No | Artifact storage path, defaults to `../artifacts` relative to `REPO_ROOT` |
| `NATS_URL` | No | NATS connection URL (e.g. `nats://localhost:4222`). If unset, event bus is no-op |
| `GITOPS_RECONCILE_INTERVAL` | No | Seconds between periodic drift checks, default `300`. `0` disables the ticker |
| `INSTANCE_URL` | Yes | Public URL of this instance (no trailing slash) |
| `INSTANCE_NAME` | No | Display name, default `ForgeBucket` |
| `OIDC_ISSUER` | No | OIDC provider URL |
@@ -224,9 +234,11 @@ ForgeBucket has its own design language — intentionally distinct from GitHub a
| Phase 2C | Pipeline DAG visualization, dashboard CI upgrade, command palette | Done |
| Phase 3A | Environment model + deployment tracking | Done |
| Phase 3B | Unified operational timeline | Done |
| Phase 3C | Workspaces + secret management hierarchy | **In progress** |
| Phase 3D–F | GitOps/drift, federation, observability | Planned |
| Phase 4 | AI diagnostics, signed artifacts, OCI registry, dep scanning | Planned |
| Phase 3C | Workspaces + secret management hierarchy (Global → Workspace → Repo → Env) | Done |
| Phase 3D | GitOps controller + drift detection + auto-sync | Done |
| Phase 3E | Observability (Prometheus endpoint, health checks, sparklines) | Next |
| Phase 3F | Federation handlers (ActivityPub inbox/outbox, cross-instance PRs) | Planned |
| Phase 4 | AI diagnostics, signed artifacts, OCI registry, secret/dep scanning | Planned |
---