HomeLab/ForgeBucket
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

implemented gitops controller + drift detection

Browse Source
This commit is contained in:
erangel1
2026-05-12 19:51:59 +02:00
parent 35afa8d8f1
commit c7df53708c
17 changed files with 1064 additions and 261 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+77 -65
View File
@@ -19,73 +19,66 @@ The full product vision lives in [`ai_agent_master_prompt_for_building_modern_gi
## Architecture Map
```
cmd/forgebucket/ — binary entry point (main.go)
cmd/forgebucket/ — binary entry point (main.go)
internal/
api/
router.go — Chi router, all route definitions (~26 routes)
middleware/ — auth, CSRF, RBAC, logging
handlers/ — one file per domain (repo, pr, issue, auth, user, ssh...)
router.go — Chi router, all route definitions (60+ routes)
middleware/ — auth.go, csrf.go, rbac.go, audit.go
handlers/ — one file per domain area (see Key Files below)
domain/
git/ — sanitized git binary wrapper (exec.Command only, no shell)
federation/ — ActivityPub / ForgeFed (DATA LAYER ONLY — no handlers yet)
ci/ — CI orchestrator (EMPTY — Phase 2 stub)
models/ — XORM structs + 7 migration files
config/ — ENV-driven config, fails fast on missing secrets
web/ — //go:embed target for the built React SPA
git/ — sanitized git binary wrapper (exec.Command only, no shell)
binary.go — Run, Log, Tree, Diff, BlobCat, RevParse, etc.
agit.go — AGit ref parsing
ci/ — CI/CD execution engine (fully built — Phase 2B)
orchestrator.go — NATS-driven DAG orchestrator
runner_manager.go — job dispatch with Docker executor
executor.go — docker run, log streaming, workspace extraction
dag.go — topological sort, ReadyJobs
parser.go — .forgebucket/workflows/*.yml parser
types.go — WorkflowFile, WorkflowJob, WorkflowStep structs
gitops/ — GitOps controller (fully built — Phase 3D)
controller.go — NATS subscriptions, startup, periodic ticker
drift.go — CheckDrift, handlePush, periodicCheck
reconciler.go — TriggerSync, handleDeploymentSucceeded/Failed
federation/ — ActivityPub / ForgeFed (DATA LAYER ONLY — Phase 3F stub)
models/ — XORM structs + 13 migration files
config/ — ENV-driven config, fails fast on missing secrets
events/ — NATS EventBus interface + NATSBus + NoOpBus
web/ — //go:embed target for the built React SPA
frontend/
src/
pages/ — 20 route-level page components
components/ — shared UI (AppShell, Sidebar, Header, DiffViewer, etc.)
pages/ — route-level page components
components/ — shared UI (AppShell, Sidebar, Header, DiffViewer, etc.)
ui/
tokens.ts — SINGLE SOURCE OF TRUTH for all design tokens
hooks/ — custom React hooks
api/ — typed API client (fetch wrappers)
tokens.ts — SINGLE SOURCE OF TRUTH for all design tokens
hooks/ — custom React hooks
api/ — typed API client (fetch wrappers)
```
**Middleware chain — this order is fixed, do not reorder:**
```
Logger → RealIP → Recoverer → CORS → CSRF → SessionAuth → RBAC → Handler
Logger → RealIP → Recoverer → CORS → CSRF → SessionAuth → AuditLog → Handler
```
---
## Current Phase Status
Understand the phases before adding code — don't build Phase 3 infrastructure when Phase 2 is incomplete.
| Phase | Scope | Status |
|-------|-------|--------|
| 1 | Auth, Git HTTP, repos, PRs, issues, RBAC, webhooks, LFS, design system, 20-page SPA | **Complete** |
| 2A | NATS event bus, WebSocket hub upgrade, audit log | **Complete** |
| 1 | Auth, Git HTTP, repos, PRs, issues, RBAC, webhooks, LFS, design system | **Complete** |
| 2A | NATS event bus, WebSocket hub, audit log | **Complete** |
| 2B | CI orchestrator, runner manager, Docker executor, artifact registry | **Complete** |
| 2C | Pipeline DAG visualization, dashboard CI upgrade, command palette wiring | **Complete** |
| 2C | Pipeline DAG visualization, dashboard CI upgrade, command palette | **Complete** |
| 3A | Environment model + deployment tracking | **Complete** |
| 3B | Unified operational timeline | **Complete** |
| 3C | Workspaces + secret management hierarchy | **Active** |
| 3D | GitOps controller + drift detection | Planned |
| 3E | Observability (Prometheus, health sparklines) | Planned |
| 3C | Workspaces + secret management (Global → Workspace → Repo → Env) | **Complete** |
| 3D | GitOps controller + drift detection + auto-sync | **Complete** |
| 3E | Observability (Prometheus endpoint, health checks, sparklines) | **Next** |
| 3F | Federation handlers (ActivityPub inbox/outbox) | Planned |
| 4 | AI diagnostics, signed artifacts, OCI registry, secret/dep scanning | Planned |
| 4 | AI diagnostics, signed artifacts, OCI registry, dep/secret scanning | Planned |
Do not implement Phase 3+ features without explicit discussion. The `domain/federation/` directory is an intentional stub — the data model exists but no HTTP handlers should be wired until Phase 3F.
### Phase 3A — What to Build
Backend and frontend are both net-new for Phase 3A. Nothing exists yet.
**Backend:**
1. `internal/models/environment.go` — `Environment` (id, repoId, name, url, protectionRules JSON) + `Deployment` (id, envId, repoId, sha, ref, status, triggeredBy, description, runId, startedAt, finishedAt)
2. `internal/models/migrations/010_environments.go` — `Run010()` syncing both structs; call from `001_init.go`
3. `internal/api/handlers/environment.go` — `ListEnvironments`, `CreateEnvironment`, `GetEnvironment`, `UpdateEnvironment`, `DeleteEnvironment`, `ListDeployments`, `CreateDeployment`, `UpdateDeploymentStatus`; publish `deployment.*` NATS events
4. `internal/api/router.go` — wire routes under `/{owner}/{repo}/environments` and `/{owner}/{repo}/environments/{envName}/deployments`
**Frontend:**
5. `frontend/src/types/api.ts` — add `Environment`, `Deployment`, `DeployStatus` types
6. `frontend/src/api/queries/environments.ts` — `useEnvironments`, `useEnvironment`, `useCreateEnvironment`, `useUpdateEnvironment`, `useDeleteEnvironment`, `useDeployments`, `useCreateDeployment`, `useUpdateDeploymentStatus`
7. `frontend/src/pages/EnvironmentsPage.tsx` — environment cards each showing latest deployment status, SHA, who deployed, time; "New environment" flow; deployment history per env
8. `frontend/src/components/layout/Sidebar.tsx` — add `Environments` nav item between Pipelines and Settings in `RepoSubNav`
9. `frontend/src/pages/RepoPage.tsx` — surface deployment status badges in the repo header (latest deploy per env at a glance)
10. `frontend/src/App.tsx` — add route `repos/:owner/:repo/environments`
The `domain/federation/` directory is an intentional stub — the data model exists but no HTTP handlers should be wired until Phase 3F.
---
@@ -107,13 +100,19 @@ This rule is non-negotiable. It prevents command injection.
### Router / handlers
- Chi router. Route definitions in `internal/api/router.go`.
- One handler file per domain area. Keep handlers thin — business logic belongs in domain packages.
- All POST/PUT/DELETE routes require `X-CSRF-Token` header matching the session cookie. The middleware enforces this, but don't remove it from routes.
- All POST/PUT/DELETE routes require `X-CSRF-Token` header matching the session cookie. The CSRF middleware enforces this, but don't remove it from route definitions.
- There is a shared `resolveRepoID(db, w, r)` function in `internal/api/handlers/repo_lookup.go` — use it instead of duplicating repo resolution logic.
### Database
- XORM for all DB access. Structs in `internal/models/`.
- Migrations are numbered files in `internal/models/migrations/`. Always add a new migration file; never edit existing ones.
- Migrations are numbered files in `internal/models/migrations/`. Always add a new file; never edit existing ones. Current highest: **013**.
- No raw SQL strings built from user input.
### Events
- Publish to NATS via `bus.Publish(events.SubjectXxx, payload)` where the subject is a constant from `internal/events/subjects.go`.
- Payload types are in `internal/events/types.go` — use them for type-safe unmarshaling in subscribers.
- `NoOpBus` silently drops events when `NATS_URL` is unset — the app must work normally without NATS.
### Secrets and config
- All secrets come from environment variables via `internal/config/`.
- Never hardcode secrets, tokens, or credentials anywhere.
@@ -121,7 +120,7 @@ This rule is non-negotiable. It prevents command injection.
### Error handling
- Return errors up the call stack. Don't swallow them silently.
- HTTP handlers use consistent JSON error responses — follow the pattern in existing handlers.
- HTTP handlers use consistent JSON error responses — follow the pattern in `jsonError` / `jsonOK` in `internal/api/handlers/helpers.go`.
---
@@ -143,7 +142,7 @@ All spacing, color, and sizing values must come from `frontend/src/ui/tokens.ts`
- Touch targets: 44px minimum height/width on all interactive elements (buttons, links, icon buttons).
### Dark mode
- Use Tailwind v4 `@variant dark` — not hardcoded dark: classes unless inside a component that explicitly handles both.
- Use Tailwind v4 `@variant dark` — not hardcoded `dark:` classes unless inside a component that explicitly handles both.
- Colors must work in both light and dark modes. Test both.
### Component patterns
@@ -155,20 +154,21 @@ All spacing, color, and sizing values must come from `frontend/src/ui/tokens.ts`
### API calls
- Use the typed API client in `frontend/src/api/` — don't write raw `fetch` calls in components.
- Always include `X-CSRF-Token` header on mutating requests.
- Always include `X-CSRF-Token` header on mutating requests (the client does this automatically via `getCSRFToken()`).
---
## What NOT to Do
- **No shell string injection** — see Go conventions above
- **No shell string injection** — see Go conventions above; always discrete `exec.Command` args
- **No hardcoded secrets** — everything via env
- **No skipping CSRF** — all mutating routes require it
- **No arbitrary design values** — tokens.ts is the law
- **No Phase 3+ features without discussion** — don't wire up GitOps, federation handlers, or the command palette until Phase 2 is complete
- **No new color tokens** — if the design requires a new color, discuss it; don't invent one
- **No modal-heavy UX** — this platform uses progressive disclosure; avoid deep modal chains
- **No YAML-centric UI** — pipeline and environment config should feel operational, not config-file editing
- **No arbitrary design values** — `tokens.ts` is the law
- **No new color tokens without discussion** — the existing palette covers all cases
- **No modal-heavy UX** — progressive disclosure; avoid deep modal chains
- **No YAML-centric UI** — pipeline and GitOps config should feel operational, not config-file editing
- **No editing existing migration files** — always add a new numbered migration
- **No direct `fmt.Println` for logging** — use `log.Printf` so structured logs work correctly
---
@@ -189,16 +189,26 @@ make lint # go vet + ESLint
| File | Purpose |
|------|---------|
| `internal/api/router.go` | All route definitions — start here for backend |
| `internal/models/` | XORM models + migrations — all DB schemas |
| `internal/config/config.go` | Env-driven config, required vars |
| `internal/domain/git/` | Git binary wrapper — safe exec patterns |
| `internal/api/router.go` | All route definitions — start here for backend work |
| `internal/api/handlers/repo_lookup.go` | Shared `resolveRepoID` helper |
| `internal/models/` | All XORM models + 13 migration files |
| `internal/config/config.go` | All env vars, fail-fast validation |
| `internal/events/subjects.go` | All NATS event subject constants |
| `internal/events/types.go` | Typed event payload structs |
| `internal/domain/git/binary.go` | Git binary wrapper — safe exec patterns, `RevParse`, `BlobCat`, etc. |
| `internal/domain/ci/orchestrator.go` | CI DAG orchestrator |
| `internal/domain/ci/executor.go` | Docker job executor + log streaming |
| `internal/domain/gitops/controller.go` | GitOps reconciliation controller |
| `internal/domain/gitops/drift.go` | `CheckDrift`, drift detection logic |
| `internal/api/handlers/environment.go` | Environment + deployment CRUD |
| `internal/api/handlers/gitops.go` | GitOps config + drift HTTP endpoints |
| `internal/api/handlers/secret.go` | Scoped secret management |
| `internal/api/handlers/workspace.go` | Workspace + member management |
| `internal/api/middleware/audit.go` | Audit log middleware |
| `frontend/src/ui/tokens.ts` | Design token source of truth |
| `frontend/src/components/AppShell.tsx` | Root layout wrapper |
| `frontend/src/components/Sidebar.tsx` | 3-state navigation sidebar |
| `frontend/src/pages/` | All 20 route-level pages |
| `frontend/src/api/` | Typed API client |
| `.env.example` | All required environment variables |
| `frontend/src/api/client.ts` | Typed API client with CSRF handling |
| `.env.example` | All environment variables with documentation |
| `CLAUDE.md` | Developer guide (rules overlap with this file — CLAUDE.md takes precedence on conflicts) |
---
@@ -207,7 +217,9 @@ make lint # go vet + ESLint
```bash
cp .env.example .env # fill SESSION_SECRET and CSRF_SECRET
make docker-up # PostgreSQL via Docker Compose
make migrate # run XORM migrations
make docker-up # PostgreSQL + NATS via Docker Compose
make migrate # run XORM migrations (currently 013)
make dev # Go :8080 + Vite :5173
```
CI execution requires Docker to be running locally. If unavailable, the runner logs a warning and CI jobs are queued but not executed.