📚 Comprehensive Documentation Suite (Mil-Spec/IBM Redbook Style)

INVESTOR DOCUMENTATION: - INVESTOR_BRIEF.md: Executive summary, capacity metrics, architecture overview - PLATFORM_CAPABILITIES.md: 5 modules, 27 subcomponents catalog - TECHNICAL_ARCHITECTURE.md: Mermaid diagrams, data flows, extension points CTO/SENIOR DEV ONBOARDING: - CTO_ONBOARDING.md: System overview, security model, operational runbook - DEVELOPER_GUIDE.md: Clone→Run→Deploy workflow, debugging - API_REFERENCE.md: 30+ endpoints with request/response specs - DATABASE_SCHEMA.md: 30+ tables in Harris Matrix order - COMPONENT_LIBRARY.md: 182 React components catalog REFERENCE: - GLOSSARY.md: 70+ platform terms defined - ADMIN_PAGES_GUIDE.md: 66 admin pages by module Style: BLUF format, zero marketing fluff, high-density tables
2025-12-14 13:58:28 -05:00
parent 260baa2f4b
commit 4c632b6229
10 changed files with 3482 additions and 0 deletions
--- a/docs/CTO_ONBOARDING.md
+++ b/docs/CTO_ONBOARDING.md
@@ -0,0 +1,296 @@
+# CTO ONBOARDING: Spark Platform Technical Leadership Guide
+
+> **BLUF**: Spark is a multi-tenant content scaling platform. 30+ PostgreSQL tables, 30+ API endpoints, 182 React components. Self-hosted via Docker Compose on Coolify. This document provides the 30,000ft view for technical leadership.
+
+---
+
+## 1. System Overview
+
+### 1.1 What Spark Does
+1. Ingests buyer personas, location data, content templates
+2. Computes Cartesian products of variations
+3. Generates unique SEO articles at scale
+4. Manages multi-site content distribution
+
+### 1.2 Key Metrics
+| Metric | Value |
+|--------|-------|
+| Max articles per campaign config | 50,000+ |
+| Database collections | 30+ |
+| API endpoints | 30+ |
+| React components | 182 |
+| Admin pages | 25 directories |
+
+---
+
+## 2. Repository Structure
+
+```
+spark/
+├── frontend/                 # Astro SSR + React
+│   ├── src/
+│   │   ├── components/      # 182 React components
+│   │   │   ├── admin/      # Admin dashboard (95 files)
+│   │   │   ├── blocks/     # Page builder (25 files)
+│   │   │   ├── ui/         # Shadcn-style (18 files)
+│   │   │   └── ...
+│   │   ├── pages/
+│   │   │   ├── admin/      # 25 admin directories
+│   │   │   ├── api/        # 15 API directories
+│   │   │   └── preview/    # 4 preview routes
+│   │   ├── lib/
+│   │   │   ├── directus/   # SDK client, fetchers
+│   │   │   └── schemas.ts  # TypeScript types
+│   │   └── hooks/          # React hooks
+│   ├── Dockerfile
+│   └── package.json
+│
+├── directus-extensions/      # Custom Directus code
+│   ├── endpoints/           # 4 custom endpoints
+│   └── hooks/              # 2 event hooks
+│
+├── docs/                     # Documentation (this folder)
+├── scripts/                  # Utility scripts
+├── complete_schema.sql       # Golden Schema
+├── docker-compose.yaml       # Infrastructure
+└── start.sh                  # Directus startup
+```
+
+---
+
+## 3. Technology Stack
+
+| Layer | Technology | Version | Rationale |
+|-------|------------|---------|-----------|
+| Frontend | Astro | 4.7 | SSR + Islands = optimal performance |
+| UI | React | 18.3 | Component ecosystem, team familiarity |
+| Styling | Tailwind | 3.4 | Utility-first, fast iteration |
+| State | React Query | 5.x | Server state management, caching |
+| Backend | Directus | 11 | Headless CMS, auto-generated API |
+| Database | PostgreSQL | 16 | ACID, JSON support, extensible |
+| Geo | PostGIS | 3.4 | Spatial queries for location data |
+| Cache | Redis | 7 | Session + queue backing store |
+| Queue | BullMQ | - | Robust job processing |
+| Deploy | Coolify | - | Self-hosted PaaS, Docker-native |
+
+---
+
+## 4. Database Schema Summary
+
+### 4.1 Creation Order (Harris Matrix)
+
+| Batch | Description | Tables |
+|-------|-------------|--------|
+| 1: Foundation | Zero dependencies | sites, campaign_masters, avatar_intelligence, avatar_variants, cartesian_patterns, geo_intelligence, offer_blocks |
+| 2: Walls | Depend on Batch 1 | generated_articles, generation_jobs, pages, posts, leads, headline_inventory, content_fragments |
+| 3: Roof | Complex dependencies | link_targets, globals, navigation, work_log, hub_pages, forms, form_submissions, site_analytics, events, pageviews, conversions, locations_* |
+
+### 4.2 Parent Tables
+
+| Table | Purpose | Children |
+|-------|---------|----------|
+| `sites` | Multi-tenant root | 10+ tables reference via `site_id` |
+| `campaign_masters` | Campaign config | 3 tables reference via `campaign_id` |
+
+### 4.3 Key Relationships
+
+```
+sites ──┬── pages
+        ├── posts
+        ├── generated_articles
+        ├── leads
+        ├── navigation
+        ├── globals (singleton per site)
+        └── site_analytics
+
+campaign_masters ──┬── headline_inventory
+                   ├── content_fragments
+                   └── generated_articles
+```
+
+Full schema: See `docs/DATABASE_SCHEMA.md`
+
+---
+
+## 5. API Surface
+
+### 5.1 Public Endpoints (No Auth)
+
+| Endpoint | Method | Purpose |
+|----------|--------|---------|
+| `/api/lead` | POST | Lead form submission |
+| `/api/forms/submit` | POST | Generic form handler |
+| `/api/track/*` | POST | Analytics tracking |
+
+### 5.2 Admin Endpoints (Token Required)
+
+| Endpoint | Method | Purpose |
+|----------|--------|---------|
+| `/api/seo/generate-*` | POST | Content generation |
+| `/api/seo/approve-batch` | POST | Workflow advancement |
+| `/api/campaigns` | GET/POST | Campaign CRUD |
+| `/api/admin/*` | Various | Administrative ops |
+
+### 5.3 God-Mode Endpoints (Elevated Token)
+
+| Endpoint | Purpose |
+|----------|---------|
+| `/god/schema/collections/create` | Create new collection |
+| `/god/schema/snapshot` | Export schema YAML |
+| `/god/data/bulk-insert` | Mass data insert |
+
+Full API reference: See `docs/API_REFERENCE.md`
+
+---
+
+## 6. Extension Points
+
+### 6.1 Adding New Features
+
+| Extension Type | Location | Process |
+|---------------|----------|---------|
+| New Collection | `complete_schema.sql` | Add table, update schemas.ts, add admin page |
+| New API Endpoint | `frontend/src/pages/api/` | Export async handler |
+| New Admin Page | `frontend/src/pages/admin/` | Create .astro file, add component |
+| New Block Type | `frontend/src/components/blocks/` | Create component, register in BlockRenderer |
+| New Directus Extension | `directus-extensions/` | Add endpoint or hook, restart container |
+
+### 6.2 Schema Modification Process
+
+1. Update `complete_schema.sql` (maintain Harris Matrix order)
+2. Update `frontend/src/lib/schemas.ts` (TypeScript types)
+3. Run `npm run build` to verify types
+4. Deploy with `FORCE_FRESH_INSTALL=true` (CAUTION: wipes DB)
+
+### 6.3 API Modification Process
+
+1. Create/edit file in `frontend/src/pages/api/`
+2. Export handler: `export async function POST({ request })`
+3. Test locally: `npm run dev`
+4. Update `docs/API_REFERENCE.md`
+5. Git push triggers deploy
+
+---
+
+## 7. Security Model
+
+### 7.1 Authentication Layers
+
+| Layer | Method | Protection |
+|-------|--------|------------|
+| Directus Admin | Email/Password | Full CMS access |
+| API Tokens | Static Bearer | Scoped collection access |
+| God-Mode | X-God-Token header | Schema operations only |
+| Public | No auth | Read-only published content |
+
+### 7.2 Multi-Tenant Isolation
+
+- All content tables have `site_id` FK
+- Queries filter by `site_id` automatically
+- No cross-tenant data leakage possible via standard API
+
+### 7.3 CORS Configuration
+
+```yaml
+CORS_ORIGIN: 'https://spark.jumpstartscaling.com,https://launch.jumpstartscaling.com,http://localhost:4321'
+```
+
+Modify in `docker-compose.yaml` for additional origins.
+
+---
+
+## 8. Performance Considerations
+
+### 8.1 Current Optimizations
+
+| Area | Technique |
+|------|-----------|
+| SSR | Islands Architecture (minimal JS) |
+| Database | Indexed FKs, status fields |
+| API | Field selection, pagination |
+| Build | Brotli compression, code splitting |
+
+### 8.2 Scaling Paths
+
+| Constraint | Solution |
+|------------|----------|
+| Database load | Read replicas, connection pooling |
+| API throughput | Horizontal frontend replicas |
+| Queue depth | Additional BullMQ workers |
+| Storage | Object storage (S3-compatible) |
+
+### 8.3 Known Bottlenecks
+
+| Area | Issue | Mitigation |
+|------|-------|------------|
+| Article generation | CPU-bound spintax | Parallelized in BullMQ |
+| Large campaigns | Memory for Cartesian | Streaming/batched processing |
+| Image generation | Canvas rendering | Queue-based async |
+
+---
+
+## 9. Operational Runbook
+
+### 9.1 Deployment
+
+```bash
+git push origin main  # Coolify auto-deploys
+```
+
+### 9.2 Logs Access
+
+Via Coolify UI: Services → [container] → Logs
+
+### 9.3 Database Access
+
+```bash
+# Via Coolify terminal
+docker exec -it [postgres-container] psql -U postgres -d directus
+```
+
+### 9.4 Fresh Install (CAUTION)
+
+Set in Coolify environment:
+```
+FORCE_FRESH_INSTALL=true
+```
+Deploys, wipes DB, runs schema. **Data loss warning**.
+
+### 9.5 Health Checks
+
+| Service | Endpoint | Expected |
+|---------|----------|----------|
+| Directus | `/server/health` | 200 OK |
+| Frontend | `/` | 200 OK |
+
+---
+
+## 10. Critical Files
+
+| File | Purpose | Change Impact |
+|------|---------|---------------|
+| `complete_schema.sql` | Database schema | Requires fresh install |
+| `docker-compose.yaml` | Infrastructure | Requires redeploy |
+| `frontend/src/lib/schemas.ts` | TypeScript types | Build failure if wrong |
+| `frontend/src/lib/directus/client.ts` | API client | Connectivity issues |
+| `start.sh` | Directus boot | Startup failure |
+
+---
+
+## 11. Team Onboarding Checklist
+
+### For New Developers
+- [ ] Clone repo, run `npm install` in frontend
+- [ ] Copy `.env.example` to `.env`
+- [ ] Run `docker-compose up -d` (or connect to staging)
+- [ ] Run `npm run dev` in frontend
+- [ ] Access `http://localhost:4321/admin`
+- [ ] Read `docs/DEVELOPER_GUIDE.md`
+
+### For Technical Leadership
+- [ ] Review this document
+- [ ] Review `docs/TECHNICAL_ARCHITECTURE.md`
+- [ ] Review `docs/DATABASE_SCHEMA.md`
+- [ ] Access Coolify dashboard
+- [ ] Access Directus admin
+- [ ] Review deployment history in Coolify