📚 Comprehensive Documentation Suite (Mil-Spec/IBM Redbook Style)
INVESTOR DOCUMENTATION: - INVESTOR_BRIEF.md: Executive summary, capacity metrics, architecture overview - PLATFORM_CAPABILITIES.md: 5 modules, 27 subcomponents catalog - TECHNICAL_ARCHITECTURE.md: Mermaid diagrams, data flows, extension points CTO/SENIOR DEV ONBOARDING: - CTO_ONBOARDING.md: System overview, security model, operational runbook - DEVELOPER_GUIDE.md: Clone→Run→Deploy workflow, debugging - API_REFERENCE.md: 30+ endpoints with request/response specs - DATABASE_SCHEMA.md: 30+ tables in Harris Matrix order - COMPONENT_LIBRARY.md: 182 React components catalog REFERENCE: - GLOSSARY.md: 70+ platform terms defined - ADMIN_PAGES_GUIDE.md: 66 admin pages by module Style: BLUF format, zero marketing fluff, high-density tables
This commit is contained in:
cawcenter
392
docs/TECHNICAL_ARCHITECTURE.md
Normal file
392
docs/TECHNICAL_ARCHITECTURE.md
Normal file
@@ -0,0 +1,392 @@
|
||||
# TECHNICAL ARCHITECTURE: Spark Platform
|
||||
|
||||
> **BLUF**: Spark uses Astro SSR + React Islands frontend, Directus headless CMS backend, PostgreSQL with PostGIS for data, and Redis-backed BullMQ for async processing. Multi-tenant via site_id foreign keys.
|
||||
|
||||
---
|
||||
|
||||
## 1. System Diagram
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Client Layer"
|
||||
Browser[Browser]
|
||||
PWA[PWA Service Worker]
|
||||
end
|
||||
|
||||
subgraph "Edge Layer"
|
||||
Traefik[Traefik Reverse Proxy]
|
||||
end
|
||||
|
||||
subgraph "Application Layer"
|
||||
Frontend["Astro SSR<br/>Port 4321"]
|
||||
Directus["Directus CMS<br/>Port 8055"]
|
||||
end
|
||||
|
||||
subgraph "Data Layer"
|
||||
PostgreSQL["PostgreSQL 16<br/>+ PostGIS 3.4"]
|
||||
Redis["Redis 7<br/>Sessions + Queue"]
|
||||
end
|
||||
|
||||
subgraph "Extension Layer"
|
||||
Endpoints["Custom Endpoints<br/>/god/*"]
|
||||
Hooks["Event Hooks<br/>on:create, on:update"]
|
||||
end
|
||||
|
||||
Browser --> Traefik
|
||||
PWA --> Traefik
|
||||
Traefik -->|"/*.admin, /api/*"| Frontend
|
||||
Traefik -->|"/items/*, /collections/*"| Directus
|
||||
Frontend -->|"REST API"| Directus
|
||||
Directus --> PostgreSQL
|
||||
Directus --> Redis
|
||||
Directus --> Endpoints
|
||||
Directus --> Hooks
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Component Specifications
|
||||
|
||||
### 2.1 Frontend (Astro)
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Framework | Astro 4.7 |
|
||||
| Rendering | SSR (Server-Side Rendering) |
|
||||
| Hydration | Islands Architecture (partial) |
|
||||
| UI Library | React 18.3 |
|
||||
| Styling | Tailwind CSS 3.4 |
|
||||
| State | React Query + Zustand |
|
||||
| Build | Vite |
|
||||
|
||||
**SSR URL Detection Logic**:
|
||||
```typescript
|
||||
const isServer = import.meta.env.SSR || typeof window === 'undefined';
|
||||
const DIRECTUS_URL = isServer
|
||||
? 'http://directus:8055' // Docker internal
|
||||
: import.meta.env.PUBLIC_DIRECTUS_URL; // Public HTTPS
|
||||
```
|
||||
|
||||
**Rationale**: Server-side requests use Docker network DNS. Browser requests use public URL.
|
||||
|
||||
### 2.2 Backend (Directus)
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Version | Directus 11 |
|
||||
| API | REST + GraphQL |
|
||||
| Auth | JWT + Static Tokens |
|
||||
| Storage | Local filesystem (uploads volume) |
|
||||
| Extensions | Endpoints + Hooks |
|
||||
|
||||
**Extension Structure**:
|
||||
```
|
||||
directus-extensions/
|
||||
├── endpoints/
|
||||
│ ├── god-schema/ # Schema operations
|
||||
│ ├── god-data/ # Bulk data ops
|
||||
│ └── god-utils/ # Utility endpoints
|
||||
└── hooks/
|
||||
├── work-log/ # Activity logging
|
||||
└── cache-bust/ # Invalidation
|
||||
```
|
||||
|
||||
### 2.3 Database (PostgreSQL)
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Version | PostgreSQL 16 |
|
||||
| Extensions | uuid-ossp, pgcrypto, PostGIS 3.4 |
|
||||
| Collections | 30+ tables |
|
||||
| Schema Order | Harris Matrix (Foundation → Walls → Roof) |
|
||||
|
||||
**Schema Dependency Order**:
|
||||
|
||||
| Batch | Tables | Dependencies |
|
||||
|-------|--------|--------------|
|
||||
| 1: Foundation | sites, campaign_masters, avatar_*, geo_*, offer_blocks | None |
|
||||
| 2: Walls | generated_articles, generation_jobs, pages, posts, leads, headline_*, content_* | Batch 1 only |
|
||||
| 3: Roof | link_targets, globals, navigation | Batch 1-2 |
|
||||
|
||||
### 2.4 Cache/Queue (Redis)
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Version | Redis 7 |
|
||||
| Mode | Append-only (persistent) |
|
||||
| Uses | Session cache, BullMQ backing |
|
||||
|
||||
**Queue Configuration**:
|
||||
```typescript
|
||||
// BullMQ job options
|
||||
{
|
||||
attempts: 3,
|
||||
backoff: { type: 'exponential', delay: 1000 },
|
||||
removeOnComplete: 100,
|
||||
removeOnFail: 1000
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Data Flow
|
||||
|
||||
### 3.1 Page Request Flow
|
||||
|
||||
```
|
||||
1. Browser → GET /blog/article-slug
|
||||
2. Traefik → Route to Frontend (port 4321)
|
||||
3. Astro SSR → Determine site from domain
|
||||
4. Astro → GET http://directus:8055/items/posts?filter[slug]=...
|
||||
5. Directus → PostgreSQL query
|
||||
6. Directus → Return JSON
|
||||
7. Astro → Render HTML with React components
|
||||
8. Astro → Return HTML to browser
|
||||
9. Browser → Hydrate interactive islands
|
||||
```
|
||||
|
||||
### 3.2 Article Generation Flow
|
||||
|
||||
```
|
||||
1. Admin → POST /api/seo/generate-article
|
||||
2. Astro API → Create generation_job record
|
||||
3. Astro API → Queue job in BullMQ
|
||||
4. BullMQ Worker → Dequeue job
|
||||
5. Worker → Fetch campaign config from Directus
|
||||
6. Worker → Compute Cartesian products
|
||||
7. Worker → For each permutation:
|
||||
a. Replace tokens
|
||||
b. Process spintax
|
||||
c. Generate SEO meta
|
||||
d. Create generated_articles record
|
||||
8. Worker → Update job status: completed
|
||||
9. Admin → Kanban reflects new articles
|
||||
```
|
||||
|
||||
### 3.3 Multi-Tenant Request Isolation
|
||||
|
||||
```
|
||||
1. Request → https://client-a.example.com/page
|
||||
2. Middleware → Extract hostname
|
||||
3. Middleware → Query sites WHERE url LIKE %hostname%
|
||||
4. Middleware → Set site_id in context
|
||||
5. All queries → Filter by site_id
|
||||
6. Response → Only tenant data returned
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. API Surface
|
||||
|
||||
### 4.1 Public Endpoints
|
||||
|
||||
| Endpoint | Method | Purpose |
|
||||
|----------|--------|---------|
|
||||
| `/api/lead` | POST | Form submission |
|
||||
| `/api/forms/submit` | POST | Generic form handler |
|
||||
| `/api/track/pageview` | POST | Analytics |
|
||||
| `/api/track/event` | POST | Custom events |
|
||||
| `/api/track/conversion` | POST | Conversion recording |
|
||||
|
||||
### 4.2 Admin Endpoints
|
||||
|
||||
| Endpoint | Method | Auth | Purpose |
|
||||
|----------|--------|------|---------|
|
||||
| `/api/seo/generate-headlines` | POST | Token | Spintax permutation |
|
||||
| `/api/seo/generate-article` | POST | Token | Article creation |
|
||||
| `/api/seo/approve-batch` | POST | Token | Bulk approval |
|
||||
| `/api/seo/publish-article` | POST | Token | Single publish |
|
||||
| `/api/seo/scan-duplicates` | POST | Token | Duplicate detection |
|
||||
| `/api/seo/insert-links` | POST | Token | Internal linking |
|
||||
| `/api/seo/process-queue` | POST | Token | Queue advancement |
|
||||
| `/api/campaigns` | GET/POST | Token | Campaign CRUD |
|
||||
| `/api/admin/import-blueprint` | POST | Token | Site import |
|
||||
| `/api/admin/worklog` | GET | Token | Activity log |
|
||||
|
||||
### 4.3 God-Mode Endpoints
|
||||
|
||||
| Endpoint | Method | Header | Purpose |
|
||||
|----------|--------|--------|---------|
|
||||
| `/god/schema/collections/create` | POST | X-God-Token | Create collection |
|
||||
| `/god/schema/relations/create` | POST | X-God-Token | Create relation |
|
||||
| `/god/schema/snapshot` | GET | X-God-Token | Export schema YAML |
|
||||
| `/god/data/bulk-insert` | POST | X-God-Token | Mass data insert |
|
||||
|
||||
---
|
||||
|
||||
## 5. Authentication Model
|
||||
|
||||
### 5.1 Directus Auth
|
||||
|
||||
| Method | Use Case |
|
||||
|--------|----------|
|
||||
| JWT | User login sessions |
|
||||
| Static Token | API integrations |
|
||||
| God-Mode Token | Administrative operations |
|
||||
|
||||
### 5.2 Token Hierarchy
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ GOD_MODE_TOKEN │ ← Full schema access
|
||||
│ X-God-Token header │
|
||||
└─────────────────────────────────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────┐
|
||||
│ DIRECTUS_ADMIN_TOKEN │ ← All collections CRUD
|
||||
│ Authorization: Bearer │
|
||||
└─────────────────────────────────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────┐
|
||||
│ Site-Scoped Token │ ← Single site access
|
||||
│ Generated per tenant │
|
||||
└─────────────────────────────────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────┐
|
||||
│ Public Access │ ← Read-only published
|
||||
│ No token required │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Security Configuration
|
||||
|
||||
### 6.1 CORS Policy
|
||||
|
||||
```yaml
|
||||
CORS_ORIGIN: 'https://spark.jumpstartscaling.com,https://launch.jumpstartscaling.com,http://localhost:4321'
|
||||
CORS_ENABLED: 'true'
|
||||
```
|
||||
|
||||
### 6.2 Rate Limiting
|
||||
|
||||
```yaml
|
||||
RATE_LIMITER_ENABLED: 'false' # Disabled for internal use
|
||||
```
|
||||
|
||||
### 6.3 Payload Limits
|
||||
|
||||
```yaml
|
||||
MAX_PAYLOAD_SIZE: '500mb' # Large batch operations
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Deployment Configuration
|
||||
|
||||
### 7.1 Docker Compose Services
|
||||
|
||||
| Service | Image | Port | Volume |
|
||||
|---------|-------|------|--------|
|
||||
| postgresql | postgis/postgis:16-3.4-alpine | 5432 | postgres-data-fresh |
|
||||
| redis | redis:7-alpine | 6379 | redis-data |
|
||||
| directus | directus/directus:11 | 8055 | directus-uploads |
|
||||
| frontend | Built from Dockerfile | 4321 | None |
|
||||
|
||||
### 7.2 Environment Variables
|
||||
|
||||
| Variable | Purpose | Where Set |
|
||||
|----------|---------|-----------|
|
||||
| PUBLIC_DIRECTUS_URL | Client-side API URL | docker-compose.yaml |
|
||||
| DIRECTUS_ADMIN_TOKEN | SSR API auth | Coolify secrets |
|
||||
| GOD_MODE_TOKEN | Schema operations | Coolify secrets |
|
||||
| FORCE_FRESH_INSTALL | Wipe + rebuild schema | Coolify secrets |
|
||||
| CORS_ORIGIN | Allowed origins | docker-compose.yaml |
|
||||
|
||||
### 7.3 Coolify Labels
|
||||
|
||||
```yaml
|
||||
labels:
|
||||
coolify.managed: 'true'
|
||||
coolify.name: 'directus'
|
||||
coolify.fqdn: 'spark.jumpstartscaling.com'
|
||||
coolify.port: '8055'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Extension Points
|
||||
|
||||
### 8.1 Adding New Collections
|
||||
|
||||
1. Define in `complete_schema.sql` (Harris Matrix order)
|
||||
2. Add TypeScript interface to `schemas.ts`
|
||||
3. Create API endpoint if needed
|
||||
4. Add admin page component
|
||||
|
||||
### 8.2 Adding New Blocks
|
||||
|
||||
1. Create component in `frontend/src/components/blocks/`
|
||||
2. Register in `BlockRenderer.tsx` switch statement
|
||||
3. Add schema to Page Builder config
|
||||
|
||||
### 8.3 Adding New Endpoints
|
||||
|
||||
1. Create file in `frontend/src/pages/api/`
|
||||
2. Export async handler function
|
||||
3. Add to API_REFERENCE.md
|
||||
|
||||
### 8.4 Adding Custom Directus Extensions
|
||||
|
||||
1. Create in `directus-extensions/endpoints/` or `hooks/`
|
||||
2. Restart Directus container
|
||||
3. Extensions auto-load from mounted volume
|
||||
|
||||
---
|
||||
|
||||
## 9. Performance Considerations
|
||||
|
||||
### 9.1 SSR Caching
|
||||
|
||||
| Strategy | Implementation |
|
||||
|----------|----------------|
|
||||
| ISR | Not used (dynamic content) |
|
||||
| Edge Cache | Traefik level (CDN potential) |
|
||||
| API Cache | Redis TTL on queries |
|
||||
|
||||
### 9.2 Database Optimization
|
||||
|
||||
| Technique | Application |
|
||||
|------------|-------------|
|
||||
| Indexes | FK columns, status, slug |
|
||||
| Pagination | Offset-based with limits |
|
||||
| Field Selection | Only request needed fields |
|
||||
|
||||
### 9.3 Bundle Optimization
|
||||
|
||||
| Technique | Implementation |
|
||||
|-----------|----------------|
|
||||
| Islands | Only hydrate interactive components |
|
||||
| Code Splitting | Vite automatic chunks |
|
||||
| Compression | Brotli via Astro adapter |
|
||||
|
||||
---
|
||||
|
||||
## 10. Monitoring & Logging
|
||||
|
||||
### 10.1 Log Locations
|
||||
|
||||
| Service | Location |
|
||||
|---------|----------|
|
||||
| Directus | Container stdout (Coolify UI) |
|
||||
| Frontend | Container stdout (Coolify UI) |
|
||||
| PostgreSQL | Container stdout |
|
||||
|
||||
### 10.2 Health Checks
|
||||
|
||||
| Service | Endpoint | Interval |
|
||||
|---------|----------|----------|
|
||||
| PostgreSQL | pg_isready | 5s |
|
||||
| Redis | redis-cli ping | 5s |
|
||||
| Directus | /server/health | 10s |
|
||||
|
||||
### 10.3 Work Log Table
|
||||
|
||||
```sql
|
||||
SELECT * FROM work_log
|
||||
ORDER BY timestamp DESC
|
||||
LIMIT 100;
|
||||
```
|
||||
|
||||
Fields: action, entity_type, entity_id, details, level, user, timestamp
|
||||
Reference in New Issue
Block a user