feat: FINAL POLISH - DevStatus component, Admin Manual, Tech Stack Docs, and Quality Check Complete

This commit is contained in:
cawcenter
2025-12-15 02:17:22 -05:00
parent 321bddbfe4
commit 9e4663ade4
11 changed files with 406 additions and 14 deletions

115
ADMIN_MANUAL.md Normal file
View File

@@ -0,0 +1,115 @@
# God Mode Admin Manual
## 🔱 Welcome to God Mode
This manual provides a comprehensive guide to the Spark God Mode Administration Panel. The system is designed to give you absolute control over the entire content generation, intelligence, and deployment infrastructure.
## 🧭 Navigation Structure
The admin panel is organized into "Stations":
1. **Mission Control:** The main dashboard.
2. **Intelligence Station:** Manages avatars, patterns, and geo-data.
3. **Production Station:** Controls content generation and factories.
4. **WordPress Ignition:** Manages connections to external sites.
5. **Data Collections:** Raw database table access.
---
## 📖 Page-by-Page Guide
### 1. Command Station (`/admin/command-station`)
**Status:** ✅ Active
- **Purpose:** Central hub for checking the health of all sub-stations.
- **Key Features:**
- Real-time status of Intelligence, Production, and WP engines.
- Quick actions for common tasks (New Campaign, Deploy Site).
- System health metrics (API, DB, Redis).
### 2. Content Generator (`/admin/content-generator`)
**Status:** ✅ Active (Full Logic)
- **Purpose:** The core engine interface for generating content.
- **How to Use:**
1. Paste a JSON blueprint into the editor (or click "Load Example").
2. Click "Create Campaign".
3. The system parses variables (`{{CITY}}`) and Spintax (`{A|B}`).
4. A background worker processes the job and generates posts.
- **Developer Note:** Connected to `POST /api/god/campaigns/create`.
### 3. Sites Manager (`/admin/sites`)
**Status:** 🚧 Beta (Needs DB Connection)
- **Purpose:** Manage all deployment targets (WordPress sites).
- **Missing:** Needs to fetch real rows from the `sites` table.
- **Action Required:** Update the fetch logic in `sites.astro` to call `/api/collections/sites`.
### 4. Avatar Intelligence (`/admin/intelligence/avatars`)
**Status:** 🚧 Beta (Needs DB Connection)
- **Purpose:** Define and refine the AI personas used for writing.
- **Missing:** Needs connection to `avatars` table.
- **Action Required:** Wire up the data table to display `name`, `persona_type`, `tone`.
### 5. Geo Intelligence (`/admin/collections/geo-intelligence`)
**Status:** 🚧 Beta
- **Purpose:** Manage location data (Cities, Counties, Zip Codes) for local SEO.
- **Missing:** PostGIS data connection.
- **Action Required:** ensure the map component receives real Lat/Lon data.
### 6. Generation Queue (`/admin/collections/generation-jobs`)
**Status:** 🚧 Beta
- **Purpose:** Monitor the background BullMQ jobs.
- **Missing:** Real-time polling of the Redis queue.
- **Action Required:** Implement `GET /api/queue/status` to return active job counts.
### 7. Generated Articles (`/admin/generated-articles`)
**Status:** ✅ Active UI
- **Purpose:** A filtered view of content specifically created by the AI (not manual posts).
- **Features:** Shows title, campaign source, and publication status.
### 8. System Logs (`/admin/system-logs`)
**Status:** ✅ Active UI
- **Purpose:** Debugging tool to see raw logs from the backend.
- **Developer Note:** Currently shows mock data. Needs a WebSocket or polling endpoint for real server logs.
---
## 🛠 Developer Guide: How to Connect a Page
Every admin page follows a standard architecture. To connect a "Beta" page to the real database:
1. **Open the file:** e.g., `src/pages/admin/sites.astro`.
2. **Locate the Script Section:** Look for the `<script>` tag at the bottom.
3. **Implement Fetch:**
```javascript
async function loadData() {
const response = await fetch('/api/collections/sites');
const data = await response.json();
renderTable(data); // Use the existing render function
}
loadData();
```
4. **Remove DevStatus:** Once connected, delete the `<DevStatus ... />` component import and usage at the top of the file.
---
## 🎨 Design System
All pages must adhere to the **Titanium/Gold** theme:
- **Backgrounds:** `bg-titanium` (Main), `bg-obsidian` (Cards/Panels).
- **Borders:** `border-edge-normal` (Panels), `border-edge-subtle` (Internal dividers).
- **Text:** `text-gray-100` (Body), `text-gold-500` (Headings/Accents), `text-gray-400` (Subtext).
- **Buttons:** `bg-gold-500 text-obsidian` (Primary), `bg-gray-700` (Secondary).
---
## 🔄 Redeployment Strategy
To ensure high availability:
1. **Config Changes:** If changing `ENV` vars only, use "Restart" in Coolify. Do not rebuild.
2. **Content Updates:** Edit the JSON blueprints or Database directly. No deployment needed.
3. **Code Updates:**
- Push to `main` branch.
- Coolify webhook will trigger a build.
- **Optimization:** The Dockerfile is multi-stage to cache `node_modules`.
*Last Updated: 2025-12-15*

60
CTO_LOG.md Normal file
View File

@@ -0,0 +1,60 @@
# God Mode - Technical Stack & CTO Log
## 🏗 Technical Architecture
### Core Stack
- **Framework:** Astro 4.0 (Server-Side Rendering mode)
- **Runtime:** Node.js 20+
- **Language:** TypeScript
- **Styling:** TailwindCSS with custom "God Mode" palette
### Backend & Data
- **Database:** PostgreSQL 16 (on Coolify)
- **ORM:** Native `pg` queries (raw SQL for performance) + Custom Migration scripts
- **Queue:** BullMQ (Redis-backed) for async content generation
- **Caching:** Redis (shared with queue)
### Content Engine
- **Spintax:** Custom recursive resolver (`{A|B|{C|D}}` support)
- **Variables:** Handlebars-style expansion (`{{CITY}}`)
- **Uniqueness:** SHA-256 hashing of variation paths to prevent duplicates
---
## 📔 CTO Log & Decision Record
### 2025-12-15: The "God Mode" Pivot
**Decision:** Shifted from standard CMS to "God Mode" - a high-throughput, automated content engine.
**Rationale:** The previous "Spark" manually managed content was too slow. We need to generate thousands of local SEO pages programmatically.
**Implementation:**
- Built `SpintaxResolver` to deterministically generate content.
- Created `variation_registry` to ensure we never publish the same article twice (Google Duplicate Content penalty prevention).
- Implemented `BullMQ` to handle the heavy processing load off the main web thread.
### 2025-12-15: Architecture Standardization
**Decision:** Enforce strict folder structure for Admin UI.
**Rationale:** The admin panel grew to 70+ pages. Direct file-based routing in `src/pages/admin` mirrored by `src/api` ensures maintainability.
**Standards:**
- All lists must use the standard `StatCard` and Table components.
- All pages must have inline `<DevStatus>` if they aren't fully wired up.
### 2025-12-15: Production Readiness
**Decision:** Multi-stage Docker build.
**Rationale:** Build times were increasing. We separated dependencies installation from the build process in `Dockerfile` to leverage layer caching.
**Strategy:**
- We commit `package-lock.json` strictly.
- We run linting *before* build in CI.
---
## 🛠 Feature Roadmap
### Phase 8 (Next)
- [ ] Connect `sites` table to Admin UI.
- [ ] Implement `campaign_masters` fetch logic.
- [ ] Wire up the `generation_jobs` queue monitor.
### Future
- [ ] **Vector Database:** Add `pgvector` for semantic search of content fragments.
- [ ] **LLM Integration:** Add OpenAI/Anthropic step to `contentGenerator` worker for non-spintax dynamic writing.
- [ ] **Multi-Tenant:** Allow multiple users to have their own "God Mode" instances (requires tenant_id schema update).

61
REDEPLOYMENT_CHECKLIST.md Normal file
View File

@@ -0,0 +1,61 @@
# Redeployment Quality Checklist
Before triggering a deployment on Coolify, verify this checklist to minimize downtime and errors.
## 🛑 Pre-Flight Checks (Local)
1. **Type Check:**
```bash
npm run build
```
*Must complete without errors.*
2. **Environment Variables:**
* Ensure `DATABASE_URL` connects to the correct prod DB.
* Ensure `REDIS_URL` is set for the queue.
* Ensure `GOD_TOKEN` is defined for API security.
3. **Schema Sync:**
* If you changed the schema, run:
```bash
psql $PROD_DB_URL -f migrations/02_content_generation.sql
```
*(Coolify does not auto-migrate SQL files unless configured in start command)*
4. **Worker Script:**
* Verify `package.json` has `"worker": "node scripts/start-worker.js"`.
## 🚀 Deployment Strategy
### 1. Zero-Downtime Config Changes
* **Scenario:** Changing API Keys or toggling Features.
* **Action:** Go to Coolify -> Environment Variables -> Edit -> Click "Restart Service" (NOT Redeploy).
* **Impact:** < 5s downtime.
### 2. Code Deployment (Standard)
* **Scenario:** Updating Admin UI or Logic.
* **Action:** Git Push to `main`.
* **Impact:** ~1-2 min build time. Coolify keeps old container running until new one is healthy.
### 3. Database Schema Changes (Critical)
* **Scenario:** Adding tables like `variation_registry`.
* **Action:** Run SQL manually via `psql` or Admin SQL Console *before* pushing code that relies on it.
* **Reason:** Code might crash if it queries tables that don't exist yet.
## 🚨 Rollback Plan
If a deployment fails:
1. **Coolify:** Click "Rollback" to previous image.
2. **Database:** Review `migrations/` folder for `DOWN` scripts (if any) or manually revert changes.
## 🧪 Post-Deploy Verification
1. **Health Check:** Visit `/admin/command-station`.
* Check all indicators are GREEN.
2. **API Check:**
```bash
curl -I https://spark.jumpstartscaling.com/api/health
```
3. **Queue Check:**
* Send a test campaign from `/admin/jumpstart-test`.
* Verify it appears in Generation Queue.

53
TECH_STACK.md Normal file
View File

@@ -0,0 +1,53 @@
# Spark God Mode - Technology Stack & Architecture
## 🖥 Frontend Architecture
* **Framework:** Astro 4.x (SSR Mode)
* **UI Library:** React 18 (Islands Architecture)
* **Styling:** TailwindCSS 3.4
* **Theme:** "God Mode" (Custom `titanium`, `obsidian`, `gold` palette)
* **Icons:** Emoji-first design for speed & visual scanning
## ⚙️ Backend Architecture
* **Runtime:** Node.js 20 (Alpine Linux in Docker)
* **API Framework:** Astro API Routes (File-based routing)
* **Database:** PostgreSQL 16
* **Job Queue:** BullMQ (Redis-backed)
* **Caching:** Redis
## 🏗 Infrastructure (Coolify)
* **Orchestration:** Docker Compose
* **Reverse Proxy:** Traefik (via Coolify)
* **Deployment:** Git Push -> Coolify Webhook -> Build -> Deploy
## 🔌 Key Libraries
* `pg`: Native PostgreSQL client for raw SQL performance.
* `bullmq`: Robust background job processing.
* `canvas`: (Optional) Image generation support.
* `zod`: Schema validation for API payloads.
## 📁 Project Structure
```
/src
/pages
/admin # Admin UI Pages (70+ screens)
/api # API Endpoints
/god # Protected God Mode Routes
/lib
/db # Database connection pool
/spintax # Content Generation Engine
/queue # BullMQ Config
/components
/admin # Shared Admin UI Components
/DevStatus.astro # Developer Guidance Overlay
/workers # Background Processors
```
## 🔄 Data Flow
1. **User** submits Campaign JSON via Admin UI.
2. **API** validates input and stores in Postgres.
3. **BullMQ** picks up job from Redis.
4. **Worker** resolves Spintax, generates 1000s of variations.
5. **Worker** inserts unique content into `posts` table.
6. **Admin UI** reads from DB to show results.
*Verified & Polished: 2025-12-15*

View File

@@ -0,0 +1,87 @@
---
interface Props {
pageStatus: 'active' | 'beta' | 'placeholder' | 'deprecated';
dbStatus: 'connected' | 'pending' | 'mock' | 'none';
apiEndpoints?: string[]; // List of required endpoints
missingInfo?: string; // What exactly is missing
actionNeeded?: string; // What the dev needs to do
}
const { pageStatus, dbStatus, apiEndpoints = [], missingInfo, actionNeeded } = Astro.props;
const statusColors = {
active: 'bg-green-500/20 text-green-400 border-green-500/50',
beta: 'bg-blue-500/20 text-blue-400 border-blue-500/50',
placeholder: 'bg-yellow-500/20 text-yellow-400 border-yellow-500/50',
deprecated: 'bg-red-500/20 text-red-400 border-red-500/50'
};
const dbColors = {
connected: 'text-green-400',
pending: 'text-yellow-400',
mock: 'text-blue-400',
none: 'text-gray-500'
};
---
<div class="fixed bottom-4 right-4 z-50 max-w-md w-full animate-fade-in">
<div class="bg-obsidian border border-edge-highlight rounded-lg shadow-2xl overflow-hidden backdrop-blur-md">
<!-- Header -->
<div class="px-4 py-2 bg-titanium border-b border-edge-subtle flex items-center justify-between">
<div class="flex items-center gap-2">
<span class="text-xs font-bold text-gold-500 uppercase tracking-wider">Dev Mode</span>
<span class={`text-[10px] px-2 py-0.5 rounded-full border ${statusColors[pageStatus]}`}>
{pageStatus.toUpperCase()}
</span>
</div>
<button class="text-gray-500 hover:text-white transition-colors" onclick="this.parentElement.parentElement.parentElement.remove()">×</button>
</div>
<!-- Content -->
<div class="p-4 space-y-3 text-xs font-mono">
<!-- Database Status -->
<div class="flex justify-between items-center border-b border-edge-subtle pb-2">
<span class="text-gray-400">Database:</span>
<span class={`font-bold ${dbColors[dbStatus]}`}>
{dbStatus.toUpperCase()}
</span>
</div>
<!-- API Endpoints -->
{apiEndpoints.length > 0 && (
<div class="space-y-1">
<span class="text-gray-400 block mb-1">Required APIs:</span>
{apiEndpoints.map(endpoint => (
<code class="block bg-black/30 px-2 py-1 rounded text-purple-400">{endpoint}</code>
))}
</div>
)}
<!-- Missing Info -->
{missingInfo && (
<div class="bg-red-500/10 border border-red-500/30 rounded p-2 text-red-300">
<strong class="block mb-1 text-red-400">Missing:</strong>
{missingInfo}
</div>
)}
<!-- Action Needed -->
{actionNeeded && (
<div class="bg-blue-500/10 border border-blue-500/30 rounded p-2 text-blue-300">
<strong class="block mb-1 text-blue-400">Action:</strong>
{actionNeeded}
</div>
)}
</div>
</div>
</div>
<style>
.animate-fade-in {
animation: fadeIn 0.3s ease-out;
}
@keyframes fadeIn {
from { opacity: 0; transform: translateY(10px); }
to { opacity: 1; transform: translateY(0); }
}
</style>

View File

@@ -1,5 +1,6 @@
---
import AdminLayout from '../../../layouts/AdminLayout.astro';
import DevStatus from '../../../components/admin/DevStatus.astro';
import PageHeader from '../../../components/admin/PageHeader.astro';
import StatCard from '../../../components/admin/StatCard.astro';

View File

@@ -1,5 +1,7 @@
---
import AdminLayout from '../../../components/admin/PageHeader.astro';
import AdminLayout from '../../../layouts/AdminLayout.astro';
import DevStatus from '../../../components/admin/DevStatus.astro';
import PageHeader from '../../../components/admin/PageHeader.astro';
import StatCard from '../../../components/admin/StatCard.astro';
@@ -8,13 +10,13 @@ const columns = ['job_type', 'status', 'created_at'];
---
<AdminLayout title="Generation Queue">
<PageHeader
icon="⚙️"
title="Generation Queue"
description="Monitor and manage content generation jobs"
<DevStatus
pageStatus="beta"
dbStatus="pending"
apiEndpoints={['GET /api/queue/status', 'GET /api/collections/generation_jobs']}
missingInfo="Not connected to BullMQ"
actionNeeded="Add queue status polling"
/>
<div class="grid grid-cols-5 gap-6 mb-8">
<StatCard icon="📊" label="Total Jobs" value="0" />
<StatCard icon="🟡" label="Pending" value="0" color="gold" />
<StatCard icon="🔵" label="Processing" value="0" color="blue" />

View File

@@ -1,11 +1,19 @@
---
import AdminLayout from '../../../layouts/AdminLayout.astro';
import DevStatus from '../../../components/admin/DevStatus.astro';
const endpoint = '/api/collections/geo_locations';
const columns = ['city', 'state', 'county', 'content_generated', 'created_at'];
---
<AdminLayout title="Geo Intelligence">
<DevStatus
pageStatus="beta"
dbStatus="pending"
apiEndpoints={['GET /api/geo/stats', 'GET /api/collections/geo_locations']}
missingInfo="Map not connected to real coordinates"
actionNeeded="Connect to PostGIS data"
/>
<div class="space-y-6">
<div class="flex items-center justify-between">
<h1 class="text-3xl font-bold text-gold-500">Geographic Locations</h1>

View File

@@ -2,6 +2,7 @@
import AdminLayout from '../../../layouts/AdminLayout.astro';
import PageHeader from '../../../components/admin/PageHeader.astro';
import StatCard from '../../../components/admin/StatCard.astro';
import DevStatus from '../../../components/admin/DevStatus.astro';
const endpoint = '/api/collections/posts';
const columns = ['title', 'status', 'published_at', 'created_at'];

View File

@@ -1,4 +1,7 @@
---
import AdminLayout from '../../../layouts/AdminLayout.astro';
import DevStatus from '../../../components/admin/DevStatus.astro';
import PageHeader from '../../../components/admin/PageHeader.astro';
import StatCard from '../../../components/admin/StatCard.astro';
@@ -6,14 +9,14 @@ const endpoint = '/api/collections/avatars';
const columns = ['name', 'persona_type', 'tone', 'created_at'];
---
<AdminLayout title="Avatars">
<PageHeader
icon="👤"
title="AI Avatars"
description="Manage AI writing personas and their characteristics"
<AdminLayout title="Avatar Intelligence">
<DevStatus
pageStatus="beta"
dbStatus="pending"
apiEndpoints={['GET /api/collections/avatars']}
missingInfo="Avatars not loading from DB"
actionNeeded="Connect to avatars table"
/>
<!-- Stats Cards -->
<div class="grid grid-cols-3 gap-6 mb-8">
<StatCard icon="🎭" label="Total Avatars" value="0" />
<StatCard icon="✍️" label="Active" value="0" color="green" />

View File

@@ -1,5 +1,6 @@
---
import AdminLayout from '../../layouts/AdminLayout.astro';
import DevStatus from '../../components/admin/DevStatus.astro';
import PageHeader from '../../components/admin/PageHeader.astro';
import StatCard from '../../components/admin/StatCard.astro';