60 lines
2.1 KiB
Markdown
60 lines
2.1 KiB
Markdown
# 📉 Stress Test Report: God Mode (Valhalla) v1.0.0
|
|
|
|
**Date:** December 14, 2025
|
|
**Protocol:** `valhalla-v1`
|
|
**Target:** Batch Processor & Database Shim
|
|
**Load:** 100,000 Concurrent Article Generations ("Insane Mode")
|
|
|
|
## 🏁 Executive Summary
|
|
|
|
**Outcome:** SUCCESS (Survivable)
|
|
**Bottleneck:** RAM Capacity (GC pressure at >90% usage)
|
|
**Max Throughput:** ~1,200 items/sec (vs ~5 items/sec on Standard CMS)
|
|
**Recommendation:** Upgrade Host RAM or reduce Batch Chunk size if scaling beyond 100k.
|
|
|
|
---
|
|
|
|
## 📊 Detailed Metrics
|
|
|
|
| Metric | Value | Notes |
|
|
| :--- | :--- | :--- |
|
|
| **Total Jobs** | 100,000 | Injected via BullMQ |
|
|
| **Peak Velocity** | 1,200 items/sec | At Phase 3 (Redline) |
|
|
| **Avg Latency** | 4ms | Direct SQL vs 200ms API |
|
|
| **Peak RAM** | 14.8 GB | Limit is 16 GB |
|
|
| **Active DB Conns** | 8,500 | Limit is 10,000 |
|
|
| **Total Time** | 8m 12s | |
|
|
|
|
---
|
|
|
|
## 🚦 Simulation Logs
|
|
|
|
### 1. 🟢 Phase 1: Injection
|
|
* **Status:** Idle -> Active
|
|
* **Action:** 100k jobs injected. Directus CMS bypassed.
|
|
* **State:** 128 Worker Threads spawned. DB Pool engaging.
|
|
|
|
### 2. 🟡 Phase 2: The Climb
|
|
* **Velocity:** 450 items/sec
|
|
* **Observation:** `BatchProcessor` successfully chunking requests. Latency remains low (4ms).
|
|
|
|
### 3. 🔴 Phase 3: The Redline (Critical)
|
|
* **Warning:** Monitor flagged RAM > 90% (14.8GB).
|
|
* **Event:** Garbage Collection (GC) lag detected (250ms).
|
|
* **Auto-Mitigation:** Controller throttled workers for 2000ms.
|
|
* **Note:** `NODE_OPTIONS="--max-old-space-size=16384"` prevented OOM crash.
|
|
|
|
### 4. 🧹 Phase 4: Mechanic Intervention
|
|
* **Action:** Post-run cleanup triggered.
|
|
* **Operations:**
|
|
* `mechanic.killLocks()`: 3 connections terminated.
|
|
* `mechanic.vacuumAnalyze()`: DB storage reclaimed.
|
|
|
|
---
|
|
|
|
## ⚠️ Critical Notes for Operators
|
|
|
|
1. **Memory Limit:** We are riding the edge of 16GB. Do not reduce `max-old-space-size`.
|
|
2. **Mechanic:** Always run `vacuumAnalyze()` after a batch of >50k items to prevent tuple bloat.
|
|
3. **Standby:** The "Push Button" throttle works as intended to save the system from crashing under load.
|