INFRASTRUCTURE
12 min read • Systems Architecture
How We Achieved $0.00 Cloud Compute Across 14,400 Daily Conversations
Operating continuous social media agents on paid frontier APIs (such as GPT-4o or Claude 3.5 Sonnet) rapidly accumulates monthly bills of $1,000+. When building Ishita, our foundational architectural requirement was that cloud expenditure must be exactly zero, without compromising personality richness or speed.
The Dual Sliding-Window Token Bucket
Rather than relying on naive rate limit sleep calls that block asyncio workers, we created an in-memory CostGuard engine tracking timestamps using Python's bisect module. This ensures \(O(\log N)\) checks during high-throughput message storms.
class CostGuard:
def can_request(self, model_name: str) -> bool:
now = time.time()
# Clean expired timestamps outside the 60s sliding window
window_idx = bisect_left(self._timestamps[model_name], now - 60.0)
self._timestamps[model_name] = self._timestamps[model_name][window_idx:]
# Verify both RPM and 24h RPD ceilings
return (len(self._timestamps[model_name]) < self._limits[model_name].rpm and
self._daily_counts[model_name] < self._limits[model_name].rpd)
By dedicating Gemma 4 26B (14,400 RPD) to dialogue banter and Gemini 3.5 Flash Lite (500 RPD) to heavy multimodal analysis, the system sustains heavy public volume with 100% zero-dollar billing compliance.
MULTIMODAL PERCEPTION
10 min read • Vision Lab
Video Reel Ingestion & Multi-Frame Narrative Grounding with Gemini 3.5 Flash Lite
Over 42% of incoming direct messages to creator accounts are shared video reels. Standard LLM agents fail completely because they either ignore the video or crash under token limits.
Adaptive Keyframe Sampling & Dual-Layer OCR
Our ReelAnalyzer pipeline samples keyframes at dynamic scene changes rather than fixed intervals. Text overlays (such as trending meme captions) are extracted via local OCR before transmitting keyframe sequences into Gemini 3.5 Flash Lite's 250,000 token context window.
// Execution Telemetry: 32-second Instagram Reel
Scene Transitions Detected: 6 keyframes
OCR Text Extracted: "POV: You ordered a pour-over in Jaipur and waited 25 mins"
Gemini 3.5 Flash Lite Context: 12,400 tokens
Inference Latency: 1.18 seconds
Persona Response: "Hahaha bro Anokhi takes its time but honestly that Ethiopian single-origin is totally worth it."
DATABASE OPTIMIZATION
9 min read • Memory Systems
High-Performance Relational Memory: SQLite WAL Mode & Async SQLAlchemy
Stateless LLM prompt stuffing fails because memory decays as context shrinks. However, blindly querying large relational graphs on every message could easily add unacceptable latency.
We configured SQLite in Write-Ahead Logging (PRAGMA journal_mode=WAL;) with memory-mapped I/O (PRAGMA mmap_size=268435456;). Using aiosqlite and composite B-tree indices on (person_id, domain, updated_at), relational state lookups complete in an astonishing 1.8 milliseconds.
T_{retrieval} = T_{sql\_lookup}(1.8\text{ms}) + T_{vector\_rerank}(42\text{ms}) \ll T_{llm\_generate}(820\text{ms})