Shipping an AI Content Platform on Someone Else's Rate Limits
Client: Product Build — SDT LTD · Industry: Marketing Technology / AI · Role: Tech Lead & Project Owner · Duration: Concurrent with SaaS Job Marketplace (SDT LTD)
The Challenge
SocialPro AI needed to generate and schedule content across multiple social platforms using OpenAI's API (text via ChatGPT, images via DALL·E) while running concurrently with a separate 23-person job-marketplace engagement. The core risk was OpenAI rate limits and cost: naive per-request calls during traffic spikes would throttle the whole platform and blow the API budget.
The Solution
Built a queued generation pipeline — a Node.js worker backed by a Redis job queue — that batches and throttles OpenAI calls behind a token-bucket rate limiter, with cached prompt/response pairs for repeated content patterns to cut redundant calls. Added a custom NLP scoring pass to flag low-quality generations for regeneration before they reached the scheduling queue, rather than after publishing. Kept the service decoupled from the job-marketplace codebase so it could be resourced and scaled independently.
Results & Impact
- Reduced OpenAI API cost per generated post by caching repeated prompt patterns
- Zero platform-wide throttling incidents after the token-bucket limiter shipped
- Unified dashboard live for multi-platform scheduling and analytics
- Delivered as a fully separate service without pulling engineering time from the concurrent job-marketplace project