How I Scaled a PHP Monolith to Handle 1,000 Concurrent Users Without Rewriting It
When I joined Onest Tech as Tech Lead, the flagship ERP was struggling at 80 concurrent users. Pages timed out during peak hours, support tickets about "the system is slow" outnumbered every other category combined, and the prevailing internal theory was that the platform had simply outgrown its architecture. The proposed fix, already informally agreed on before I arrived, was a full rewrite into microservices.
I asked for three weeks before committing to that plan, to actually profile the system rather than diagnose it from intuition. That turned out to be the most consequential decision on the project — not because a rewrite would have been wrong in principle, but because it would have solved the wrong problem, slowly and expensively.
Step 1: Measure Before You Guess
The instinct to blame architecture for performance problems is understandable — "monolith" has become shorthand for "old and slow" in a lot of engineering conversations. But instinct isn't data. We ran EXPLAIN ANALYZE against every query flagged as slow in the application's logs, and layered in APM tooling to trace request timing end to end.
What we found didn't match the prevailing theory at all: 80% of latency came from three N+1 query patterns in the inventory module — not from the monolithic architecture itself. A rewrite into microservices would have faithfully reproduced those same N+1 patterns in a new, more complicated shape, and likely added network latency between services on top of them.
The architecture wasn't the bottleneck. Three specific, fixable query patterns were. That distinction is the entire reason the next six weeks looked completely different from the twelve-month rewrite that had been assumed as the only path forward.
Step 2: Redis for the Right Things
Once the N+1 patterns were fixed at the query level, we still had two categories of load that didn't need to hit the database on every request at all:
- Read-heavy dashboard queries — the same aggregate numbers, requested by many users, changing slowly. We added a short TTL cache (60 seconds) for these, which is long enough to eliminate redundant computation but short enough that nobody notices stale data.
- Permission lookups — every single request was hitting the database to check what the current user was allowed to do, even though permissions change rarely compared to how often they're checked. We moved these to a persistent Redis cache, invalidated explicitly whenever a role or permission actually changes.
This alone cut database load by 40%. Note that neither of these is "cache everything" — caching the wrong thing (data that changes on every write, or that needs strong consistency) creates more bugs than it fixes. The rule we used: cache things that are read far more often than they're written, and where a few seconds of staleness genuinely doesn't matter.
Step 3: Database Connection Pooling
PHP-FPM, by default, was opening a fresh MySQL connection on every incoming request and tearing it down at the end. Under load, that connection setup/teardown overhead adds up in ways that don't show up clearly in a single request's profile but dominate aggregate throughput. Switching to connection pooling — reusing a small pool of persistent connections across requests rather than creating one per request — reduced connection overhead from double-digit milliseconds down to under 1ms per request.
# Before: new connection per request (simplified)
$pdo = new PDO($dsn, $user, $pass);
# After: pooled persistent connections via ProxySQL in front of MySQL
# php.ini / app config
PDO::ATTR_PERSISTENT => true
# ProxySQL handles the actual pooling and connection reuse
# across PHP-FPM workers, so the DB sees a stable, bounded
# number of real connections regardless of request volume.
Step 4: Horizontal Scaling with Session Offload
With the database no longer the bottleneck, the next ceiling was a single application server. Adding a second app server behind a load balancer sounds trivial, but it exposed a hidden assumption in the codebase: PHP sessions were being stored on local disk on each server. That works fine with one server — it silently breaks with two, because a user's session data might live on a server the load balancer doesn't route their next request to.
We moved sessions to Redis, which made the application genuinely stateless from the load balancer's point of view — any server could serve any request, because session state lived centrally rather than on whichever disk happened to handle the login request.
Why This Order Mattered
Horizontal scaling before fixing the N+1 queries and the connection overhead would have "worked" in the sense that it would have thrown more hardware at the problem — but it would have masked the underlying inefficiency rather than fixing it, and cost more to run indefinitely. Fixing the actual bottlenecks first meant the second server bought real headroom rather than just papering over waste.
Results
| Metric | Before | After (6 weeks) |
|---|---|---|
| Concurrent users supported | ~80 | 1,000+ |
| p95 response time | Multiple seconds | Under 400ms |
| Database load | Baseline | -40% |
| Migration downtime | — | Zero |
Best Practices That Held Up
- Profile before you architect. A rewrite is the most expensive possible fix for a bug you haven't identified yet.
- Cache selectively, based on read/write ratio and tolerance for staleness — not by reflexively caching everything that's slow.
- Fix the cheapest, most impactful problem first. Query patterns are cheaper to fix than architecture, and often account for more of the actual latency.
- Treat statelessness as a prerequisite for horizontal scaling, not an assumption — audit for hidden local state (sessions, file uploads, local caches) before adding a second server.
Common Mistakes
- Assuming "monolith" and "slow" are the same problem — they frequently aren't, and conflating them leads to expensive rewrites that don't fix the actual bottleneck.
- Adding caching without an invalidation strategy, which trades a slow-but-correct system for a fast-but-wrong one.
- Scaling horizontally before auditing for local state like disk-based sessions, file uploads, or in-process caches that silently break under a load balancer.
- Treating a performance project as a rewrite-or-nothing decision, when incremental fixes — measured and prioritized by actual impact — are very often the faster and lower-risk path.
FAQ
Q: How did you decide the rewrite wasn't necessary, given the pressure to modernize?
The profiling data made the case on its own. Once we could show that 80% of latency traced to three specific, fixable query patterns, the argument for a twelve-month rewrite lost its footing — the data didn't support the premise that the architecture itself was the bottleneck.
Q: Would microservices have been the right call eventually?
Possibly, for different reasons — team-scaling or deployment-independence reasons, not performance ones. But that's a separate decision from "the monolith is too slow," and conflating the two nearly led to solving a performance problem with an organizational-scaling tool.
Q: What would you profile first on a similarly "slow" legacy system?
Query patterns, always. N+1 queries are extremely common in ORM-heavy codebases that have grown organically, and they're usually the single highest-leverage fix available.
What Profiling Actually Looked Like Day to Day
It's worth being concrete about what "three weeks of profiling" meant in practice, because the phrase can sound like a vague placeholder for "we thought about it for a while." It wasn't. We instrumented the application with APM tooling that captured a trace for every request above a latency threshold, then manually reviewed a sample of the slowest traces each day, looking for repeated patterns rather than one-off anomalies.
The N+1 patterns in the inventory module weren't obvious from a single slow request — they only became visible once we compared dozens of traces and noticed the same shape of query repeated dozens of times within a single request, each one individually fast but collectively dominating the request's total time. That pattern-matching-across-many-traces step is the part that's easy to skip if you're debugging under pressure and looking for a single smoking gun instead.
Rolling Out the Fixes Without a Freeze
None of these four steps required taking the ERP offline. Query fixes were deployed incrementally, module by module, with before/after latency comparisons captured for each one so we had concrete evidence the fix helped before moving to the next. The Redis caching layer was introduced behind a feature flag, so a caching bug could be disabled instantly without a rollback deploy. Connection pooling and the load balancer addition were the only changes that touched infrastructure directly, and both were tested extensively in a staging environment that mirrored production traffic patterns before going live.
Communicating Progress to a Skeptical Stakeholder Group
Because the rewrite had already been informally agreed on before this project started, part of the work was communicating progress in a way that built confidence in the incremental approach rather than reading as delay. We shared concrete before/after numbers after each of the four steps — not vague progress updates — which made it easy for stakeholders to see the plan was working well before all six weeks had elapsed.
Conclusion
The rewrite would have taken roughly twelve months and introduced an entirely new set of bugs in an unfamiliar architecture. The optimization took six weeks and made the existing, well-understood codebase genuinely fast. The lesson generalizes well beyond this one ERP: profiling beats intuition every time, and the most disciplined thing you can do when a system feels architecturally broken is measure before you rebuild.