Userlog 24× Performance Fix
A 24× speed-up that came from fixing the query layer, not the symptom.
The user-log report became unusable on large libraries. Pagination hung the worker, and "go to last page" never returned. The easy read was "it's a slow screen" — but the real cause was deeper.
- 01
Isolated two compounding bugs: there was no SQL-level pagination on the grouped queries, so the report pulled the entire dataset through the job queue on every single page click — and an inverted slicing condition on top of that.
- 02
Pushed pagination down to where it belongs: LIMIT/OFFSET at the SQL layer, with the total-count wrapped in a subquery so counting didn't re-materialise the whole result set.
- 03
Added a UI timeframe selector that bounds the page size at the source, so the worst-case query can't be requested in the first place.
- 04
Chased a late regression on the QA environment to drifted PostgreSQL planner statistics on a multi-gigabyte table — and turned the fix (VACUUM ANALYZE + REINDEX) into a standing post-release maintenance pattern rather than a one-off.
- 05
Earlier groundwork on the same class of slowness ruled out the data path entirely by reproducing half a million log rows locally without triggering it — which is what pointed the investigation at database statistics instead of code.
- 06
Took three MR iterations to land cleanly — quality over speed.
P95 latency dropped from 15.4 seconds to 0.67 seconds — a roughly 24× improvement — and the operational maintenance pattern that surfaced along the way now protects the whole platform after each release.
- PostgreSQL
- Python
- EXPLAIN ANALYZE
RM#9008 (the rewrite, three MR iterations) and RM#8672 (the earlier root-cause work that seeded the maintenance pattern).