The Runtime Theory
mediumServerInternals#profiling#distributed-tracing#queueing

Why Might a Service Be Slow When CPU Usage Is Low?

A performance prompt about queues, connection pools, lock waits, network latency, tail behavior, and tracing the full request.

TRT practice prompt — not a verified question from a named employer.

The Runtime Theory Team1 min read

A strong answer

Low CPU means the process is not spending much time executing instructions; it does not mean requests are progressing. They may be waiting for a database connection, lock, remote service, disk, rate-limit token, or a worker slot. A queue can grow while CPUs remain idle if the constrained resource is elsewhere.

I would trace one slow request and split its time into ingress, connection acquisition, database execution, downstream calls, application compute, and response transfer. Compare histograms and queue depth, not only averages. Check whether p50 is stable while p95 or p99 grows, which can indicate contention or a small number of long waits.

If the database pool is saturated, increasing its size may push more concurrent work into a database that already has limited capacity. First identify why connections remain occupied, inspect transaction and query duration, and set bounded acquisition timeouts.

Follow-up direction

Propose a measurement that could falsify the pool-wait hypothesis before changing a production pool size.

This answer walks

Practice follow-ups

  1. 01Which spans would you add to distinguish pool wait from query execution?
  2. 02How would queue depth and tail latency change your diagnosis?
  3. 03Why can increasing a connection pool make the database less stable?

One dispatch a week

The trace behind each question, the tradeoff that explains it, and one technical dispatch per week — no noise.

One technical dispatch per week. No noise.

Not started

Sign in to save your learning progress.

Sign in to save