Overhead
The hooks on the I/O path run for every command, and ntprof records one command in 1000 by default. Sampled or not, every command passes the hooks, so the cost depends little on the sampling rate. Measured on the two servers of the use cases, both sides recording, three runs each, alternating with ntprof off:
| 4 KiB random read | NVMe/TCP | NVMe/RDMA |
|---|---|---|
| Queue depth 1, mean latency | 87.1 → 88.4 µs (+1.3 µs, +1.5 %) | 75.0 → 75.8 µs (+0.8 µs, +1.0 %) |
| 4 jobs × queue depth 32, throughput | 331.1k → 309.9k IOPS (−6.4 %) | 730.1k → 715.5k IOPS (−2.0 %) |
| 4 jobs × queue depth 32, mean latency | 386.3 → 412.7 µs | 175.0 → 178.6 µs |
On two machines each pays only for its own side. In the NVMe/TCP case at 4 × 32, fio and the host's nvme-tcp
share the same four CPUs, so ntprof's work there comes out of throughput; where CPUs have room the cost is mostly
the latency above.
Ways to pay less:
- Profile one side only:
ntprof recordon the host alone reports the network and the target together, as one row. - Filter:
--session,--io-typeand--sizekeep unwanted commands out of the records (the hooks still run). - Prefer fentry: kprobes cost more per call. ntprof uses them only where a module has no BTF, or a busy CPU would make attaching fentry wait.
- Leave
--exact-queuesoff: it updates queue counts on every I/O.