Skip to main content

Use cases

Each use case is one question about a slow NVMe-oF path and the ntprof run that answers it. All four were measured on the same two servers on 2026-10-15:

Serverstwo, 2 × Intel Xeon Gold 6526Y each (64 CPUs), one as host, one as target
NetworkNVIDIA ConnectX-7, 400 GbE RoCEv2, MTU 9000
SoftwareUbuntu 22.04, Linux 5.15 with MLNX OFED 24.10's nvme-tcp, nvme-rdma, nvmet-tcp, nvmet-rdma
Targetone Samsung PM9A3 (1.92 TB) exported through the kernel NVMe target (block back end)
Workloadfio 3.38, io_uring, direct I/O, on the host's NUMA node; reads on the first 64 GiB of the drive, writes on the next 64 GiB, both written sequentially first
ntprofboth sides recorded separately and joined; one command in 100 sampled (1000 at queue depth 32)
QuestionWhat ntprof showed
Is the SSD the bottleneck?A second writer on the drive: storage grows 2.7–2.8×, every other stage stays within 6 %.
Is the network the bottleneck?A bulk flow on the link: over NVMe/RDMA the network grows 1.9×, storage does not move; over NVMe/TCP a 24 Gb/s flow doubled the network in one run and left it alone in another.
Why does latency grow with queue depth?NVMe/TCP at queue depth 32: the network grows 9.5×, storage 1.1×.
Network or storage, per request?NVMe/RDMA: each request's data transfer and drive time from one run.