Recording

ntprof record attaches the hooks, samples commands until it stops, and prints a report. It needs root.

sudo ntprof record                      # the host, NVMe/TCP, every namespace, until Ctrl-C
sudo ntprof record -r both -d 10        # host and target on this machine, 10 seconds
sudo ntprof record -r target -o t.ntp   # the target side of a two-machine run
sudo ntprof record -t rdma -o h.ntp     # the host over NVMe/RDMA

Role and transport

Option	Meaning
-r host	the host (default)
-r target	the target; its records are reported joined with the host's, so save them with -o
-r both	host and target on one machine (loopback test beds)
-t tcp, -t rdma	the transport (default tcp)

Two machines

Each side records on its own clock; nothing is exchanged while recording. The two files are joined afterwards:

1. On the target: sudo ntprof record -r target -o target.ntp (start it first).
2. On the host: sudo ntprof record -o host.ntp -d 30 while the workload runs.
3. Stop the target last, copy one file to the other machine, and run ntprof report host.ntp target.ntp.

Both sides must use the same -s: they choose the same commands by hashing each command's ID, block address and length.

What to record

Option	Meaning
-s N	record one command in N (default 1000)
--session nvmeXnY	one namespace (all of its paths with native multipath)
--io-type read|write	only reads or only writes
--size 4K,128K	only these I/O sizes
-d SEC	stop after SEC seconds (default: Ctrl-C)

Output while recording

Every second (-i SEC), ntprof prints one line per group of commands (I/O type, size, namespace) with the number sampled, the command rate, the mean and p99 latency, and each stage's share. --offline leaves these out and prints only the final report.

                                                                          host                   target
    time  group                       n   cmds/s   mean us    p99 us   block  driver network  driver storage
    3.0s  read 4KiB nvme6n1         154    15.5k     73.40     80.73      1%      2%    98%*

This line comes from the host of a two-machine run. The host records without the target's records, so its network column holds the network and the target together and is marked *, and the target's columns stay empty. With -r both on one machine, every column is filled. The report that joins the two files has all five stages.

Files

-o FILE saves every record; ntprof report, ntprof compare and ntprof dump (CSV) read them later, on any machine. -j FILE writes the final report as JSON.

Busy CPUs

Attaching an fentry/fexit program waits for every runnable thread to switch out voluntarily, and a thread that never blocks (a polling thread, a fio job whose completions are always ready) holds that wait up. ntprof attaches its programs in parallel, checks first whether fentry can attach now, and falls back to kprobes if it cannot or if attaching takes more than 10 seconds. Each time it says why. --kprobes and --fentry force a flavour.