Hello, Adam.
Thanks for the reply.
On Sun, Mar 19, 2017 at 11:45:34 +1100, Adam Carter wrote:
> On Sun, Mar 19, 2017 at 3:36 AM, Alan Mackenzie <a...@muc.de> wrote:
> > I've just bought myself a Samsung NVMe 960 EVO M.2 SSD.
> > <snip>
> > Some timings:
> >
> > An emerge -puND @world (when there's nothing to merge) took 38.5s. With
> > my mirrored HDDs, this took 45.6s. (Though usually it takes nearer a
> > minute.)
> >
> > An emerge of Firefox took 34m23s, compared with 37m34s with the HDDs.
> > The lack of the sound of the HDD heads moving was either disconcerting
> > or a bit of a relief, I'm not sure which.
> >
> > Copying my email spool file (~110,000 entries, ~1.4 GB) from SSD -> SSD
> > took 6.1s. From HDD RAID -> HDD RAID it took 30.0s.
> >
> > <snip>
> >
> I was also hoping for more speed up when i got mine, but of course it only
> helps with the system is IO bound. Its great for loading VMs.
> It may be mandatory with NVM, but you can check multiqueue is setup/working
> with;
> # cat /proc/interrupts | egrep '(CPU|nvm)'
> CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
> CPU6 CPU7
> 30: 21596 0 0 0 0 0
> 0 0 PCI-MSI 1572864-edge nvme0q0, nvme0q1
> 40: 0 12195 0 0 0 0
> 0 0 PCI-MSI 1572865-edge nvme0q2
> 41: 0 0 12188 0 0 0
> 0 0 PCI-MSI 1572866-edge nvme0q3
> 42: 0 0 0 13696 0 0
> 0 0 PCI-MSI 1572867-edge nvme0q4
> 43: 0 0 0 0 11698 0
> 0 0 PCI-MSI 1572868-edge nvme0q5
> 44: 0 0 0 0 0 45820
> 0 0 PCI-MSI 1572869-edge nvme0q6
> 45: 0 0 0 0 0 0
> 10917 0 PCI-MSI 1572870-edge nvme0q7
> 46: 0 0 0 0 0 0
> 0 12865 PCI-MSI 1572871-edge nvme0q8
> If its not setup there'll be just a single IRQ/core handling all the IO.
That, indeed, seems to to be the case. When I do cat /proc/interrupts |
egrep '(CPU/nvm)', I get just the header line with one data line:
CPU0 CPU1 CPU2 CPU3
17: 0 0 15 14605 IO-APIC 17-fasteoi
ehci_hcd:usb1, nvme0q0, nvme0q1
I'm kind of feeling a bit out of my depth here. What are the nvme0q0,
etc.? "Queues" of some kind? You appear to have nine of these things,
I've just got two. I'm sure there's a fine manual I ought to be
reading. Do you know where I might find this manual?
When I look at the entire /proc/interrupts, there are just 30 lines
listed, and I suspect there are no more than 32 interrupt numbers
available. Is there any way I can configure Linux to give my SSD more
than one interrupt line to work with?
> FWIW
> # hdparm -tT /dev/nvme0n1
> /dev/nvme0n1:
> Timing cached reads: 9884 MB in 2.00 seconds = 4945.35 MB/sec
> Timing buffered disk reads: 4506 MB in 3.00 seconds = 1501.84 MB/sec
I get:
/dev/nvme0n1:
Timing cached reads: 4248 MB in 2.00 seconds = 2124.01 MB/sec
Timing buffered disk reads: 1214 MB in 3.00 seconds = 404.51 MB/sec
So my "cached reads" speed is (a little under) half of yours. This is
to be expected, since my PCIe lanes are only version 2 (and yours are
probably version 3). But the "buffered disk read" are much slower. Is
this just the age of my PC, or might I have something suboptimally
configured?
--
Alan Mackenzie (Nuremberg, Germany).
