Hi Robert, On Wed, Jun 22, 2016 at 07:33:01AM -0700, Robert Mustacchi wrote: > On 6/22/16 5:08 , Hans Rosenfeld wrote: > > I have looked at the system and experimented with it. This is what I > > found: > > > > This is an issue with MSI-X in VMware. Manually disabling MSI-X in the > > nvme driver makes it work. That is, it works as intended using MSI or > > fixed interrupts. There is currently no tunable in the driver to do > > this, though. > > Do you have more information about where this is happening or what > particular part is going wrong and causing things to hang? e.g. > presumably we're writing to some part of PCI config space or something > else is going on?
I finally got around to take another look at this. It isn't really hanging anywhere, it's just not receiving interrupts. So blkdev gets stuck in biowait, waiting for nvme. Actually, what I observed is this: the driver gets 2 MSI-X interrupt vectors, 0 and 1. Vector 0 is used for the admin queue and for I/O queue 2, and vector 1 is used for I/O queue 1. Then interrupts are only delivered through vector 0. I tried setting the interrupt count to 1 to request only one vector, and that unsurprisingly also works. I have a few more ideas for further experiments with this, but it really looks like VMware is misbehaving. Hans -- %SYSTEM-F-ANARCHISM, The operating system has been overthrown ------------------------------------------- illumos-discuss Archives: https://www.listbox.com/member/archive/182180/=now RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be Modify Your Subscription: https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4 Powered by Listbox: http://www.listbox.com
