Thanks Dan for the reply!
On 6/14/19, 3:06 AM, "Dan Williams" <dan.j.willi...@intel.com> wrote:
On Wed, Jun 12, 2019 at 9:08 PM Yue Li <yue...@memverge.com> wrote:
>
> hi Dan and Steve,
>
>
Hi,
I just happened to see this by luck, please use my Intel address, and
copy the libnvdimm mailing list on issues like this
(linux-nvdimm@lists.01.org).
OK.
> We recently ran into a strange issue where ndctl command hangs on dev dax
after our software uses it.
The last thing that device-dax teardown does is wait for any pinned
pages to be released before allowing the exit to proceed.
OK.
> Inside our application, we basically will first RDMA register the whole
device, then deregister, and exit.
Is this just using simple ibverbs to unregister or something specific
to this driver.
There was a bug upstream that addressed cases where device teardown
proceeded when it shouldn't, but the sequence you describe is the
opposite the pages pins should be torn down before the device
reconfiguration.
> However, if we remove the registration and deregistration code, ndctl
works correctly without hanging. The problem occurs both on DRAM emulated dax
as well as real PMEM backed dax.
>
> Here is our system information:
>
>
>
> CentOS 7.6
>
> Vanilla kernel 3.10.0-957.el7.x86_64
Are you familiar with rebuilding the kernel? I'd ask you to try to
reproduce with the latest development kernel that includes these
fixes:
4422ee8476f0 mm/devm_memremap_pages: fix final page put race
771f0714d0dc PCI/P2PDMA: track pgmap references per resource, not globally
af37085de906 lib/genalloc: introduce chunk owners
e0047ff8aa77 PCI/P2PDMA: fix the gen_pool_add_virt() failure path
0315d47d6ae9 mm/devm_memremap_pages: introduce devm_memunmap_pages
216475c7eaa8 drivers/base/devres: introduce devm_release_action()
...but it sounds like you may be hitting a different issue.
Thanks for the suggestion, we will download the upstream kernel and try it
again. Will post the results soon.
Best,
Yue
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
