jermainewang opened a new issue #13658: Converting MX array to DLPack crashes when MX array goes out-of-scope URL: https://github.com/apache/incubator-mxnet/issues/13658 ## Description Converting MX NDArray to DLPack, then to other framework's DLPack-compatible NDArray causes memory corruption when the origin MX NDArray goes out-of-scope. ## Environment info (Required) ``` ----------Python Info---------- Version : 3.5.2 Compiler : GCC 5.4.0 20160609 Build : ('default', 'Nov 12 2018 13:43:14') Arch : ('64bit', '') ------------Pip Info----------- Version : 18.1 Directory : /usr/local/lib/python3.5/dist-packages/pip ----------MXNet Info----------- Version : 1.4.0 Directory : /usr/local/lib/python3.5/dist-packages/mxnet Commit Hash : 1f73c5d9d308a690b57ea1b474d2ba99ca06c476 ----------System Info---------- Platform : Linux-4.19.4-arch1-1-ARCH-x86_64-with-Ubuntu-16.04-xenial system : Linux node : 17d02f89890e release : 4.19.4-arch1-1-ARCH version : #1 SMP PREEMPT Fri Nov 23 09:06:58 UTC 2018 ----------Hardware Info---------- machine : x86_64 processor : x86_64 Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Thread(s) per core: 2 Core(s) per socket: 4 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 62 Model name: Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz Stepping: 4 CPU MHz: 1812.064 CPU max MHz: 3900.0000 CPU min MHz: 1200.0000 BogoMIPS: 7384.55 Virtualization: VT-x Hypervisor vendor: vertical Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 10240K NUMA node0 CPU(s): 0-7 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts flush_l1d ----------Network Test---------- Setting timeout: 10 Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0056 sec, LOAD: 0.4655 sec. Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0277 sec, LOAD: 0.4154 sec. Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0039 sec, LOAD: 0.1236 sec. Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.1844 sec, LOAD: 1.0354 sec. Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0045 sec, LOAD: 0.0329 sec. Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0125 sec, LOAD: 0.6710 sec. ``` Package used (Python/R/Scala/Julia): Python ## Error Message: ``` Segmentation fault: 11 Stack trace returned 10 entries: [bt] (0) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x1fef5a) [0x7f4a09186f5a] [bt] (1) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x31383b6) [0x7f4a0c0c03b6] [bt] (2) /lib/x86_64-linux-gnu/libc.so.6(+0x354b0) [0x7f4a259324b0] [bt] (3) /usr/local/lib/python3.5/dist-packages/torch/lib/libcaffe2.so(at::TypeDefault::tensorFromBlob(void*, c10::ArrayRef<long>, c10::ArrayRef<long>, std::function<void (void*)> const&) const+0x61) [0x7f4996c4c741] [bt] (4) /usr/local/lib/python3.5/dist-packages/torch/lib/libcaffe2.so(at::fromDLPack(DLManagedTensor const*)+0x29f) [0x7f4996871e2f] [bt] (5) /usr/local/lib/python3.5/dist-packages/torch/lib/libtorch_python.so(THPModule_fromDLPack(_object*, _object*)+0x41) [0x7f49e2e2f341] [bt] (6) python3(PyEval_EvalFrameEx+0x4d06) [0x53b486] [bt] (7) python3(PyEval_EvalFrameEx+0x4b14) [0x53b294] [bt] (8) python3() [0x53fc97] [bt] (9) python3(PyEval_EvalCode+0x1f) [0x5409bf] ``` ## Minimum reproducible example ```python import mxnet as mx from torch.utils import dlpack def foo(): x = mx.nd.array([0, 5], dtype='int64') dl = x.to_dlpack_for_read() return dlpack.from_dlpack(dl) for i in range(10): y = foo() y.numpy() ``` Torch version v1.0.0 ## Steps to reproduce (Paste the commands you ran that produced the error.) 1. Use a ubuntu 16.04 image (with mx and torch installed) 2. Run the above code ## What have you tried to solve it? Found this bug in DGL project https://github.com/dmlc/dgl/pull/312 . Tried: 1. MXArray -> DLPack -> DGL Array : FAILED 2. MXArray -> DLPack -> MXArray : SUCCEED 3. MXArray -> DLPack -> Torch Tensor : FAILED 4. Torch Tensor -> DLPack -> DGL Array : SUCCEED
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
