vyasr commented on issue #44342: URL: https://github.com/apache/arrow/issues/44342#issuecomment-2406232501
Well OK, to my (pleasant) surprise upgrading to the latest nightly did not make the error vanish (well I suppose not pleasant that I have a seg fault, but at least pleasant that there's something reproducible happening): ``` root@g242-p33-0009:/repo# python -c "import cupy; import cudf;" Segmentation fault (core dumped) root@g242-p33-0009:/repo# python iPython 3.12.7 (main, Oct 4 2024, 15:35:43) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import pyarrow >>> pyarrow.__version__ '18.0.0.dev445' >>> ``` The backtrace is the same, still in `jemalloc_bg_thd`. > So, perhaps there's nothing particular that we should do in PyArrow? I think compiling out jemalloc or recompiling using the appropriate page size for arm could still make sense. While I haven't been able to reduce my example much further yet, the fact that pyarrow < 17.0.0 works while 17.0.0 and 18 alphas both fail indicate that something meaningful has changed there in the pyarrow binary and anyone could hit it. > (at least if you could git bisect and find out when precisely the issue starts happening with PyArrow, that could perhaps give a clue) I would be happy to try that, but I would also need to be able to build pyarrow wheels that are equivalent to the build process you have. As I mentioned above > attempted to rebuild libarrow.so using the same flags used to build the wheel (I can't be sure that I got them all right though, I based my compilation on the flags in https://github.com/apache/arrow/blob/main/ci/scripts/python_wheel_manylinux_build.sh). and then preload the library, but that too caused the segmentation fault to disappear Since the latest pyarrow nightlies fail for me, that suggests that I was indeed not compiling exactly equivalent C++ to what you produce (or perhaps I was but there's also something in the Python build that's relevant since I simply LD_PRELOADed libarrow.so). The nightly index linked above unfortunately doesn't go back far enough for me to install nightlies in between 16.1 and 17 to see where the issue might have arisen. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
