Looks like this might be related: https://unix.stackexchange.com/questions/691479/how-to-deal-with-missing-libcrypt-so-1-on-arch-linux as manjaro also uses pacman and Arch Linux's packages.
I'm re-running the verification right now after installing the recommended package in that thread. I'll report back if it solves the issue. --Matt On Fri, Apr 28, 2023 at 11:29 AM Matthew Topol <m...@voltrondata.com> wrote: > @Kou: I was able to reproduce the libcrypto failure that Jacob saw using > https://hub.docker.com/r/manjarolinux/base though i did need to manually > install git first since it doesn't come with it. > > $ pacman -Syu git > $ git clone https://github.com/apache/arrow.git > $ cd arrow > $ TEST_DEFAULT=0 TEST_SOURCE=0 TEST_CPP=1 USE_CONDA=1 > dev/release/verify-release-candidate.sh 12.0.0 0 > > That set of commands was sufficient to reproduce the error I believe (I > did this on monday when I was poking around the failures but I definitely > managed to see the same error pop up in a run). I'm running it again right > now to confirm. > > --Matt > > On Thu, Apr 27, 2023 at 8:28 PM Sutou Kouhei <k...@clear-code.com> wrote: > >> Hi, >> >> Thanks for sharing the log. >> >> libcrypto.so isn't related on the segmentation fault. It's >> just for relating to showing backtrace. >> >> > perl: error while loading shared libraries: libcrypt.so.1: >> > cannot open shared object file: No such file or directory >> >> This is happen at >> >> https://github.com/apache/arrow/blob/main/cpp/build-support/run-test.sh#L42 >> : >> >> TEST_NAME=$(echo $TEST_FILENAME | perl -pe 's/\..+?$//') # Remove path >> and extension (if any). >> >> BTW, it seems that we should remove a Perl dependency from >> https://github.com/apache/arrow/blob/main/cpp/build-support/run-test.sh >> ... >> >> >> I want to reproduce this problem on my environment. Could >> you share your environment information? Did you use Manjaro >> Linux this too? >> >> >> Thanks, >> -- >> kou >> >> >> In <canva0dgkodpfde7_b8xuvmtkh5kdmzvmtpbofo82hqj17gu...@mail.gmail.com> >> "Re: [VOTE] Release Apache Arrow 12.0.0 - RC0" on Thu, 27 Apr 2023 >> 23:54:58 +0200, >> Jacob Wujciak <ja...@voltrondata.com.INVALID> wrote: >> >> > I have uploaded the log [1] for the run using conda with gandiva >> active. It >> > looks like there is an issue with libcrypt.so causing these tests to >> > segfault. >> > >> > 1: https://gist.github.com/assignUser/cba0a13875de9d6a4f31000f585244f0 >> > >> > On Thu, Apr 27, 2023 at 11:32 PM Will Jones <will.jones...@gmail.com> >> wrote: >> > >> >> Hi Raul, >> >> >> >> It might be worth creating a new RC that fixes more of the test issues, >> >> even if they shouldn't be blockers. I've run the release script a few >> >> different times, and after 1.5 hours (is that a normal runtime for >> >> verification?) I get various test failures. So far the errors are in >> the >> >> TEST_PYTHON, TEST_WHEELS, and TEST_JAVA. I'll have more time tomorrow >> to >> >> look at the failures I am seeing. >> >> >> >> On Thu, Apr 27, 2023 at 8:09 AM Raúl Cumplido <raulcumpl...@gmail.com> >> >> wrote: >> >> >> >> > Hi, >> >> > >> >> > The vote for the RC has been open for 5 days. >> >> > >> >> > I will wait until tomorrow, if no more +1 votes are casted I >> >> > understand that the issue related to the pandas failures >> >> > (https://github.com/apache/arrow/issues/35321) is causing >> verification >> >> > to fail and we require a new RC with the above fix. >> >> > >> >> > Let me know if there are other blockers that should be included in >> that >> >> > case. >> >> > >> >> > Kind regards, >> >> > Raúl >> >> > >> >> > El jue, 27 abr 2023 a las 16:08, Sutou Kouhei (<k...@clear-code.com>) >> >> > escribió: >> >> > > >> >> > > Hi, >> >> > > >> >> > > I tried this on a manjarolinux/base Docker image. >> >> > > >> >> > > I think that this is a problem of the Arch Linux's llvm >> >> > > package. LLVMExports.cmake in the package doesn't provide >> >> > > the LLVMX86CodeGen target: >> >> > > >> >> > > # grep add_library /usr/lib/cmake/llvm/LLVMExports.cmake >> >> > > add_library(LLVMDemangle STATIC IMPORTED) >> >> > > add_library(LLVMSupport STATIC IMPORTED) >> >> > > add_library(LLVMTableGen STATIC IMPORTED) >> >> > > add_library(LTO SHARED IMPORTED) >> >> > > add_library(LLVMgold MODULE IMPORTED) >> >> > > add_library(LLVM SHARED IMPORTED) >> >> > > add_library(Remarks SHARED IMPORTED) >> >> > > >> >> > > FYI: LLVMExports.cmake on Debian GNU/Linux provides many >> >> > > targets: >> >> > > >> >> > > $ grep add_library >> /usr/lib/llvm-15/lib/cmake/llvm/LLVMExports.cmake | >> >> > head >> >> > > add_library(LLVMDemangle STATIC IMPORTED) >> >> > > add_library(LLVMSupport STATIC IMPORTED) >> >> > > add_library(LLVMTableGen STATIC IMPORTED) >> >> > > add_library(LLVMTableGenGlobalISel STATIC IMPORTED) >> >> > > add_library(LLVMCore STATIC IMPORTED) >> >> > > add_library(LLVMFuzzerCLI STATIC IMPORTED) >> >> > > add_library(LLVMFuzzMutate STATIC IMPORTED) >> >> > > add_library(LLVMFileCheck STATIC IMPORTED) >> >> > > add_library(LLVMInterfaceStub STATIC IMPORTED) >> >> > > add_library(LLVMIRReader STATIC IMPORTED) >> >> > > >> >> > > $ grep add_library >> /usr/lib/llvm-15/lib/cmake/llvm/LLVMExports.cmake | >> >> > wc -l >> >> > > 195 >> >> > > >> >> > > >> >> > > Thanks, >> >> > > -- >> >> > > kou >> >> > > >> >> > > In <CANva0dgrkkaAb_dbnSLXuVxi= >> h22i0yjhbtde-y3-rflox8...@mail.gmail.com >> >> > >> >> > > "Re: [VOTE] Release Apache Arrow 12.0.0 - RC0" on Tue, 25 Apr >> 2023 >> >> > 23:50:21 +0200, >> >> > > Jacob Wujciak <ja...@voltrondata.com.INVALID> wrote: >> >> > > >> >> > > > I checked out a trace for the cmake issue and LLVM 15.07 is found >> >> > > > correctly. The issue come from `llvm_map_components_to_libnames` >> >> which >> >> > > > complains about X86 not being in the lsit of libraries. But we >> don't >> >> > add >> >> > > > that but rather it gets appended in the function? >> >> > > > >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(228): >> >> > > > get_property(LLVM_TARGETS_CONFIGURED GLOBAL PROPERTY >> >> > > > LLVM_TARGETS_CONFIGURED ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(234): if(NOT >> >> > LLVM_TARGETS_CONFIGURED >> >> > > > ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(244): list(FIND >> >> > > > LLVM_TARGETS_TO_BUILD X86 have_native_backend ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(245): list(FIND >> >> link_components >> >> > > > engine engine_required ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(246): if(NOT >> engine_required >> >> > EQUAL >> >> > > > -1 ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(255): list(FIND >> >> link_components >> >> > > > native native_required ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(256): if(NOT >> native_required >> >> > EQUAL >> >> > > > -1 ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(257): if(NOT >> >> have_native_backend >> >> > > > EQUAL -1 ) >> >> > > > */usr/lib/cmake/llvm/LLVM-Config.cmake(258): list(APPEND >> >> > link_components >> >> > > > X86 )* >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(263): >> >> > > > llvm_expand_pseudo_components(link_components >> >> > > > >> >> > >> >> >> core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(127): set(link_components >> >> > > > >> >> > >> >> >> core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(128): foreach(c >> >> > > > >> >> > >> >> >> core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(130): list(FIND >> >> > > > LLVM_TARGETS_TO_BUILD core idx ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(131): if(NOT idx LESS 0 ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(159): elseif(c STREQUAL >> >> > > > nativecodegen ) >> >> > > > [snip] >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(212): list(APPEND >> >> > > > expanded_components debuginfodwarf ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(130): list(FIND >> >> > > > LLVM_TARGETS_TO_BUILD X86 idx ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(131): if(NOT idx LESS 0 ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(132): if(TARGET >> >> LLVMX86CodeGen ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(134): else() >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(135): if(TARGET LLVMX86 ) >> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(137): else() >> >> > > > */usr/lib/cmake/llvm/LLVM-Config.cmake(138): message(FATAL_ERROR >> >> > Target >> >> > > > X86 is not in the set of libraries. )* >> >> > > > >> >> > > > On Tue, Apr 25, 2023 at 10:57 AM Raúl Cumplido < >> >> raulcumpl...@gmail.com >> >> > > >> >> > > > wrote: >> >> > > > >> >> > > >> I have created the following issue for the new wheels test >> failure >> >> > > >> around pandas 2.0.1 : >> https://github.com/apache/arrow/issues/35321 >> >> > > >> >> >> > > >> I don't think we should create a new RC for that issue but I'm >> happy >> >> > > >> to know other people's thoughts around that. >> >> > > >> >> >> > > >> El lun, 24 abr 2023 a las 21:12, Raúl Cumplido >> >> > > >> (<raulcumpl...@gmail.com>) escribió: >> >> > > >> > >> >> > > >> > El lun, 24 abr 2023 a las 18:53, Will Jones >> >> > > >> > (<will.jones...@gmail.com>) escribió: >> >> > > >> > > >> >> > > >> > > I'm seeing failing Pandas tests in PyArrow when verifying >> with >> >> > > >> > > >> >> > > >> > > USE_CONDA=1 dev/release/verify-release-candidate.sh 12.0.0 0 >> >> > > >> > > >> >> > > >> > > >> >> > > >> >> >> > >> >> >> pyarrow/tests/test_extension_type.py::test_extension_to_pandas_storage_type[registered_period_type0] >> >> > > >> > > - NotImplementedError: extension<test.period<PeriodType>> >> >> > > >> > >> >> > > >> > This is also happening on our nightlies from today: >> >> > > >> > >> >> > > >> >> >> > >> >> >> https://github.com/ursacomputing/crossbow/actions/runs/4786502455/jobs/8510514881 >> >> > > >> > >> >> > > >> > There has been a new pandas release: 2.0.1 around 9 hours ago >> >> which >> >> > > >> > seems to be the causing issue: >> >> > > >> > https://pypi.org/project/pandas/#history >> >> > > >> > >> >> > > >> > > >> >> > > >> > > No one else is getting that? >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > On Sun, Apr 23, 2023 at 9:21 AM Raúl Cumplido < >> >> > raulcumpl...@gmail.com> >> >> > > >> > > wrote: >> >> > > >> > > >> >> > > >> > > > +1 (non binding) >> >> > > >> > > > >> >> > > >> > > > I have tested both SOURCES and BINARIES successfully with: >> >> > > >> > > > TEST_DEFAULT=0 TEST_SOURCE=1 >> >> > dev/release/verify-release-candidate.sh >> >> > > >> > > > 12.0.0 0 >> >> > > >> > > > TEST_DEFAULT=0 TEST_WHEELS=1 >> >> > dev/release/verify-release-candidate.sh >> >> > > >> > > > 12.0.0 0 >> >> > > >> > > > TEST_DEFAULT=0 TEST_BINARIES=1 >> >> > > >> dev/release/verify-release-candidate.sh >> >> > > >> > > > 12.0.0 0 >> >> > > >> > > > with: >> >> > > >> > > > * Python 3.10.6 >> >> > > >> > > > * gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0 >> >> > > >> > > > * NVIDIA CUDA cuda_11.5.r11.5/compiler.30672275_0 >> >> > > >> > > > * openjdk version "17.0.6" 2023-01-17 >> >> > > >> > > > * ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) >> >> > > >> [x86_64-linux-gnu] >> >> > > >> > > > * dotnet 7.0.203 >> >> > > >> > > > * Ubuntu 22.04 LTS >> >> > > >> > > > >> >> > > >> > > > El dom, 23 abr 2023 a las 12:59, Yibo Cai (< >> yibo....@arm.com >> >> >) >> >> > > >> escribió: >> >> > > >> > > > > >> >> > > >> > > > > +1 >> >> > > >> > > > > >> >> > > >> > > > > I ran the followings on Ubuntu-22.04, aarch64. >> >> > > >> > > > > >> >> > > >> > > > > TEST_DEFAULT=0 \ >> >> > > >> > > > > TEST_CPP=1 \ >> >> > > >> > > > > TEST_PYTHON=1 \ >> >> > > >> > > > > TEST_GO=1 \ >> >> > > >> > > > > dev/release/verify-release-candidate.sh 12.0.0 0 >> >> > > >> > > > > >> >> > > >> > > > > TEST_DEFAULT=0 \ >> >> > > >> > > > > TEST_WHEELS=1 \ >> >> > > >> > > > > dev/release/verify-release-candidate.sh 12.0.0 0 >> >> > > >> > > > > >> >> > > >> > > > > >> >> > > >> > > > > On 4/23/23 14:40, Sutou Kouhei wrote: >> >> > > >> > > > > > +1 >> >> > > >> > > > > > >> >> > > >> > > > > > I ran the followings on Debian GNU/Linux sid: >> >> > > >> > > > > > >> >> > > >> > > > > > * TEST_DEFAULT=0 \ >> >> > > >> > > > > > TEST_SOURCE=1 \ >> >> > > >> > > > > > LANG=C \ >> >> > > >> > > > > > TZ=UTC \ >> >> > > >> > > > > > CUDAToolkit_ROOT=/usr \ >> >> > > >> > > > > > ARROW_CMAKE_OPTIONS="-DBoost_NO_BOOST_CMAKE=ON >> >> > > >> > > > -Dxsimd_SOURCE=BUNDLED" \ >> >> > > >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 >> 0 >> >> > > >> > > > > > >> >> > > >> > > > > > * TEST_DEFAULT=0 \ >> >> > > >> > > > > > TEST_APT=1 \ >> >> > > >> > > > > > LANG=C \ >> >> > > >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 >> 0 >> >> > > >> > > > > > >> >> > > >> > > > > > * TEST_DEFAULT=0 \ >> >> > > >> > > > > > TEST_BINARY=1 \ >> >> > > >> > > > > > LANG=C \ >> >> > > >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 >> 0 >> >> > > >> > > > > > >> >> > > >> > > > > > * TEST_DEFAULT=0 \ >> >> > > >> > > > > > TEST_JARS=1 \ >> >> > > >> > > > > > LANG=C \ >> >> > > >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 >> 0 >> >> > > >> > > > > > >> >> > > >> > > > > > * TEST_DEFAULT=0 \ >> >> > > >> > > > > > TEST_PYTHON_VERSIONS=3.11 \ >> >> > > >> > > > > > TEST_WHEELS=1 \ >> >> > > >> > > > > > LANG=C \ >> >> > > >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 >> 0 >> >> > > >> > > > > > >> >> > > >> > > > > > * TEST_DEFAULT=0 \ >> >> > > >> > > > > > TEST_YUM=1 \ >> >> > > >> > > > > > LANG=C \ >> >> > > >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 >> 0 >> >> > > >> > > > > > >> >> > > >> > > > > > with: >> >> > > >> > > > > > >> >> > > >> > > > > > * .NET SDK (6.0.406) >> >> > > >> > > > > > * Python 3.11.2 >> >> > > >> > > > > > * gcc (Debian 12.2.0-14) 12.2.0 >> >> > > >> > > > > > * nvidia-cuda-dev 11.7.99~11.7.1-4 >> >> > > >> > > > > > * openjdk version "17.0.6" 2023-01-17 >> >> > > >> > > > > > * ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) >> >> > > >> [x86_64-linux-gnu] >> >> > > >> > > > > > >> >> > > >> > > > > > >> >> > > >> > > > > > Thanks, >> >> > > >> > > > >> >> > > >> >> >> > >> >> >> >
