Hi, Thanks for digging into this! I couldn't notice that we can fix the libcrypt.so.1 error from Perl by installing libxcrypt-compat.
https://github.com/apache/arrow/pull/35362 is another approach to avoid the error. Anyway, I think that we solved all reported problems for RC0. I think that we can close this vote and release 12.0.0. Thanks, -- kou In <cajdg-vt-_yux1wcc5_jyh+p30qockahzzpm4tv+06jr4_se...@mail.gmail.com> "Re: [VOTE] Release Apache Arrow 12.0.0 - RC0" on Fri, 28 Apr 2023 11:59:09 -0400, Matthew Topol <m...@voltrondata.com.INVALID> wrote: > Okay, I confirmed that by running the following two commands, the test > failures that Jacob found on Manjarolinux were solved (at least in the > container I was using) > > $ pacman -S libxcrypt-compat > $ ln -s /usr/share/zoneinfo/America/New_York /etc/localtime > > For the second command it looks like the /etc/localtime symbolic link > wasn't being set in the container and is leveraged by the Orc adapter > tests. So setting the localtime (to any valid zone info) was sufficient to > let the tests run and pass. > > Hope this helps! > > > On Fri, Apr 28, 2023 at 11:49 AM Matthew Topol <m...@voltrondata.com> wrote: > >> Looks like this might be related: >> https://unix.stackexchange.com/questions/691479/how-to-deal-with-missing-libcrypt-so-1-on-arch-linux >> as manjaro also uses pacman and Arch Linux's packages. >> >> I'm re-running the verification right now after installing the recommended >> package in that thread. I'll report back if it solves the issue. >> >> --Matt >> >> On Fri, Apr 28, 2023 at 11:29 AM Matthew Topol <m...@voltrondata.com> >> wrote: >> >>> @Kou: I was able to reproduce the libcrypto failure that Jacob saw using >>> https://hub.docker.com/r/manjarolinux/base though i did need to manually >>> install git first since it doesn't come with it. >>> >>> $ pacman -Syu git >>> $ git clone https://github.com/apache/arrow.git >>> $ cd arrow >>> $ TEST_DEFAULT=0 TEST_SOURCE=0 TEST_CPP=1 USE_CONDA=1 >>> dev/release/verify-release-candidate.sh 12.0.0 0 >>> >>> That set of commands was sufficient to reproduce the error I believe (I >>> did this on monday when I was poking around the failures but I definitely >>> managed to see the same error pop up in a run). I'm running it again right >>> now to confirm. >>> >>> --Matt >>> >>> On Thu, Apr 27, 2023 at 8:28 PM Sutou Kouhei <k...@clear-code.com> wrote: >>> >>>> Hi, >>>> >>>> Thanks for sharing the log. >>>> >>>> libcrypto.so isn't related on the segmentation fault. It's >>>> just for relating to showing backtrace. >>>> >>>> > perl: error while loading shared libraries: libcrypt.so.1: >>>> > cannot open shared object file: No such file or directory >>>> >>>> This is happen at >>>> >>>> https://github.com/apache/arrow/blob/main/cpp/build-support/run-test.sh#L42 >>>> : >>>> >>>> TEST_NAME=$(echo $TEST_FILENAME | perl -pe 's/\..+?$//') # Remove path >>>> and extension (if any). >>>> >>>> BTW, it seems that we should remove a Perl dependency from >>>> https://github.com/apache/arrow/blob/main/cpp/build-support/run-test.sh >>>> ... >>>> >>>> >>>> I want to reproduce this problem on my environment. Could >>>> you share your environment information? Did you use Manjaro >>>> Linux this too? >>>> >>>> >>>> Thanks, >>>> -- >>>> kou >>>> >>>> >>>> In <canva0dgkodpfde7_b8xuvmtkh5kdmzvmtpbofo82hqj17gu...@mail.gmail.com> >>>> "Re: [VOTE] Release Apache Arrow 12.0.0 - RC0" on Thu, 27 Apr 2023 >>>> 23:54:58 +0200, >>>> Jacob Wujciak <ja...@voltrondata.com.INVALID> wrote: >>>> >>>> > I have uploaded the log [1] for the run using conda with gandiva >>>> active. It >>>> > looks like there is an issue with libcrypt.so causing these tests to >>>> > segfault. >>>> > >>>> > 1: https://gist.github.com/assignUser/cba0a13875de9d6a4f31000f585244f0 >>>> > >>>> > On Thu, Apr 27, 2023 at 11:32 PM Will Jones <will.jones...@gmail.com> >>>> wrote: >>>> > >>>> >> Hi Raul, >>>> >> >>>> >> It might be worth creating a new RC that fixes more of the test >>>> issues, >>>> >> even if they shouldn't be blockers. I've run the release script a few >>>> >> different times, and after 1.5 hours (is that a normal runtime for >>>> >> verification?) I get various test failures. So far the errors are in >>>> the >>>> >> TEST_PYTHON, TEST_WHEELS, and TEST_JAVA. I'll have more time tomorrow >>>> to >>>> >> look at the failures I am seeing. >>>> >> >>>> >> On Thu, Apr 27, 2023 at 8:09 AM Raúl Cumplido <raulcumpl...@gmail.com >>>> > >>>> >> wrote: >>>> >> >>>> >> > Hi, >>>> >> > >>>> >> > The vote for the RC has been open for 5 days. >>>> >> > >>>> >> > I will wait until tomorrow, if no more +1 votes are casted I >>>> >> > understand that the issue related to the pandas failures >>>> >> > (https://github.com/apache/arrow/issues/35321) is causing >>>> verification >>>> >> > to fail and we require a new RC with the above fix. >>>> >> > >>>> >> > Let me know if there are other blockers that should be included in >>>> that >>>> >> > case. >>>> >> > >>>> >> > Kind regards, >>>> >> > Raúl >>>> >> > >>>> >> > El jue, 27 abr 2023 a las 16:08, Sutou Kouhei (<k...@clear-code.com >>>> >) >>>> >> > escribió: >>>> >> > > >>>> >> > > Hi, >>>> >> > > >>>> >> > > I tried this on a manjarolinux/base Docker image. >>>> >> > > >>>> >> > > I think that this is a problem of the Arch Linux's llvm >>>> >> > > package. LLVMExports.cmake in the package doesn't provide >>>> >> > > the LLVMX86CodeGen target: >>>> >> > > >>>> >> > > # grep add_library /usr/lib/cmake/llvm/LLVMExports.cmake >>>> >> > > add_library(LLVMDemangle STATIC IMPORTED) >>>> >> > > add_library(LLVMSupport STATIC IMPORTED) >>>> >> > > add_library(LLVMTableGen STATIC IMPORTED) >>>> >> > > add_library(LTO SHARED IMPORTED) >>>> >> > > add_library(LLVMgold MODULE IMPORTED) >>>> >> > > add_library(LLVM SHARED IMPORTED) >>>> >> > > add_library(Remarks SHARED IMPORTED) >>>> >> > > >>>> >> > > FYI: LLVMExports.cmake on Debian GNU/Linux provides many >>>> >> > > targets: >>>> >> > > >>>> >> > > $ grep add_library >>>> /usr/lib/llvm-15/lib/cmake/llvm/LLVMExports.cmake | >>>> >> > head >>>> >> > > add_library(LLVMDemangle STATIC IMPORTED) >>>> >> > > add_library(LLVMSupport STATIC IMPORTED) >>>> >> > > add_library(LLVMTableGen STATIC IMPORTED) >>>> >> > > add_library(LLVMTableGenGlobalISel STATIC IMPORTED) >>>> >> > > add_library(LLVMCore STATIC IMPORTED) >>>> >> > > add_library(LLVMFuzzerCLI STATIC IMPORTED) >>>> >> > > add_library(LLVMFuzzMutate STATIC IMPORTED) >>>> >> > > add_library(LLVMFileCheck STATIC IMPORTED) >>>> >> > > add_library(LLVMInterfaceStub STATIC IMPORTED) >>>> >> > > add_library(LLVMIRReader STATIC IMPORTED) >>>> >> > > >>>> >> > > $ grep add_library >>>> /usr/lib/llvm-15/lib/cmake/llvm/LLVMExports.cmake | >>>> >> > wc -l >>>> >> > > 195 >>>> >> > > >>>> >> > > >>>> >> > > Thanks, >>>> >> > > -- >>>> >> > > kou >>>> >> > > >>>> >> > > In <CANva0dgrkkaAb_dbnSLXuVxi= >>>> h22i0yjhbtde-y3-rflox8...@mail.gmail.com >>>> >> > >>>> >> > > "Re: [VOTE] Release Apache Arrow 12.0.0 - RC0" on Tue, 25 Apr >>>> 2023 >>>> >> > 23:50:21 +0200, >>>> >> > > Jacob Wujciak <ja...@voltrondata.com.INVALID> wrote: >>>> >> > > >>>> >> > > > I checked out a trace for the cmake issue and LLVM 15.07 is >>>> found >>>> >> > > > correctly. The issue come from `llvm_map_components_to_libnames` >>>> >> which >>>> >> > > > complains about X86 not being in the lsit of libraries. But we >>>> don't >>>> >> > add >>>> >> > > > that but rather it gets appended in the function? >>>> >> > > > >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(228): >>>> >> > > > get_property(LLVM_TARGETS_CONFIGURED GLOBAL PROPERTY >>>> >> > > > LLVM_TARGETS_CONFIGURED ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(234): if(NOT >>>> >> > LLVM_TARGETS_CONFIGURED >>>> >> > > > ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(244): list(FIND >>>> >> > > > LLVM_TARGETS_TO_BUILD X86 have_native_backend ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(245): list(FIND >>>> >> link_components >>>> >> > > > engine engine_required ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(246): if(NOT >>>> engine_required >>>> >> > EQUAL >>>> >> > > > -1 ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(255): list(FIND >>>> >> link_components >>>> >> > > > native native_required ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(256): if(NOT >>>> native_required >>>> >> > EQUAL >>>> >> > > > -1 ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(257): if(NOT >>>> >> have_native_backend >>>> >> > > > EQUAL -1 ) >>>> >> > > > */usr/lib/cmake/llvm/LLVM-Config.cmake(258): list(APPEND >>>> >> > link_components >>>> >> > > > X86 )* >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(263): >>>> >> > > > llvm_expand_pseudo_components(link_components >>>> >> > > > >>>> >> > >>>> >> >>>> core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(127): set(link_components >>>> >> > > > >>>> >> > >>>> >> >>>> core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(128): foreach(c >>>> >> > > > >>>> >> > >>>> >> >>>> core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(130): list(FIND >>>> >> > > > LLVM_TARGETS_TO_BUILD core idx ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(131): if(NOT idx LESS 0 ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(159): elseif(c STREQUAL >>>> >> > > > nativecodegen ) >>>> >> > > > [snip] >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(212): list(APPEND >>>> >> > > > expanded_components debuginfodwarf ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(130): list(FIND >>>> >> > > > LLVM_TARGETS_TO_BUILD X86 idx ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(131): if(NOT idx LESS 0 ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(132): if(TARGET >>>> >> LLVMX86CodeGen ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(134): else() >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(135): if(TARGET LLVMX86 ) >>>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(137): else() >>>> >> > > > */usr/lib/cmake/llvm/LLVM-Config.cmake(138): >>>> message(FATAL_ERROR >>>> >> > Target >>>> >> > > > X86 is not in the set of libraries. )* >>>> >> > > > >>>> >> > > > On Tue, Apr 25, 2023 at 10:57 AM Raúl Cumplido < >>>> >> raulcumpl...@gmail.com >>>> >> > > >>>> >> > > > wrote: >>>> >> > > > >>>> >> > > >> I have created the following issue for the new wheels test >>>> failure >>>> >> > > >> around pandas 2.0.1 : >>>> https://github.com/apache/arrow/issues/35321 >>>> >> > > >> >>>> >> > > >> I don't think we should create a new RC for that issue but I'm >>>> happy >>>> >> > > >> to know other people's thoughts around that. >>>> >> > > >> >>>> >> > > >> El lun, 24 abr 2023 a las 21:12, Raúl Cumplido >>>> >> > > >> (<raulcumpl...@gmail.com>) escribió: >>>> >> > > >> > >>>> >> > > >> > El lun, 24 abr 2023 a las 18:53, Will Jones >>>> >> > > >> > (<will.jones...@gmail.com>) escribió: >>>> >> > > >> > > >>>> >> > > >> > > I'm seeing failing Pandas tests in PyArrow when verifying >>>> with >>>> >> > > >> > > >>>> >> > > >> > > USE_CONDA=1 dev/release/verify-release-candidate.sh 12.0.0 >>>> 0 >>>> >> > > >> > > >>>> >> > > >> > > >>>> >> > > >> >>>> >> > >>>> >> >>>> pyarrow/tests/test_extension_type.py::test_extension_to_pandas_storage_type[registered_period_type0] >>>> >> > > >> > > - NotImplementedError: extension<test.period<PeriodType>> >>>> >> > > >> > >>>> >> > > >> > This is also happening on our nightlies from today: >>>> >> > > >> > >>>> >> > > >> >>>> >> > >>>> >> >>>> https://github.com/ursacomputing/crossbow/actions/runs/4786502455/jobs/8510514881 >>>> >> > > >> > >>>> >> > > >> > There has been a new pandas release: 2.0.1 around 9 hours ago >>>> >> which >>>> >> > > >> > seems to be the causing issue: >>>> >> > > >> > https://pypi.org/project/pandas/#history >>>> >> > > >> > >>>> >> > > >> > > >>>> >> > > >> > > No one else is getting that? >>>> >> > > >> > > >>>> >> > > >> > > >>>> >> > > >> > > On Sun, Apr 23, 2023 at 9:21 AM Raúl Cumplido < >>>> >> > raulcumpl...@gmail.com> >>>> >> > > >> > > wrote: >>>> >> > > >> > > >>>> >> > > >> > > > +1 (non binding) >>>> >> > > >> > > > >>>> >> > > >> > > > I have tested both SOURCES and BINARIES successfully >>>> with: >>>> >> > > >> > > > TEST_DEFAULT=0 TEST_SOURCE=1 >>>> >> > dev/release/verify-release-candidate.sh >>>> >> > > >> > > > 12.0.0 0 >>>> >> > > >> > > > TEST_DEFAULT=0 TEST_WHEELS=1 >>>> >> > dev/release/verify-release-candidate.sh >>>> >> > > >> > > > 12.0.0 0 >>>> >> > > >> > > > TEST_DEFAULT=0 TEST_BINARIES=1 >>>> >> > > >> dev/release/verify-release-candidate.sh >>>> >> > > >> > > > 12.0.0 0 >>>> >> > > >> > > > with: >>>> >> > > >> > > > * Python 3.10.6 >>>> >> > > >> > > > * gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0 >>>> >> > > >> > > > * NVIDIA CUDA cuda_11.5.r11.5/compiler.30672275_0 >>>> >> > > >> > > > * openjdk version "17.0.6" 2023-01-17 >>>> >> > > >> > > > * ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) >>>> >> > > >> [x86_64-linux-gnu] >>>> >> > > >> > > > * dotnet 7.0.203 >>>> >> > > >> > > > * Ubuntu 22.04 LTS >>>> >> > > >> > > > >>>> >> > > >> > > > El dom, 23 abr 2023 a las 12:59, Yibo Cai (< >>>> yibo....@arm.com >>>> >> >) >>>> >> > > >> escribió: >>>> >> > > >> > > > > >>>> >> > > >> > > > > +1 >>>> >> > > >> > > > > >>>> >> > > >> > > > > I ran the followings on Ubuntu-22.04, aarch64. >>>> >> > > >> > > > > >>>> >> > > >> > > > > TEST_DEFAULT=0 \ >>>> >> > > >> > > > > TEST_CPP=1 \ >>>> >> > > >> > > > > TEST_PYTHON=1 \ >>>> >> > > >> > > > > TEST_GO=1 \ >>>> >> > > >> > > > > dev/release/verify-release-candidate.sh 12.0.0 0 >>>> >> > > >> > > > > >>>> >> > > >> > > > > TEST_DEFAULT=0 \ >>>> >> > > >> > > > > TEST_WHEELS=1 \ >>>> >> > > >> > > > > dev/release/verify-release-candidate.sh 12.0.0 0 >>>> >> > > >> > > > > >>>> >> > > >> > > > > >>>> >> > > >> > > > > On 4/23/23 14:40, Sutou Kouhei wrote: >>>> >> > > >> > > > > > +1 >>>> >> > > >> > > > > > >>>> >> > > >> > > > > > I ran the followings on Debian GNU/Linux sid: >>>> >> > > >> > > > > > >>>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>>> >> > > >> > > > > > TEST_SOURCE=1 \ >>>> >> > > >> > > > > > LANG=C \ >>>> >> > > >> > > > > > TZ=UTC \ >>>> >> > > >> > > > > > CUDAToolkit_ROOT=/usr \ >>>> >> > > >> > > > > > ARROW_CMAKE_OPTIONS="-DBoost_NO_BOOST_CMAKE=ON >>>> >> > > >> > > > -Dxsimd_SOURCE=BUNDLED" \ >>>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>>> 12.0.0 0 >>>> >> > > >> > > > > > >>>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>>> >> > > >> > > > > > TEST_APT=1 \ >>>> >> > > >> > > > > > LANG=C \ >>>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>>> 12.0.0 0 >>>> >> > > >> > > > > > >>>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>>> >> > > >> > > > > > TEST_BINARY=1 \ >>>> >> > > >> > > > > > LANG=C \ >>>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>>> 12.0.0 0 >>>> >> > > >> > > > > > >>>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>>> >> > > >> > > > > > TEST_JARS=1 \ >>>> >> > > >> > > > > > LANG=C \ >>>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>>> 12.0.0 0 >>>> >> > > >> > > > > > >>>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>>> >> > > >> > > > > > TEST_PYTHON_VERSIONS=3.11 \ >>>> >> > > >> > > > > > TEST_WHEELS=1 \ >>>> >> > > >> > > > > > LANG=C \ >>>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>>> 12.0.0 0 >>>> >> > > >> > > > > > >>>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>>> >> > > >> > > > > > TEST_YUM=1 \ >>>> >> > > >> > > > > > LANG=C \ >>>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>>> 12.0.0 0 >>>> >> > > >> > > > > > >>>> >> > > >> > > > > > with: >>>> >> > > >> > > > > > >>>> >> > > >> > > > > > * .NET SDK (6.0.406) >>>> >> > > >> > > > > > * Python 3.11.2 >>>> >> > > >> > > > > > * gcc (Debian 12.2.0-14) 12.2.0 >>>> >> > > >> > > > > > * nvidia-cuda-dev 11.7.99~11.7.1-4 >>>> >> > > >> > > > > > * openjdk version "17.0.6" 2023-01-17 >>>> >> > > >> > > > > > * ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) >>>> >> > > >> [x86_64-linux-gnu] >>>> >> > > >> > > > > > >>>> >> > > >> > > > > > >>>> >> > > >> > > > > > Thanks, >>>> >> > > >> > > > >>>> >> > > >> >>>> >> > >>>> >> >>>> >>>
