Ah, thanks James. Maybe we can disable most/all optional components to start with on x86? Including Parquet, etc. (so we shouldn't need Boost at first)
If you rebase again the MinGW Flight issues should be fixed. [1] [1]: https://github.com/apache/arrow/pull/13696 On Mon, Jul 25, 2022, at 17:48, James Duong wrote: > Arkadiy's PR can take the changes for CI from my PR to get it past the > build. > The build process revealed more changes necessary though. > > In the library code there are a few places where we'll use > ARROW_POPCOUNT64 > directly and they fail to build on 32-bit. > Aside from this, some of the tests are failing to link now because they > are > trying to link to 64-bit boost. I haven't solved this problem yet: > https://github.com/apache/arrow/runs/7491731864?check_suite_focus=true#step:7:181 > https://github.com/apache/arrow/runs/7491731864?check_suite_focus=true#step:7:1455 > > I'm also seeing a new failure in MinGW test runs in Flight, but I think > this happened after rebasing recently. > > On Mon, Jul 25, 2022 at 2:24 PM David Li <lidav...@apache.org> wrote: > >> The PRs are: >> >> https://github.com/apache/arrow/pull/13659 >> https://github.com/apache/arrow/pull/13532 >> >> The JIRA is https://issues.apache.org/jira/browse/ARROW-16778 >> >> I suppose I agree with Arkadiy: we could merge the initial PR, suppressing >> warnings for the x86 build. But then we can merge James's fixes on top of >> that to make usage easier and make sure everyone's use cases are covered. >> It'll be a team effort that way. But first CI needs to pass on Arkadiy's >> initial PR >> >> On Mon, Jul 25, 2022, at 17:13, Wes McKinney wrote: >> > Suppressing the warnings on 32-bit MSVC sounds like a reasonable >> > compromise. Is there an open PR for this (and what is the >> > corresponding Jira issue so we don't lose track of it)? >> > >> > On Fri, Jul 22, 2022 at 1:23 PM Arkadiy Vertleyb (BLOOMBERG/ 120 PARK) >> > <avertl...@bloomberg.net> wrote: >> >> >> >> Or live with the warnings. Or cast his sizes to _int64, if the >> warnings are critical. >> >> >> >> From: dev@arrow.apache.org At: 07/22/22 14:06:34 UTC-4:00To: Arkadiy >> Vertleyb (BLOOMBERG/ 120 PARK ) , dev@arrow.apache.org >> >> Subject: Re: Help needed with PR #13659: Fixing build/unit test issues >> in msvc/win32 >> >> >> >> The problem with (2) is that a library user can only work around it by >> >> suppressing warnings with pragmas when including Arrow headers. >> >> >> >> This is somewhat unfriendly from a user perspective. >> >> >> >> On Fri., Jul. 22, 2022, 08:11 Arkadiy Vertleyb (BLOOMBERG/ 120 PARK), < >> >> avertl...@bloomberg.net> wrote: >> >> >> >> > This is what I would do: >> >> > >> >> > 1) Suppress warnings in cmake input file for this particular platform. >> >> > This will let the user build the arrow libs. >> >> > >> >> > 2) Let those users who care about 32 bits worry about the warnings in >> the >> >> > headers. >> >> > >> >> > Regards, >> >> > Arkadiy >> >> > >> >> > From: dev@arrow.apache.org At: 07/22/22 10:18:09 UTC-4:00To: >> >> > dev@arrow.apache.org >> >> > Subject: Re: Help needed with PR #13659: Fixing build/unit test >> issues in >> >> > msvc/win32 >> >> > >> >> > >> >> > Ah, that's a good point. Then we should probably use explicit casts. >> >> > >> >> > >> >> > Le 22/07/2022 à 15:58, James Duong a écrit : >> >> > > Seems reasonable to suppress this warning on this single platform. >> >> > However >> >> > > if we do this using a compiler option, we may hide warnings in >> public >> >> > > headers from the build. This isn't great since library users may >> have >> >> > > policies that disallow warnings. >> >> > > >> >> > > On Fri., Jul. 22, 2022, 05:47 Antoine Pitrou, <anto...@python.org> >> >> > wrote: >> >> > > >> >> > >> >> >> > >> We could perhaps suppress the integer downcast warnings, but only >> on >> >> > >> 32-bit Windows (not 64-bit, not other platforms). >> >> > >> >> >> > >> Regards >> >> > >> >> >> > >> Antoine. >> >> > >> >> >> > >> >> >> > >> Le 22/07/2022 à 14:42, Arkadiy Vertleyb (BLOOMBERG/ 120 PARK) a >> écrit : >> >> > >>> Hi James. >> >> > >>> >> >> > >>> I don't have strong feelings about whose PR is used and how >> exactly the >> >> > >> issue is fixed. All I care about at this point is working (and >> >> > maintained) >> >> > >> MSVC 32 bit version. I need this for my project at work. >> >> > >>> >> >> > >>> If you feel that your PR will solve this issue, and is close to >> being >> >> > >> approved, please let me know and I will stop worrying about this. >> >> > >>> >> >> > >>> If you PR is not being approved any time soon, I will proceed with >> >> > mine, >> >> > >> and you can add your changes on top of it. Should not be a big >> deal, >> >> > since >> >> > >> I only make a few small changes. >> >> > >>> >> >> > >>> Please let me know how you think we should proceed with this. >> >> > >>> >> >> > >>> Thanks, >> >> > >>> Arkadiy >> >> > >>> >> >> > >>> From: dev@arrow.apache.org At: 07/21/22 18:29:33 UTC-4:00To: >> Arkadiy >> >> > >> Vertleyb (BLOOMBERG/ 120 PARK ) , dev@arrow.apache.org >> >> > >>> Subject: Re: Help needed with PR #13659: Fixing build/unit test >> issues >> >> > >> in msvc/win32 >> >> > >>> >> >> > >>> Feedback I got here was to use static_cast: >> >> > >>> >> https://github.com/apache/arrow/pull/13532#issuecomment-1177488433 >> >> > >>> >> >> > >>> I'm indifferent as to whether we want to do the static_casts or >> just >> >> > >>> suppress the warning as you've done. >> >> > >>> >> >> > >>> Your PR isn't building the 32-bit build in CI btw. It fails >> finding >> >> > >> OpenSSL: >> >> > >>> >> >> > >> >> >> > >> >> > >> >> >> https://github.com/apache/arrow/runs/7432759888?check_suite_focus=true#step:7:20 >> >> > >>> 1 >> >> > >>> >> >> > >>> I've fixed this in the changes to the github workflow in my PR. >> >> > >>> >> >> > >>> >> >> > >>> On Thu, Jul 21, 2022 at 12:47 PM Arkadiy Vertleyb (BLOOMBERG/ 120 >> >> > PARK) < >> >> > >>> avertl...@bloomberg.net> wrote: >> >> > >>> >> >> > >>>> Hi James. >> >> > >>>> >> >> > >>>> My PR makes the compiler ignore the warnings. >> >> > >>>> >> >> > >>>> As far as I understand, this issue cannot be consistently >> resolved >> >> > >> within >> >> > >>>> the Google paradigm arrow follows on this subject. The google >> >> > paradigm >> >> > >>>> requires to treat all the sizes as signed 64 bit integers, >> regardless >> >> > of >> >> > >>>> the architecture. This paradigm is obviously at odds with the >> >> > standard >> >> > >> C++ >> >> > >>>> paradigm. >> >> > >>>> >> >> > >>>> Changing of the paradigm is obviously not anything I want to >> propose >> >> > at >> >> > >>>> this point, hence I don't see any other way as to just switch >> off the >> >> > >>>> warnings. >> >> > >>>> >> >> > >>>> Thanks, >> >> > >>>> Arkadiy >> >> > >>>> >> >> > >>>> >> >> > >>>> From: dev@arrow.apache.org At: 07/21/22 13:59:51 UTC-4:00To: >> >> > >>>> dev@arrow.apache.org >> >> > >>>> Cc: Arkadiy Vertleyb (BLOOMBERG/ 120 PARK ) >> >> > >>>> Subject: Re: Help needed with PR #13659: Fixing build/unit test >> issues >> >> > >> in >> >> > >>>> msvc/win32 >> >> > >>>> >> >> > >>>> Hi Arkadiy, >> >> > >>>> >> >> > >>>> I've been working on a PR for fixing 32-bit Visual Studio here >> which >> >> > has >> >> > >>>> some of the same changes. >> >> > >>>> It also enables the 32-bit VS build in CI, which fails due to a >> ton of >> >> > >>>> integer implicit cast warnings: >> >> > >>>> https://github.com/apache/arrow/pull/13532 >> >> > >>>> >> >> > >>>> Most of this commit is fixing 32-bit cast errors, along with a >> few >> >> > >> changes >> >> > >>>> to call bit_util::PopCount instead of ARROW_POPCOUNT64() >> >> > >>>> >> >> > >>>> On Thu, Jul 21, 2022 at 7:22 AM Raul Cumplido Dominguez >> >> > >>>> <r...@voltrondata.com.invalid> wrote: >> >> > >>>> >> >> > >>>>> Yes, issues 1-3 are not related to your PR. >> >> > >>>>> >> >> > >>>>> On Thu, Jul 21, 2022 at 4:04 PM Arkadiy Vertleyb (BLOOMBERG/ 120 >> >> > PARK) >> >> > >> < >> >> > >>>>> avertl...@bloomberg.net> wrote: >> >> > >>>>> >> >> > >>>>>> Thanks Raul. >> >> > >>>>>> >> >> > >>>>>> Does this mean issues 1-3 are not really caused by my PR and I >> just >> >> > >>>> need >> >> > >>>>>> to wait for them to be fixed? >> >> > >>>>>> >> >> > >>>>>> >> >> > >>>>>> From: dev@arrow.apache.org At: 07/21/22 09:51:09 UTC-4:00To: >> >> > Arkadiy >> >> > >>>>>> Vertleyb (BLOOMBERG/ 120 PARK ) , dev@arrow.apache.org >> >> > >>>>>> Subject: Re: Help needed with PR #13659: Fixing build/unit test >> >> > issues >> >> > >>>> in >> >> > >>>>>> msvc/win32 >> >> > >>>>>> >> >> > >>>>>> Hi Arkadiy, >> >> > >>>>>> >> >> > >>>>>> For issues 2 and 3 there is currently an issue [1] with the >> protobuf >> >> > >>>>>> version [2] distributed with homebrew [3] happening on master. >> These >> >> > >>>> ones >> >> > >>>>>> should be fixed once the upstream homebrew package is >> distributed. >> >> > >>>>>> Issue 1 is also happening on master and I am not sure whether >> the >> >> > >> issue >> >> > >>>>> is >> >> > >>>>>> tracked independently but there was a fix [4] on a PR [5]. I'll >> >> > follow >> >> > >>>>> that >> >> > >>>>>> one up. >> >> > >>>>>> >> >> > >>>>>> Thanks, >> >> > >>>>>> Raúl >> >> > >>>>>> >> >> > >>>>>> [1] https://issues.apache.org/jira/browse/ARROW-17162 >> >> > >>>>>> [2] https://github.com/protocolbuffers/protobuf/pull/10271 >> >> > >>>>>> [3] https://github.com/Homebrew/homebrew-core/pull/106252 >> >> > >>>>>> [4] >> >> > >>>>>> >> >> > >>>>>> >> >> > >>>>> >> >> > >>>> >> >> > >>>> >> >> > >>> >> >> > >> >> >> > >> >> > >> >> >> https://github.com/apache/arrow/pull/13634/commits/9e10f6c3399d83ebce5af551561fa >> >> > >>>>>> 3a16da9cd5e >> >> > >>>>>> < >> >> > >>>>> >> >> > >>>> >> >> > >>>> >> >> > >>> >> >> > >> >> >> > >> >> > >> >> >> https://github.com/apache/arrow/pull/13634/commits/9e10f6c3399d83ebce5af551561fa >> >> > >>>> 3a16da9cd5e >> >> > >>>> >> >> > >>> < >> >> > >> >> >> > >> >> > >> >> >> https://github.com/apache/arrow/pull/13634/commits/9e10f6c3399d83ebce5af551561fa >> >> > 3a16da9cd5e >> >> > >> >> < >> https://github.com/apache/arrow/pull/13634/commits/9e10f6c3399d83ebce5af551561fa3a16da9cd5e >> > >> >> > >>> >> >> > >>>>>> >> >> > >>>>>> [5] https://github.com/apache/arrow/pull/13634 >> >> > >>>>>> >> >> > >>>>>> On Thu, Jul 21, 2022 at 3:24 PM Arkadiy Vertleyb (BLOOMBERG/ >> 120 >> >> > PARK) >> >> > >>>> < >> >> > >>>>>> avertl...@bloomberg.net> wrote: >> >> > >>>>>> >> >> > >>>>>>> Hi all. >> >> > >>>>>>> >> >> > >>>>>>> Can someone help me understand how the changes in this PR ( >> >> > >>>>>>> >> >> > >>>>>> >> >> > >>>>>> >> >> > >>>>> >> >> > >>>> >> >> > >>>> >> >> > >>> >> >> > >> >> >> > >> >> > >> >> >> https://github.com/apache/arrow/pull/13659/commits/e77ec9a84dab750bf016f9f5bd02e >> >> > >>>>>> a48f2c8d77f >> >> > >>>>>> < >> >> > >>>>> >> >> > >>>> >> >> > >>>> >> >> > >>> >> >> > >> >> >> > >> >> > >> >> >> https://github.com/apache/arrow/pull/13659/commits/e77ec9a84dab750bf016f9f5bd02e >> >> > >>>> a48f2c8d77f >> >> > >>>> >> >> > >>> < >> >> > >> >> >> > >> >> > >> >> >> https://github.com/apache/arrow/pull/13659/commits/e77ec9a84dab750bf016f9f5bd02e >> >> > a48f2c8d77f >> >> > >> >> < >> https://github.com/apache/arrow/pull/13659/commits/e77ec9a84dab750bf016f9f5bd02ea48f2c8d77f >> > >> >> > >>> >> >> > >>>>>> >> >> > >>>>>> ) >> >> > >>>>>>> caused the following build failures? >> >> > >>>>>>> >> >> > >>>>>>> Thanks, >> >> > >>>>>>> Arkadiy >> >> > >>>>>>> >> >> > >>>>>>> Here are the failures: >> >> > >>>>>>> >> >> > >>>>>>> 1) AMD64 MacOS 10.15 GLib & Ruby >> >> > >>>>>>> >> >> > >>>>>>> c_glib/arrow-glib/meson.build:216:0: ERROR: Program >> 'glib-mkenums >> >> > >>>>>> mkenums' >> >> > >>>>>>> not found or not executable >> >> > >>>>>>> >> >> > >>>>>>> 2) AMD64 MacOS 10.15 Python 3 >> >> > >>>>>>> >> >> > >>>>>>> E ImportError: >> >> > >>>> dlopen(/usr/local/lib/python3.9/site-packages/pyarrow/ >> >> > >>>>>>> lib.cpython-39-darwin.so, 2): Symbol not found: >> >> > >>>>>>> __ZN6google8protobuf8internal16InternalMetadataD1Ev >> >> > >>>>>>> E Referenced from: /usr/local/lib/libarrow.900.dylib >> >> > >>>>>>> E Expected in: flat namespace >> >> > >>>>>>> E in /usr/local/lib/libarrow.900.dylib >> >> > >>>>>>> >> >> > >>>>>>> 3) AMD64 MacOS 10.15 C++ >> >> > >>>>>>> >> >> > >>>>>>> Undefined symbols for architecture x86_64: >> >> > >>>>>>> >> >> > >>>> >> >> > "google::protobuf::internal::InternalMetadata::~InternalMetadata()", >> >> > >>>>>>> referenced from: >> >> > >>>>>>> google::protobuf::MessageLite::~MessageLite() in >> >> > >>>>>>> libopentelemetry_proto.a(trace_service.pb.cc.o) >> >> > >>>>>>> google::protobuf::MessageLite::~MessageLite() in >> >> > >>>>>>> libopentelemetry_proto.a(trace.pb.cc.o) >> >> > >>>>>>> google::protobuf::MessageLite::~MessageLite() in >> >> > >>>>>>> libopentelemetry_proto.a(common.pb.cc.o) >> >> > >>>>>>> google::protobuf::MessageLite::~MessageLite() in >> >> > >>>>>>> libopentelemetry_proto.a(resource.pb.cc.o) >> >> > >>>>>>> ld: symbol(s) not found for architecture x86_64 >> >> > >>>>>>> >> >> > >>>>>>> 4) AMD64 Windows 2019 Win32 C++17 >> >> > >>>>>>> >> >> > >>>>>>> -- Could NOT find SnappyAlt (missing: Snappy_LIB >> >> > Snappy_INCLUDE_DIR) >> >> > >>>>>>> -- Building snappy from source >> >> > >>>>>>> CMake Error at C:/Program >> >> > >>>>>>> >> >> > >>>>>> >> >> > >>>>> >> >> > >>>> >> >> > >> >> >> > >> Files/CMake/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 >> >> > >>>>>>> (message): >> >> > >>>>>>> Could NOT find OpenSSL, try to set the path to OpenSSL >> root >> >> > folder >> >> > >>>> in >> >> > >>>>>> the >> >> > >>>>>>> system variable OPENSSL_ROOT_DIR (missing: >> >> > OPENSSL_CRYPTO_LIBRARY) >> >> > >>>>>> (found >> >> > >>>>>>> suitable version "1.1.1i", minimum required is "1.0.2") >> >> > >>>>>>> Call Stack (most recent call first): >> >> > >>>>>>> C:/Program >> >> > >>>>>>> >> >> > >>>>>> >> >> > >>>>> >> >> > >>>> >> >> > >> >> >> > >> Files/CMake/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 >> >> > >>>>>>> (_FPHSA_FAILURE_MESSAGE) >> >> > >>>>>>> C:/Program >> >> > >>>> Files/CMake/share/cmake-3.23/Modules/FindOpenSSL.cmake:578 >> >> > >>>>>>> (find_package_handle_standard_args) >> >> > >>>>>>> cmake_modules/ThirdpartyToolchain.cmake:1253 >> (find_package) >> >> > >>>>>>> CMakeLists.txt:575 (include) >> >> > >>>>>>> >> >> > >>>>>>> >> >> > >>>>>>> >> >> > >>>>>> >> >> > >>>>>> >> >> > >>>>>> >> >> > >>>>> >> >> > >>>> >> >> > >>>> >> >> > >>>> -- >> >> > >>>> >> >> > >>>> *James Duong* >> >> > >>>> Lead Software Developer >> >> > >>>> Bit Quill Technologies Inc. >> >> > >>>> Direct: +1.604.562.6082 | jam...@bitquilltech.com >> >> > >>>> https://www.bitquilltech.com >> >> > >>>> >> >> > >>>> This email message is for the sole use of the intended >> recipient(s) >> >> > and >> >> > >> may >> >> > >>>> contain confidential and privileged information. Any >> unauthorized >> >> > >> review, >> >> > >>>> use, disclosure, or distribution is prohibited. If you are not >> the >> >> > >>>> intended recipient, please contact the sender by reply email and >> >> > destroy >> >> > >>>> all copies of the original message. Thank you. >> >> > >>>> >> >> > >>>> >> >> > >>>> >> >> > >>> >> >> > >> >> >> > > >> >> > >> >> > >> >> > >> >> >> >> >> > > > -- > > *James Duong* > Lead Software Developer > Bit Quill Technologies Inc. > Direct: +1.604.562.6082 | jam...@bitquilltech.com > https://www.bitquilltech.com > > This email message is for the sole use of the intended recipient(s) and may > contain confidential and privileged information. Any unauthorized review, > use, disclosure, or distribution is prohibited. If you are not the > intended recipient, please contact the sender by reply email and destroy > all copies of the original message. Thank you.
