[GitHub] [arrow] eladroz edited a comment on pull request #9285: ARROW-10349: [Python] Build and publish aarch64 wheels [WIP]
eladroz edited a comment on pull request #9285: URL: https://github.com/apache/arrow/pull/9285#issuecomment-766086589 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] eladroz edited a comment on pull request #9285: ARROW-10349: [Python] Build and publish aarch64 wheels [WIP]
eladroz edited a comment on pull request #9285: URL: https://github.com/apache/arrow/pull/9285#issuecomment-766086589 @xhochy @kszucs hi, I've published my experience with building on AWS arm64 VM's [here](https://lists.apache.org/thread.html/r274a6f9a3d0133fe3b247ea9bb1da6c6f24fc1511508d1b36ead0ae1%40%3Cuser.arrow.apache.org%3E), but once I've tried to added ARROW_S3 component I got the same issue as above However, compiling the AWS SDK from source rather than through vcpkg does seem to work after some fighting with it - I cloned latest 1.8.129 tag rather than 1.8.126 which vcpkg currently pulls). Is that an option? if so, I'll document the steps. I've proceeded with buillding manually, to see what the next issues are: * Turned `-DARROW_GANDIVA=OFF` (some issue with llvm for aarch64) * Did not attempt to turn on CUDA yet Following the above -> Arrow C++ libs were built. Tests passed except for: ``` The following tests FAILED: 41 - arrow-s3fs-test (Failed) ``` (didn't go into that yet) Moving on to PyArrow, built with all components except `PYARROW_WITH_GANDIVA` and `CUDA`: * Building with `PYARROW_WITH_TENSORFLOW` - I got [this existing issue](https://github.com/apache/arrow/issues/7065) when running tests * Turning off TF component, build passed. * Tests passed, some are skipped: `3329 passed, 351 skipped, 17 xfailed, 49 warnings in 53.40s` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] eladroz edited a comment on pull request #9285: ARROW-10349: [Python] Build and publish aarch64 wheels [WIP]
eladroz edited a comment on pull request #9285: URL: https://github.com/apache/arrow/pull/9285#issuecomment-766086589 @xhochy @kszucs hi, I've published my experience with building on AWS arm64 VM's [here](https://lists.apache.org/thread.html/r274a6f9a3d0133fe3b247ea9bb1da6c6f24fc1511508d1b36ead0ae1%40%3Cuser.arrow.apache.org%3E), but once I've tried to added ARROW_S3 component I got the same issue as above However, compiling the AWS SDK from source rather than through vcpkg does seem to work after some fighting with it - I cloned latest 1.7.x tag rather than 1.8.126 which vcpkg currently pulls). Is that an option? if so, I'll document the steps. I've proceeded with buillding manually, to see what the next issues are: * Turned `-DARROW_GANDIVA=OFF` (some issue with llvm for aarch64) * Needed to copy OpenSSL (`libcrypto.a`) that was fetched by vcpkg to my `/usr/local/lib64`, instead of the one built when compiling the AWS SDK locally (which seems to have an issue with arm-related symbol `OPENSSL_armcap_P`) * Did not attempt to turn on CUDA yet Following the above -> Arrow C++ libs were built. Tests passed except for: ``` The following tests FAILED: 41 - arrow-s3fs-test (Failed) 43 - arrow-flight-test (Failed) 50 - arrow-python-test (Failed) ``` (didn't go into that yet) Moving on to PyArrow, built with all components except `PYARROW_WITH_GANDIVA` and `_CUDA`: * Building with `PYARROW_WITH_TENSORFLOW` - I got [this existing issue](https://github.com/apache/arrow/issues/7065) when running tests * Turning off TF component, build passed. * Tests passed, some are skipped: `3329 passed, 351 skipped, 17 xfailed, 49 warnings in 53.40s` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] eladroz edited a comment on pull request #9285: ARROW-10349: [Python] Build and publish aarch64 wheels [WIP]
eladroz edited a comment on pull request #9285: URL: https://github.com/apache/arrow/pull/9285#issuecomment-766086589 @xhochy @kszucs hi, I've published my experience with building on AWS arm64 VM's [here](https://lists.apache.org/thread.html/r274a6f9a3d0133fe3b247ea9bb1da6c6f24fc1511508d1b36ead0ae1%40%3Cuser.arrow.apache.org%3E), but once I've tried to added ARROW_S3 component I got the same issue as above However, compiling the AWS SDK from source rather than through vcpkg does seem to work after some fighting with it. Is that an option? if so, I'll document the steps. I've proceeded with buillding manually, to see what the next issues are: * Turned `-DARROW_GANDIVA=OFF` (some issue with llvm for aarch64) * Needed to copy OpenSSL (`libcrypto.a`) that was fetched by vcpkg to my `/usr/local/lib64`, instead of the one built when compiling the AWS SDK locally (which seems to have an issue with arm-related symbol `OPENSSL_armcap_P`) * Did not attempt to turn on CUDA yet Following the above -> Arrow C++ libs were built. Tests passed except for: ``` The following tests FAILED: 41 - arrow-s3fs-test (Failed) 43 - arrow-flight-test (Failed) 50 - arrow-python-test (Failed) ``` (didn't go into that yet) Moving on to PyArrow, built with all components except `PYARROW_WITH_GANDIVA` and `_CUDA`: * Building with `PYARROW_WITH_TENSORFLOW` - I got [this existing issue](https://github.com/apache/arrow/issues/7065) when running tests * Turning off TF component, build passed. * Tests passed, some are skipped: `3329 passed, 351 skipped, 17 xfailed, 49 warnings in 53.40s` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] eladroz edited a comment on pull request #9285: ARROW-10349: [Python] Build and publish aarch64 wheels [WIP]
eladroz edited a comment on pull request #9285: URL: https://github.com/apache/arrow/pull/9285#issuecomment-766086589 @xhochy @kszucs hi, I've published my experience with building on AWS arm64 VM's [here](https://lists.apache.org/thread.html/r274a6f9a3d0133fe3b247ea9bb1da6c6f24fc1511508d1b36ead0ae1%40%3Cuser.arrow.apache.org%3E), but once I've tried to added ARROW_S3 component I got the same issue as above However, compiling the AWS SDK from source rather than through vcpkg does seem to work after some fighting with it. Is that an option? if so, I'll document the steps. I've proceeding with buillding manually, to see what the next issues are: * Turned `-DARROW_GANDIVA=OFF` (some issue with llvm for aarch64) * Needed to copy OpenSSL (`libcrypto.a`) that was fetched by vcpkg to my `/usr/local/lib64`, instead of the one built when compiling the AWS SDK locally (which seems to have an issue with arm-related symbol `OPENSSL_armcap_P`) * Did not attempt to turn on CUDA yet Following the above -> Arrow C++ libs were built. Tests passed except for: ``` The following tests FAILED: 41 - arrow-s3fs-test (Failed) 43 - arrow-flight-test (Failed) 50 - arrow-python-test (Failed) ``` (didn't go into that yet) Moving on to PyArrow, built with all components except `PYARROW_WITH_GANDIVA` and `_CUDA`: * Building with `PYARROW_WITH_TENSORFLOW` - I got [this existing issue](https://github.com/apache/arrow/issues/7065) when running tests * Turning off TF component, build passed. * Tests passed, some are skipped: `3329 passed, 351 skipped, 17 xfailed, 49 warnings in 53.40s` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] eladroz edited a comment on pull request #9285: ARROW-10349: [Python] Build and publish aarch64 wheels [WIP]
eladroz edited a comment on pull request #9285: URL: https://github.com/apache/arrow/pull/9285#issuecomment-766086589 @xhochy @kszucs hi, I've published my experience with building on AWS arm64 VM's [here](https://lists.apache.org/thread.html/r274a6f9a3d0133fe3b247ea9bb1da6c6f24fc1511508d1b36ead0ae1%40%3Cuser.arrow.apache.org%3E), but once I've tried to added ARROW_S3 component I got the same issue as above However, compiling the AWS SDK from source rather than through vcpkg does seem to work after some fighting with it. Is that an option? if so, I'll document the steps. I'm proceeding with buillding manually, seeing what the next issues are. Currently turned Gandiva component OFF (something with llvm for aarch64) and now seeing some issue with Plasma. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
