Yikun commented on pull request #35088: URL: https://github.com/apache/spark/pull/35088#issuecomment-1080131756
Thanks for your attention, I've been busy with other things lately. The support of this arm64, there is no progress for the time being, but I will continue this work. The purpose of this PR is to (same with https://github.com/apache/spark/pull/35049): 1. Find some problems of Spark on arm in time, and report them to the community quickly. 2. Provide an open mechanism that allows Spark developers to quickly verify on arm: Just like [some previous PR](https://github.com/apache/spark/pull/35078#issuecomment-1004516832) did by me manually. I've tried a few things ([support pyspark arm64 ci images](https://github.com/dongjoon-hyun/ApacheSparkGitHubActionImage/pull/6), [self-hosted demo](https://lists.apache.org/thread/mq121fmggs9bbo5ll4ld1qqjv7cshldp)), but there seem to be some diff suggestions in the community. So I temporarily use the downstream repo to ensure the verification of spark on arm linux: https://github.com/spark-arm/spark/actions/workflows/build_and_test.yml (However, tasks often fail due to conflicts with upstream.) What's the next step: 1. Backport some useful configurations to upstream spark and using workflow_run way to make configuration separately to reduce merge conflict. 2. Add workflow trigger on demand in downstream.  Of course, I also very much hope that it can be truly integrated in the upstream community, thereby reducing downstream costs and ensuring that spark on arm64 passed always. feel free to ping me if you have any other suggestion! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
