I forked apache/spark project and propose a job( https://github.com/theopenlab/spark/pull/1) for spark building in OpenLab ARM instance, this is the first step to build spark on ARM, I can enable a periodic job for arm building for apache/spark master if you guys like. Later I will run tests for spark. I also willing to be the maintainer of the arm ci of spark.
Thanks for you attention. On Thu, Jun 20, 2019 at 10:17 AM Tianhua huang <huangtianhua...@gmail.com> wrote: > Thanks Sean. > > I am very happy to hear that the community will put effort to fix the > ARM-related issues. I'd be happy to help if you like. And could you give > the trace link of this issue, then I can check it is fixed or not, thank > you. > As far as I know the old versions of spark support ARM, and now the new > versions don't, this just shows that we need a CI to check whether the > spark support ARM and whether some modification break it. > I will add a demo job in OpenLab to build spark on ARM and do a simple UT > test. Later I will give the job link. > > Let me know what you think. > > Thank you all! > > > On Wed, Jun 19, 2019 at 8:47 PM Sean Owen <sro...@gmail.com> wrote: > >> I'd begin by reporting and fixing ARM-related issues in the build. If >> they're small, of course we should do them. If it requires significant >> modifications, we can discuss how much Spark can support ARM. I don't >> think it's yet necessary for the Spark project to run these CI builds >> until that point, but it's always welcome if people are testing that >> separately. >> >> On Wed, Jun 19, 2019 at 7:41 AM Holden Karau <hol...@pigscanfly.ca> >> wrote: >> > >> > Moving to dev@ for increased visibility among the developers. >> > >> > On Wed, Jun 19, 2019 at 1:24 AM Tianhua huang < >> huangtianhua...@gmail.com> wrote: >> >> >> >> Thanks for your reply. >> >> >> >> As I said before, I met some problem of build or test for spark on >> aarch64 server, so it will be better to have the ARM CI to make sure the >> spark is compatible for AArch64 platforms. >> >> >> >> I’m from OpenLab team(https://openlabtesting.org/ ,a community to do >> open source project testing. And we can support some Arm virtual machines >> to AMPLab Jenkins, and also we have a developer team that willing to work >> on this, we willing to maintain build CI jobs and address the CI issues. >> What do you think? >> >> >> >> >> >> Thanks for your attention. >> >> >> >> >> >> On Wed, Jun 19, 2019 at 6:39 AM shane knapp <skn...@berkeley.edu> >> wrote: >> >>> >> >>> yeah, we don't have any aarch64 systems for testing... this has been >> asked before but is currently pretty low on our priority list as we don't >> have the hardware. >> >>> >> >>> sorry, >> >>> >> >>> shane >> >>> >> >>> On Mon, Jun 10, 2019 at 7:08 PM Tianhua huang < >> huangtianhua...@gmail.com> wrote: >> >>>> >> >>>> Hi, sorry to disturb you. >> >>>> The CI testing for apache spark is supported by AMPLab Jenkins, and >> I find there are some computers(most of them are Linux (amd64) arch) for >> the CI development, but seems there is no Aarch64 computer for spark CI >> testing. Recently, I build and run test for spark(master and branch-2.4) on >> my arm server, and unfortunately there are some problems, for example, ut >> test is failed due to a LEVELDBJNI native package, the details for java >> test see http://paste.openstack.org/show/752063/ and python test see >> http://paste.openstack.org/show/752709/ >> >>>> So I have a question about the ARM CI testing for spark, is there >> any plan to support it? Thank you very much and I will wait for your reply! >> >>> >> >>> >> >>> >> >>> -- >> >>> Shane Knapp >> >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >> >>> https://rise.cs.berkeley.edu >> > >> > >> > >> > -- >> > Twitter: https://twitter.com/holdenkarau >> > Books (Learning Spark, High Performance Spark, etc.): >> https://amzn.to/2MaRAG9 >> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau >> >
