Thanks Shane :) This sounds good, and yes I agree that it's best to keep the test/build infrastructure in one place. If you can't find the ARM resource we are willing to support the ARM instance :) Our goal is to make more open source software to be more compatible for aarch64 platform, so let's to do it. I will be happy if I can give some help for the goal.
Waiting for you good news :) On Wed, Jun 26, 2019 at 9:47 AM shane knapp <skn...@berkeley.edu> wrote: > ...or via VM as you mentioned earlier. :) > > shane (who will file a JIRA tomorrow) > > On Tue, Jun 25, 2019 at 6:44 PM shane knapp <skn...@berkeley.edu> wrote: > >> i'd much prefer that we keep the test/build infrastructure in one place. >> >> we don't have ARM hardware, but there's a slim possibility i can scare >> something up in our older research stock... >> >> another option would be to run the build in a arm-based docker container, >> which (according to the intarwebs) is possible. >> >> shane >> >> On Tue, Jun 25, 2019 at 6:35 PM Tianhua huang <huangtianhua...@gmail.com> >> wrote: >> >>> I forked apache/spark project and propose a job( >>> https://github.com/theopenlab/spark/pull/1) for spark building in >>> OpenLab ARM instance, this is the first step to build spark on ARM, I can >>> enable a periodic job for arm building for apache/spark master if you >>> guys like. Later I will run tests for spark. I also willing to be the >>> maintainer of the arm ci of spark. >>> >>> Thanks for you attention. >>> >>> On Thu, Jun 20, 2019 at 10:17 AM Tianhua huang < >>> huangtianhua...@gmail.com> wrote: >>> >>>> Thanks Sean. >>>> >>>> I am very happy to hear that the community will put effort to fix the >>>> ARM-related issues. I'd be happy to help if you like. And could you give >>>> the trace link of this issue, then I can check it is fixed or not, thank >>>> you. >>>> As far as I know the old versions of spark support ARM, and now the new >>>> versions don't, this just shows that we need a CI to check whether the >>>> spark support ARM and whether some modification break it. >>>> I will add a demo job in OpenLab to build spark on ARM and do a simple >>>> UT test. Later I will give the job link. >>>> >>>> Let me know what you think. >>>> >>>> Thank you all! >>>> >>>> >>>> On Wed, Jun 19, 2019 at 8:47 PM Sean Owen <sro...@gmail.com> wrote: >>>> >>>>> I'd begin by reporting and fixing ARM-related issues in the build. If >>>>> they're small, of course we should do them. If it requires significant >>>>> modifications, we can discuss how much Spark can support ARM. I don't >>>>> think it's yet necessary for the Spark project to run these CI builds >>>>> until that point, but it's always welcome if people are testing that >>>>> separately. >>>>> >>>>> On Wed, Jun 19, 2019 at 7:41 AM Holden Karau <hol...@pigscanfly.ca> >>>>> wrote: >>>>> > >>>>> > Moving to dev@ for increased visibility among the developers. >>>>> > >>>>> > On Wed, Jun 19, 2019 at 1:24 AM Tianhua huang < >>>>> huangtianhua...@gmail.com> wrote: >>>>> >> >>>>> >> Thanks for your reply. >>>>> >> >>>>> >> As I said before, I met some problem of build or test for spark on >>>>> aarch64 server, so it will be better to have the ARM CI to make sure the >>>>> spark is compatible for AArch64 platforms. >>>>> >> >>>>> >> I’m from OpenLab team(https://openlabtesting.org/ ,a community to >>>>> do open source project testing. And we can support some Arm virtual >>>>> machines to AMPLab Jenkins, and also we have a developer team that willing >>>>> to work on this, we willing to maintain build CI jobs and address the CI >>>>> issues. What do you think? >>>>> >> >>>>> >> >>>>> >> Thanks for your attention. >>>>> >> >>>>> >> >>>>> >> On Wed, Jun 19, 2019 at 6:39 AM shane knapp <skn...@berkeley.edu> >>>>> wrote: >>>>> >>> >>>>> >>> yeah, we don't have any aarch64 systems for testing... this has >>>>> been asked before but is currently pretty low on our priority list as we >>>>> don't have the hardware. >>>>> >>> >>>>> >>> sorry, >>>>> >>> >>>>> >>> shane >>>>> >>> >>>>> >>> On Mon, Jun 10, 2019 at 7:08 PM Tianhua huang < >>>>> huangtianhua...@gmail.com> wrote: >>>>> >>>> >>>>> >>>> Hi, sorry to disturb you. >>>>> >>>> The CI testing for apache spark is supported by AMPLab Jenkins, >>>>> and I find there are some computers(most of them are Linux (amd64) arch) >>>>> for the CI development, but seems there is no Aarch64 computer for spark >>>>> CI >>>>> testing. Recently, I build and run test for spark(master and branch-2.4) >>>>> on >>>>> my arm server, and unfortunately there are some problems, for example, ut >>>>> test is failed due to a LEVELDBJNI native package, the details for java >>>>> test see http://paste.openstack.org/show/752063/ and python test see >>>>> http://paste.openstack.org/show/752709/ >>>>> >>>> So I have a question about the ARM CI testing for spark, is there >>>>> any plan to support it? Thank you very much and I will wait for your >>>>> reply! >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> -- >>>>> >>> Shane Knapp >>>>> >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>>>> >>> https://rise.cs.berkeley.edu >>>>> > >>>>> > >>>>> > >>>>> > -- >>>>> > Twitter: https://twitter.com/holdenkarau >>>>> > Books (Learning Spark, High Performance Spark, etc.): >>>>> https://amzn.to/2MaRAG9 >>>>> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>>> >>>> >> >> -- >> Shane Knapp >> UC Berkeley EECS Research / RISELab Staff Technical Lead >> https://rise.cs.berkeley.edu >> > > > -- > Shane Knapp > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu >
