It came through ok - https://lists.apache.org/list.html?bui...@impala.apache.org
On Thu, Oct 8, 2020 at 6:42 PM Tianhua huang <huangtianhua...@gmail.com> wrote: > Hi Jim and Tim, > > Thank you very much. > I am back from vacation, sorry to reply so late, I will configure the arm > job to send the test result to bui...@impala.apache.org today. > > On Tue, Oct 6, 2020 at 7:50 AM Jim Apple <apa...@jbapple.com> wrote: > > > I did a test on EC2 Graviton 2, and I was able to start from a blank > Ubuntu > > 18.04 image and get all the way to all tests passing. This *included > > building the native-toolchain from scratch, with no manual input > required* > > - it just bootstrapped with no fuss! I used a r6g.2xlarge, which has a > list > > price of 40 cents per hour, though you have to pay for EBS, too. If we > get > > to a place where we want this to be in the gerrit merge step, I expect it > > to be smooth sailing. > > > > I'm really impressed with those of you who have worked on the pieces of > > this to put it together! Thank you for your contributions to the > community! > > > > Here, roughly, are how one could duplicate my test: > > > > 1. Launch an r6g.2xlarge using Ubuntu 18.04, Arm64, with a 500GB root > > volume. Leave the following ports open: > > - $YOUR_IP, udp 60000-61000 for mosh, in both IPV4 and IPv6 > > - $YOUR_IP, tcp 22 for ssh, in both IPV4 and IPv6 > > - All TCP and UDP from itself, for loopback > > 2. ssh in and > > - apt-get update > > - apt-get install tmux git mosh > > 3. Terminate your ssh session and connect again, this time with mosh > > 4. Start a tmux session > > 5. git clone https://github.com/apache/impala > > 6. cd impala > > 7. export IMPALA_HOME=$(pwd) > > 8. ./bin/bootstrap_development.sh && ./bin/run-all-tests.sh > > > > I'd love to see a performance comparison between two different equal-cost > > clusters on AWS: one Arm64, one x86-64, but I'll leave that to the > testers > > with larger budgets of both time and money! :-D > > > > On Fri, Oct 2, 2020 at 11:05 AM Jim Apple <apa...@jbapple.com> wrote: > > > > > SG; submitted request for builds@ > > > > > > Tim, I made you and I moderators. You're welcome to decline. :-) > > > Volunteers are welcome, but I expect zero moderation work - I set the > > list > > > to "subscribers can post; all others are rejected", so no moderation > > > messages should show up. > > > > > > The form said it will be acted on in 24 hours and the PMC will be > > informed > > > when the mailing list is ready. > > > > > > Also, FWIW, aside from Docker, AWS supports some ARM EC2 types. > > > > > > On Thu, Oct 1, 2020 at 4:22 PM Tim Armstrong <tarmstr...@cloudera.com> > > > wrote: > > > > > >> I think we got stalled here. I never got anywhere with configuring > > >> commits@ > > >> to accept emails from openlab and not sure that it's possible. > > >> > > >> In order to get something going, my suggestion is to copy what Kudu > did > > >> and > > >> have this go to a custom mailing list. If you follow the thread here > it > > >> describes what they did > > >> > > >> > > > https://lists.apache.org/thread.html/rfcd230882ec34171462a0c8c53ef6e8efcad50c9d67f8da18a508d07%40%3Cdev.kudu.apache.org%3E > > >> > > >> Jim, it looks like the PMC chair is needed to create a new mailing > list > > >> via > > >> https://selfserve.apache.org/ (see > > >> https://issues.apache.org/jira/browse/INFRA-20604), so maybe we can > go > > >> ahead with that route? > > >> > > >> - Tim > > >> > > >> I think it would be good to run the tests on impala.jenkins.io but I > > >> don't > > >> want to hold up things further. Another interesting technology is the > > ARM > > >> emulation support in Docker. > > >> > > >> On Sun, Sep 20, 2020 at 8:53 PM Jim Apple <jbap...@apache.org> wrote: > > >> > > >> > I'm very excited about this development! I don't see any jobs at > > >> > status.openlabtesting.org, though -- is > > >> > > > >> > > http://status.openlabtesting.org/builds?job_name=impala-build-test-arm64 > > >> > the right link? Maybe only the owner of a job can see it? > > >> > > > >> > As far as the price, based on https://ec2instances.info/, you might > > be > > >> > able > > >> > to eyeball the cost. I'm no longer a Jenkins admin, so I'll let the > > >> other > > >> > community members who are admins speak more to this, if it seems > > >> > feasible to them. > > >> > > > >> > On Sat, Sep 19, 2020 at 2:31 AM Tianhua huang < > > >> huangtianhua...@gmail.com> > > >> > wrote: > > >> > > > >> > > I am sorry there is no API with Openlab. But we use ZUUL in our > > >> system, > > >> > you > > >> > > can call ZUUL api to get the job information, > > >> > > simple like: curl > > >> > > > > >> > > > > >> > > > >> > > > http://status.openlabtesting.org/api/builds?job_name=impala-build-test-arm64 > > >> > > | python -m json.tool > > >> > > The result we will get is jobs info of 'impala-build-test-arm64', > > >> > including > > >> > > job status, job log url and so on, I am not sure if we can > integrate > > >> with > > >> > > the job info into impala.jenkins.io? Then the arm job still runs > > >> > triggered > > >> > > by OpenLab, and jenkins.impala.io only to get the result and show > > >> > > the result in it. > > >> > > > > >> > > Not sure it's worth doing this, maybe let > comm...@impala.apache.org > > >> > allow > > >> > > receiving the result email from Openlab is more easy:) > > >> > > > > >> > > > > >> > > > > >> > > On Sat, Sep 19, 2020 at 11:03 AM Shant Hovsepian < > > >> > > sh...@superdupershant.com> > > >> > > wrote: > > >> > > > > >> > > > Would it be possible to setup an API Key or webhook with > > >> > openlabstesting? > > >> > > > > > >> > > > Then we'd have an Impala Jenkins job that uses the API to call > > >> > > > openlabstesting to do the build and report the results back. > > >> > > > > > >> > > > -Shant > > >> > > > > > >> > > > On Fri, Sep 18, 2020 at 10:24 PM Tianhua huang < > > >> > > huangtianhua...@gmail.com> > > >> > > > wrote: > > >> > > > > > >> > > > > Hi Shant, > > >> > > > > > > >> > > > > Thanks for your reply, maybe it's a good idea to trigger arm > > >> testing > > >> > by > > >> > > > > jenkins.impala.io, we can donate an arm instance, but I don't > > >> know > > >> > how > > >> > > > to > > >> > > > > do it. Maybe the Jenkins owner of Impala can help me? > > >> > > > > > > >> > > > > > > >> > > > > On Sat, Sep 19, 2020 at 2:46 AM Shant Hovsepian < > > >> > > > sh...@superdupershant.com > > >> > > > > > > > >> > > > > wrote: > > >> > > > > > > >> > > > > > Great work Tianhua! > > >> > > > > > > > >> > > > > > This is a very exciting enhancement. > > >> > > > > > > > >> > > > > > dev@ isn't usually meant for automated test run output. > Would > > >> it > > >> > be > > >> > > > > > possible to get the openlabtesting runs triggered by > > >> > > jenkins.impala.io > > >> > > > > > then > > >> > > > > > have the results of the test run come from that jenkins > > >> instance? > > >> > > This > > >> > > > > > should solve the email problem as the notification can come > > from > > >> > the > > >> > > > > > jenkins account and still give us one centralized place to > > look > > >> at > > >> > > > > various > > >> > > > > > test statuses. > > >> > > > > > > > >> > > > > > -Shant > > >> > > > > > > > >> > > > > > On Thu, Sep 17, 2020 at 8:08 AM Tianhua huang < > > >> > > > huangtianhua...@gmail.com > > >> > > > > > > > >> > > > > > wrote: > > >> > > > > > > > >> > > > > > > Hi all, > > >> > > > > > > > > >> > > > > > > Impala can run/test on arm64 platform after our efforts, > the > > >> > issues > > >> > > > we > > >> > > > > > > fixed you can find in > > >> > > > > https://issues.apache.org/jira/browse/IMPALA-9376 > > >> > > > > > > > > >> > > > > > > We are particularly grateful to Tim and Quanlong for their > > >> help, > > >> > > > thanks > > >> > > > > > > very much! > > >> > > > > > > > > >> > > > > > > And I have integrated Impala arm64 CI with OpenLab( > > >> > > > > > > https://openlabtesting.org/ , a community to do open > source > > >> > > project > > >> > > > > > > testing), > > >> > > > > > > see > > >> > > > > > > > > >> > > > > > > >> > > > > >> > > http://status.openlabtesting.org/builds?job_name=impala-build-test-arm64 > > >> > > > > > > the > > >> > > > > > > periodic test job runs for a few days already and runs > once > > >> per > > >> > > day. > > >> > > > > > > > > >> > > > > > > Tim suggested us to send the arm64 test result to > > >> > > > > > > comm...@impala.apache.org, > > >> > > > > > > but seems it can receive the emails only ncome from @ > > >> apache.org. > > >> > > > > > > Now I wonder if we can send the test result to Impala > > >> community > > >> > > > > maillist > > >> > > > > > > here dev@impala.apache.org, to tell developers that > Impala > > >> can > > >> > > runs > > >> > > > on > > >> > > > > > > arm64, to find the compatibility in time. > > >> > > > > > > > > >> > > > > > > What do you think? Or maybe there is any other good > > >> suggestion? > > >> > > > > > > > > >> > > > > > > Waiting for your reply. > > >> > > > > > > > > >> > > > > > > Thank you very much! > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > >
