If anyone would like to follow along in JIRA, here's the ticket
https://issues.apache.org/jira/browse/INFRA-19369. I've been updating
that ticket with any issues. arm-poc has been moved to a node in
Singapore and will need to be tested again with builds.
I'm going to mention again that someone from Hadoop should be changing
these builds in order to run against arm-poc. In my reply below, I
thought that the project knew about the ARM nodes and was involved with
setting up new builds, which is why I said I'd be willing to make simple
changes for testing. However I don't want to change things without the
knowledge of the project. The builds themselves are created by the
project, not Infra, which means I have no idea which build should run
against ARM vs any other CPU.
-Chris T.
#asfinfra
On 11/22/19 9:28 AM, Chris Thistlethwaite wrote:
In order to run builds against arm-poc, someone (me included) will
need to change a build config to only use that label. The node itself
isn't fully built out like our other ASF nodes, due to the fact that
it's ARM and we don't have all the packaged tools built for that
architecture, it will likely take some time to fix issues.
-Chris T.
#asfinfra
On 11/22/19 3:46 AM, bo zhaobo wrote:
Thanks. That would be great if a project can use the ARM test worker
to do the specific testing on ARM.
Also I think it's better to make @Chris Thistlethwaite
<mailto:chr...@apache.org> know this email. Could you please give
some kind advices? Thank you.
BR
ZhaoBo
Mailtrack
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&;>
Sender notified by
Mailtrack
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&;>
19/11/22 下午04:42:30
Zhenyu Zheng <zhengzhenyul...@gmail.com
<mailto:zhengzhenyul...@gmail.com>> 于2019年11月22日周五 下午4:32写道：
Hi Hadoop,
First off, I want to thanks to Wei-Chiu for having me on the last
week's Hadoop community sync to introduce our ideas of ARM
support on Hadoop. And also for all the attendees for listening
and providing suggestions.
I want to provide some update on the status:
1. Our teammate has successfully donated an ARM machine to the
ApacheInfra team, and it is setup for running:
https://builds.apache.org/computer/arm-poc/it might be a good
idea to make use of it, like running some periodic jobs for some
experiment, and it will also benifit us for discussions and
asking for help on identified problems.
2. I've been keep try to test and debug sub-project by
sub-project, and here is the current status for YARN:
When running the whole test suits, some of the test suit will be
skipped due to the rules of if some previous test fails, then
skip this suit. So I manually run those test suits again to see
if they can pass, the full test result is that:
Total: 5688; Failure: 0; Error 15; Skipped 60
Among the 15 errors, 13 of them came from the ``Apache Hadoop
YARN TimelineService HBase tests`` test suit. The other 2 came
from ``Apache Hadoop YARN DistributedShell`` suit.
3. Some walk-arounds:
1) The only walk-arounds for build Hadoop on ARM is to pre-build
grpc-java, which my teammates are working with the community to
release a newer version with ARM support:
github.com/grpc/grpc-java/issues/6364
<http://github.com/grpc/grpc-java/issues/6364>
2) For YARN tests, the TimelineService HBase suit need either
HBase 1.4.8 or 2.0.2 which can only be built under protocbuf
2.5.0(HBase 1.4.8, HBase 2.0.2 external) and protocbuf
3.5.1(HBase 2.0.2 internal), so we have to pre-build them. And
the new cause of the error is still under debugging.
3) The rest of the know issue and possible walk-arounds are
reported to Hadoop Jira and are now under Wei-Chiu's tent jira
report: https://issues.apache.org/jira/browse/HADOOP-16723
I have put all the test logs in the attachment and error related
surefire reports in my github
https://github.com/ZhengZhenyu/HadoopTestLogs/issues/1(the
attachment size is limited for sending mailling list), please
have a check if you are interested.
So, how should we move a little bit forward and make use of the
ARM resources in ApacheInfra?
Best Regards,
Zhenyu
