Hi All, I have some updates, our team has succesfully donated ARM resources and setup an ARM CI for Apache Spark last week, and it has been succesfully running for several times: https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-arm/ it will set to periodic job and then PR trigger when we think it is stable enough. And it includes some basic YARN tests, and it seems OK.
Probably this could be some example for evaluation of my proposal. I really hope we can do the same for YARN. Thanks alot. On Mon, Oct 14, 2019 at 11:06 AM Zhenyu Zheng <[email protected]> wrote: > Hi All, > > Sorry for flooding the channel, but I really want to promote this for a > wider discussion. > > As YARN is the resource manager of Hadoop and there are large number of > other software that also uses YARN for resource management. The capability > of running YARN on platforms with different architecture and managing > hardware resources with different architecture could be very important and > useful. > > Aarch64(ARM) architecture is currently the dominate architecture in small > devices like phone, IOT devices, security cameras, drones etc. With the > increasing compuiting capability and the increasing connection speed like > 5G network, there could be greate posibility and opportunity for world > chaging inovations and new market if we can managing and make use of those > devices as well. > > Currently, all YARN CIs are based on x86 architecture and we have been > performing tests on Aarch64 and proposing possible solutions for problems > we have meet, like: > https://issues.apache.org/jira/browse/HADOOP-16614 > we have done all YARN tests and it turns out there are only a few > problems, and we can provide possible solutions for discussion. > > We want to propose to add an Aarch64 CI for YARN to promote the support > for YARN on Aarch64 platforms. We are willing to provide machines to the > current CI system and manpower to mananging the CI and fxing problems that > occours. > > I've also submitted a Jira issue here: > https://issues.apache.org/jira/browse/YARN-9897 > comments and thoughts are welcome. > > Thanks alot, > > Kevin Zheng >
