Hi All,
A couple of months ago, I wrote to a few project private lists mentioning
the need to migrate Hadoop labelled nodes (H0-H21) over to a new dedicated
Jenkins Master [1] (a Cloudbees Client Master.).
I'd like to revisit this now that I have more time to dedicate to getting
this done. However, keeping track across multiple mailing lists,
separate conversations that spring up in various places is cumbersome and
not realistic. To that end, I have created a new specific mailing list
dedicated to the migrations of these nodes, and the projects that use them,
over to the new system.
The mailing list 'hadoop-migrati...@infra.apache.org' is up and running now
(and this will be the first post to it). Previous discussions were on the
private PMC lists, (there was some debate about that but I wanted the PMCs
initially to be aware of the change,) this new list is public and archived.
This email is BCC'd to 13 projects dev lists [2] determined by the https:/
hadoop.apache.org list of Related projects, minus Cassandra whom already
have their own dedicated client master [3] and I added Yetus as I think
they cross collaborate with many Hadoop based projects. If anyone thinks a
project is missing, or should not be on the list, let me know.
What I would like from each community, is to decide who is going to help
with their project in performing these migrations - ideally 2 or 3 folks
who use the current builds.a.o regularly. Those folks should then subscribe
to the new dedicated hadoop-migrati...@infra.apache.org mailing lists as
soon as possible so we can get started.
About the current setup - and I hope this answers previously asked
questions on private lists - the new dedicated master is a Cloudbees Client
Master 2.204.3.7-rolling. It is not the same setup as the current Jenkins
master on builds.a.o - it is not intended to be. It is more or less a
'clean install' in that I have not installed over 500 plugins as is the
case on builds.a.o , I would rather we install plugins as we find we need
them. So yes, there may be some features missing - the point of having
people sign up to the new list is to find out what those are, get them
installed, and get your builds to at least the same state they are in
currently.
We have 2 nodes on there currently for testing, as things progress we can
transfer over a couple more, projects can start to migrate their jobs over
at any time they are happy , until done. We also need to test auth - the
master; and its nodes will be restricted to just Hadoop + Related projects
(which is important this list of related projects is correct). No longer
will other projects be able to hop on to Hadoop nodes, and no longer will
Hadoop related projects be able to hop onto other folks nodes. This is a
good thing, and may encourage some providers to donate a few more VMs for
dedicated use.
For now then, decide who will help with this process, and sign up to the
new mailing list, and lets get started!
Note I am NOT subscribed to any of your dev lists, so replies please cc the
new list. and I will await your presence there to get started.
Thanks all.
Gavin McDonald (ASF Infra)
[1] - https://ci-hadoop.apache.org
[2] -
hadoop,chukwa,avro,ambari,hbase,hive,mahout,pig,spark,submarine,tez,zookeeper,yetus
[3] - https://ci-cassandra.apache.org
