Hi All, A couple of months ago, I wrote to a few project private lists mentioning the need to migrate Hadoop labelled nodes (H0-H21) over to a new dedicated Jenkins Master [1] (a Cloudbees Client Master.).
I'd like to revisit this now that I have more time to dedicate to getting this done. However, keeping track across multiple mailing lists, separate conversations that spring up in various places is cumbersome and not realistic. To that end, I have created a new specific mailing list dedicated to the migrations of these nodes, and the projects that use them, over to the new system. The mailing list 'hadoop-migrati...@infra.apache.org' is up and running now (and this will be the first post to it). Previous discussions were on the private PMC lists, (there was some debate about that but I wanted the PMCs initially to be aware of the change,) this new list is public and archived. This email is BCC'd to 13 projects dev lists [2] determined by the https:/ hadoop.apache.org list of Related projects, minus Cassandra whom already have their own dedicated client master [3] and I added Yetus as I think they cross collaborate with many Hadoop based projects. If anyone thinks a project is missing, or should not be on the list, let me know. What I would like from each community, is to decide who is going to help with their project in performing these migrations - ideally 2 or 3 folks who use the current builds.a.o regularly. Those folks should then subscribe to the new dedicated hadoop-migrati...@infra.apache.org mailing lists as soon as possible so we can get started. About the current setup - and I hope this answers previously asked questions on private lists - the new dedicated master is a Cloudbees Client Master 2.204.3.7-rolling. It is not the same setup as the current Jenkins master on builds.a.o - it is not intended to be. It is more or less a 'clean install' in that I have not installed over 500 plugins as is the case on builds.a.o , I would rather we install plugins as we find we need them. So yes, there may be some features missing - the point of having people sign up to the new list is to find out what those are, get them installed, and get your builds to at least the same state they are in currently. We have 2 nodes on there currently for testing, as things progress we can transfer over a couple more, projects can start to migrate their jobs over at any time they are happy , until done. We also need to test auth - the master; and its nodes will be restricted to just Hadoop + Related projects (which is important this list of related projects is correct). No longer will other projects be able to hop on to Hadoop nodes, and no longer will Hadoop related projects be able to hop onto other folks nodes. This is a good thing, and may encourage some providers to donate a few more VMs for dedicated use. For now then, decide who will help with this process, and sign up to the new mailing list, and lets get started! Note I am NOT subscribed to any of your dev lists, so replies please cc the new list. and I will await your presence there to get started. Thanks all. Gavin McDonald (ASF Infra) [1] - https://ci-hadoop.apache.org [2] - hadoop,chukwa,avro,ambari,hbase,hive,mahout,pig,spark,submarine,tez,zookeeper,yetus [3] - https://ci-cassandra.apache.org
