Hi Pono,
I’ve discussed with Mu about the process to migrate MXNet to Apache, and I
believe you’re aware too. I’ve documented the detailed steps below for open
feedback and discussion.
Essentially Amazon will be providing the GPU build slaves to be hooked into
Apache’s Jenkins build Master. We’ll first make sure that Apache can build a
fork of MXNet, before officially transferring ownership of the MXNet repo.
Steps to migration:
1. Provide Apache with Linux slaves & slave tags
a. Provide Apache with slave configuration (tags, remote root dir, etc.)
b. Spin up 6 slaves
c. Launch connection via JNLP
2. Apache forks MXNet repo and makes sure builds are successful on their
build set up
a. Ask Apache to give me committer rights
b. I remove the Windows jobs until a later time
c. Apache sets up Jenkins jobs and Github webhooks
i. Build
every commit and origin/fork PR’s without merge (main Jenkinsfile)
ii. Nightly
job (nightly Jenkins file, will start with a dummy one and add more
configurations later)
d. If Windows slave setup is available, provide it to Apache and enable
the jobs again
3. Transfer the repo and point the build set up there
4. Apache deploys the docs to their website
Open security questions:
1. How can we ensure that our slaves are not used by other projects?
a. It’s not, it’s a social contract.
2. To protect the slave hosts, would running Jenkins slave inside a Docker
container be a solution, or is there a recommended best practice?
a. Run slave behind a NAT gateway and launch via JNLP
3. Does Apache place SSH key inside the build host for Docs deployment to
the website? Are there security concerns there?
a. The only slaves that are allowed to deploy docs are ASF-controlled.
Just provide the build command.
