hello.the master ci test is broken,please download and decompression
http://data.mxnet.io/data/test_images.tar.gz
to c:\mxnet\data for every jenkins salve.

2017-07-11 3:41 GMT+08:00 Ly Nguyen <nguyen...@gmail.com>:

> Also, DON'T mark the branch as "*Require status checks to pass before
> merging*" since builds may be unstable at first. The existing configuration
> should be preserved after the transfer but we just have to validate that.
>
> https://help.github.com/articles/enabling-required-status-checks/
>
> On Mon, Jul 10, 2017 at 12:36 PM, Ly Nguyen <nguyen...@gmail.com> wrote:
>
> > From Mu:
> > Please preserve the following hooks after transferring:
> > -http://mxnet.io:8080/github-webhook/  (docs deployment)
> > -https://ci.appveyor.com/api/github/webhook
> > all other hooks can be removed
> >
> > On Mon, Jul 10, 2017 at 12:00 PM, Ly Nguyen <nguyen...@gmail.com> wrote:
> >
> >> @Suneel I don't think so. We'll just be transferring the organization of
> >> the GitHub repo from DMLC -> Apache.
> >>
> >>
> >> Pono has agreed to help with those steps from 2-3:30pm PDT today.
> Pending
> >> Mu adding Pono as an owner.
> >>
> >> On Sat, Jul 8, 2017 at 11:33 AM, Suneel Marthi <smar...@apache.org>
> >> wrote:
> >>
> >>> I suppose we'll be migrating to apache Gitbox, correct ??
> >>>
> >>> On Sat, Jul 8, 2017 at 1:48 PM, Li, Mu <m...@amazon.com> wrote:
> >>>
> >>> > Can we schedule a time on monday to finish the transfer together?
> >>> > ________________________________________
> >>> > From: Daniel Pono Takamori <p...@apache.org>
> >>> > Sent: Friday, July 7, 2017 5:22 PM
> >>> > To: dev@mxnet.incubator.apache.org
> >>> > Subject: Re: MXNet -> Apache Migration proposal
> >>> >
> >>> > This looks great Ly.  On Monday we can transfer the main repo over
> and
> >>> > then you can change the builds to point to the official repo.  I'll
> >>> > make a clone of the website next week so we can do a similar thing to
> >>> > test building the website and then we can publish it to
> >>> > mxnet.incubator.apache.org.  Once we have that worked out we can
> move
> >>> > forward with transferring the domain to our control.
> >>> >
> >>> > Have a great weekend everyone!
> >>> >
> >>> > On Fri, Jul 7, 2017 at 6:07 PM, Dominic Divakaruni
> >>> > <dominic.divakar...@gmail.com> wrote:
> >>> > > great stuff!! glad to see this getting close!
> >>> > >
> >>> > > On Fri, Jul 7, 2017 at 3:47 PM, Ly Nguyen <nguyen...@gmail.com>
> >>> wrote:
> >>> > >>
> >>> > >> We have successfully validated that merges and pull requests
> >>> against an
> >>> > >> Apache fork of MXNet runs successfully on builds.apache.org:
> >>> > >>
> >>> > >> https://builds.apache.org/blue/organizations/jenkins/
> >>> > incubator-mxnet-master2/detail/master/13/pipeline
> >>> > >>
> >>> > >> https://builds.apache.org/blue/organizations/jenkins/
> >>> > incubator-mxnet-master2/detail/PR-3/1/pipeline
> >>> > >>
> >>> > >> We have also added a dummy nightly run to be populated with builds
> >>> and
> >>> > >> test
> >>> > >> cases after migration, as discussed. We can now move forward with
> >>> the
> >>> > >> migration to Apache and I recommend the following steps:
> >>> > >> - [ ] Add Pono as owner
> >>> > >> - [ ] Pono adds Apache git hooks to MXNet repo
> >>> > >> - [ ] Change source control of Apache Jenkins jobs to point to
> MXNet
> >>> > repo,
> >>> > >> verify a run is successful
> >>> > >> - [ ] Change MXNet org to Apache, verify a run is successful,
> >>> mxnet.io
> >>> > >> still building
> >>> > >> - [ ] Start docs build to mxnet.apache.org
> >>> > >> Note that one kink to iron out is that PR build statuses aren’t
> >>> being
> >>> > >> updated. Here’s a ticket to follow:
> >>> > >>
> >>> > >> https://issues.apache.org/jira/secure/RapidBoard.jspa?
> >>> > rapidView=25&projectKey=INFRA&view=detail&selectedIssue=INFRA-14540
> >>> > >>
> >>> > >>
> >>> > >> On Sat, Jul 1, 2017 at 9:15 PM, shiwen hu <yajiedes...@gmail.com>
> >>> > wrote:
> >>> > >>
> >>> > >> > 1. The `mxnet directory` is a directory on the current CI
> server.
> >>> He
> >>> > >> > contains the necessary files, including library dependencies,
> data
> >>> > files
> >>> > >> > needed for testing, compiling scripts needed, and so on.You can
> >>> find
> >>> > Mu
> >>> > >> > Li
> >>> > >> > and ask him to copy from the current Ci
> >>> > >> > 2.Graphics Driver downlaod from
> >>> > >> > http://www.nvidia.com/Download/index.aspx
> >>> > >> > 3.Luanch is a small program. As long as you run it, you should
> be
> >>> able
> >>> > >> > to
> >>> > >> > see what to do at a glance
> >>> > >> >
> >>> > >> > 2017-07-02 10:03 GMT+08:00 Naveen Swamy <mnnav...@gmail.com>:
> >>> > >> >
> >>> > >> > > @yajiedesign
> >>> > >> > > we are building a new slave to be used in Apache Infra, the
> >>> > >> > > instructions
> >>> > >> > > here
> >>> > >> > > https://gist.github.com/yajiedesign/40b3809b51a1706d353e9129
> >>> 071b14
> >>> > >> > fb
> >>> > >> > > to setup a new slave from scratch is insufficient(probably
> >>> > outdated),
> >>> > >> > > we
> >>> > >> > > ran into quite a bit of problem setting up OpenBlas and
> >>> OpenCV(those
> >>> > >> > > instructions were missing) though we were able to get through
> >>> these
> >>> > >> > > problems we anticipate further problems.
> >>> > >> > > since we want to move our Infrastructure to Apache by the end
> of
> >>> > next
> >>> > >> > week
> >>> > >> > > we have paused the effort of setting up a Windows slave and
> >>> testing
> >>> > >> > > the
> >>> > >> > > Linux slaves that are already setup.
> >>> > >> > >
> >>> > >> > > Is it possible for you to update those instructions?
> meanwhile,
> >>> we
> >>> > >> > > have
> >>> > >> > > requested Mu Li to create an AMI out of the existing slave.
> >>> > >> > >
> >>> > >> > > Can I also request you to provide instructions on how to
> create
> >>> pip
> >>> > >> > package
> >>> > >> > > for Windows? currently, 0.10 version does not have windows pip
> >>> > >> > > package?
> >>> > >> > >
> >>> > >> > > Thanks, Naveen
> >>> > >> > >
> >>> > >> > >
> >>> > >> > >
> >>> > >> > > On Sat, Jul 1, 2017 at 12:14 AM, shiwen hu <
> >>> yajiedes...@gmail.com>
> >>> > >> > wrote:
> >>> > >> > >
> >>> > >> > > > what problem with windows ci?
> >>> > >> > > >
> >>> > >> > > > 2017-07-01 9:06 GMT+08:00 Ly Nguyen <nguyen...@gmail.com>:
> >>> > >> > > >
> >>> > >> > > > > This week's summary:
> >>> > >> > > > > 1. Wrote FAQ and publicized CI wiki
> >>> > >> > > > > 2. Plan was to complete migration by end of next week
> >>> > >> > > > >     1. Spent 1.5 days trying to set up Windows slave - was
> >>> not
> >>> > >> > > successful
> >>> > >> > > > > and would find it more productive to create an AMI from
> >>> > currently
> >>> > >> > > running
> >>> > >> > > > > slaves. Mu says a running Windows slave is not necessary
> for
> >>> > >> > migration
> >>> > >> > > > but
> >>> > >> > > > > that means we would be losing Windows coverage.
> >>> > >> > > > >     2. The goal for this week was to ensure that PRs,
> >>> merges,
> >>> > >> > nightlies
> >>> > >> > > > > against the fork trigger builds that pass. There were a
> lot
> >>> of
> >>> > >> > hurdles.
> >>> > >> > > > > Many items had to happen in sequence and depended on
> others’
> >>> > >> > schedules.
> >>> > >> > > > > Namely,
> >>> > >> > > > >         1. accepted invitation to be committer on Monday
> >>> morning
> >>> > >> > > > >         2. received Apache account Tuesday morning
> >>> > >> > > > >         3. got access to Jenkins & repo Wednesday morning
> >>> > >> > > > >         4. filed tickets for the Infra team to add
> webhooks
> >>> > which
> >>> > >> > > > > was
> >>> > >> > > > > addressed this morning https://issues.apache.org/
> >>> > >> > > jira/browse/INFRA-14472
> >>> > >> > > > >         5. Apache builds of all projects including MXNet’s
> >>> were
> >>> > >> > > > > not
> >>> > >> > > > > happening because of some infra issue so there was not
> much
> >>> > >> > > > > traction
> >>> > >> > > > today
> >>> > >> > > > > https://issues.apache.org/jira/browse/INFRA-14476
> >>> > >> > > > >     3. Filed a ticket for support on building docs website
> >>> > >> > > > > https://issues.apache.org/jira/browse/INFRA-14479
> >>> > >> > > > >     4. Filed a ticket to reconfigure donated linux slaves
> >>> > >> > > > > https://issues.apache.org/jira/browse/INFRA-14478
> >>> > >> > > > >
> >>> > >> > > > > On Tue, Jun 27, 2017 at 1:10 PM, Ly Nguyen <
> >>> nguyen...@gmail.com
> >>> > >
> >>> > >> > > wrote:
> >>> > >> > > > >
> >>> > >> > > > > > We are aiming to complete migration of MXNet to Apache
> by
> >>> July
> >>> > >> > > > > > 10.
> >>> > >> > > This
> >>> > >> > > > > > involves transferring the GitHub repo ownership to
> Apache.
> >>> > >> > > > > >
> >>> > >> > > > > > Migration is tracked at this project board:
> >>> > >> > > > > https://github.com/dmlc/mxnet/
> >>> > >> > > > > > projects/6
> >>> > >> > > > > > As a part of the migration, we also need to adopt the
> >>> Apache
> >>> > >> > release
> >>> > >> > > > > > process for our next release which is mid-July. This
> wiki
> >>> > >> > > > > > <https://cwiki.apache.org/confluence/display/MXNET/
> >>> > >> > > > > Continuous+Integration>
> >>> > >> > > > > > gives an overview of of how the process works. It also
> >>> lists
> >>> > >> > > > > > some
> >>> > >> > > > > > automation tasks that come after the completion of code
> >>> base
> >>> > >> > > migration
> >>> > >> > > > > and
> >>> > >> > > > > > the next release.
> >>> > >> > > > > >
> >>> > >> > > > > > FAQ:
> >>> > >> > > > > >
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >    Why are we migrating the code base to Apache
> ownership?
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >       This is one of the steps on graduating from Apache
> >>> > >> > incubation.
> >>> > >> > > > > >       2.
> >>> > >> > > > > >
> >>> > >> > > > > >    When is this happening?
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >       We are aiming for migration to complete by July
> >>> 10th.
> >>> > >> > > > > >       3.
> >>> > >> > > > > >
> >>> > >> > > > > >    Will my commits/contributions still exist after
> >>> migration?
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >       Yes. Existing commits will still appear under your
> >>> > >> > > > > > existing
> >>> > >> > > > github
> >>> > >> > > > > >       id, and stats will carry over. New commits will
> also
> >>> > >> > > > > > appear
> >>> > >> > > under
> >>> > >> > > > > your
> >>> > >> > > > > >       existing github id, so long as you’ve configured
> >>> your
> >>> > >> > > > ~/.gitconfig
> >>> > >> > > > > with an
> >>> > >> > > > > >       email address which you’ve linked in your github
> >>> > profile.
> >>> > >> > > > > >       2.
> >>> > >> > > > > >
> >>> > >> > > > > >       Committers will also need to link their Apache ids
> >>> with
> >>> > >> > > > > > the
> >>> > >> > > > github
> >>> > >> > > > > >       ids to gain write access, in which case, the above
> >>> > answer
> >>> > >> > still
> >>> > >> > > > > applies.
> >>> > >> > > > > >       See #9 on how to link your Apache id.
> >>> > >> > > > > >       4.
> >>> > >> > > > > >
> >>> > >> > > > > >    What will happen to my in flight pull requests?
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >       It will remain intact
> >>> > >> > > > > >       5.
> >>> > >> > > > > >
> >>> > >> > > > > >    Will I still be a member/owner after migration?
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >       Current list of Apache MXNet committers:
> >>> > >> > > > https://wiki.apache.org/
> >>> > >> > > > > >       incubator/MXNetProposal
> >>> > >> > > > > >       2.
> >>> > >> > > > > >
> >>> > >> > > > > >       If you’re not an Apache committer, you lose
> >>> > >> > > membership/ownership
> >>> > >> > > > > >       rights
> >>> > >> > > > > >       3.
> >>> > >> > > > > >
> >>> > >> > > > > >       Apache Infra are the only people with Owner/Admin
> >>> > >> > > > > > permissions
> >>> > >> > > > there
> >>> > >> > > > > >       4.
> >>> > >> > > > > >
> >>> > >> > > > > >       Apache committers will have write access
> >>> > >> > > > > >       6.
> >>> > >> > > > > >
> >>> > >> > > > > >    What other things will be transferred with the
> >>> repository?
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >       Wiki, stars, watchers, webhooks, services, deploy
> >>> keys
> >>> > >> > > > > >       7.
> >>> > >> > > > > >
> >>> > >> > > > > >    What will my fork be associated with after migration?
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >       It will remain associated with the transferred
> >>> > repository
> >>> > >> > > > > >       8.
> >>> > >> > > > > >
> >>> > >> > > > > >    Will I have to change all references to
> >>> > >> > > > http://github.com/dmlc/mxnet
> >>> > >> > > > > ?
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >       All links to http://github.com/dmlc/mxnet will
> >>> > >> > > > > > automatically
> >>> > >> > > be
> >>> > >> > > > > >       redirected to new location when issuing `git
> clone`,
> >>> > `git
> >>> > >> > > fetch`,
> >>> > >> > > > > `git
> >>> > >> > > > > >       push`, etc, (as long as we don’t create another
> >>> “mxnet”
> >>> > >> > > > repository
> >>> > >> > > > > under
> >>> > >> > > > > >       DMLC). However, to avoid confusion, you can change
> >>> the
> >>> > >> > > > > > links
> >>> > >> > > > where
> >>> > >> > > > > >       possible, and change remote: `git remote set-url
> >>> origin
> >>> > >> > > new_url`
> >>> > >> > > > > >       9.
> >>> > >> > > > > >
> >>> > >> > > > > >    How do I gain write access to the repo?
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >       First, you need to be a committer. Then use
> >>> > >> > > > > >       https://gitbox.apache.org/setup/
> >>> > >> > > > > > <https://gitbox.apache.org/
> >>> > >> > > > setup/
> >>> > >> > > > > >
> >>> > >> > > > > >       to associate the Apache and GitHub accounts. Note
> >>> that
> >>> > all
> >>> > >> > > > > committers will
> >>> > >> > > > > >       need to enable 2-factor authentication on GitHub
> >>> > >> > > > > >       10.
> >>> > >> > > > > >
> >>> > >> > > > > >    Are we also moving mxnet CI? If so, what is the new
> >>> > location?
> >>> > >> > Will
> >>> > >> > > > > >    nightly tests continue to run? How can I add new
> tests?
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >       We will rely on Apache’s build server to run our
> >>> builds.
> >>> > >> > > > > >       2.
> >>> > >> > > > > >
> >>> > >> > > > > >       It will first only run unit tests for PRs and
> >>> merges.
> >>> > >> > > > > > Tests
> >>> > >> > can
> >>> > >> > > > be
> >>> > >> > > > > >       added following the structure setup in
> >>> > >> > > > > >       https://github.com/dmlc/mxnet
> >>> /blob/master/Jenkinsfile
> >>> > >> > > > > >       <https://github.com/dmlc/mxne
> >>> t/blob/master/Jenkinsfile>
> >>> > .
> >>> > >> > > > > >       3.
> >>> > >> > > > > >
> >>> > >> > > > > >       Nightly tests are currently running at
> >>> > >> > > > http://jenkins-master-elb-
> >>> > >> > > > > >       1979848568.us-east-1.elb.amazonaws.com/
> >>> > >> > > > > >       <http://jenkins-master-elb-
> 1979848568.us-east-1.elb
> >>> .
> >>> > >> > > > amazonaws.com/
> >>> > >> > > > > >
> >>> > >> > > > > >       and will gradually run in Apache’s build server
> too.
> >>> > >> > > > > > There,
> >>> > >> > we
> >>> > >> > > > > will provide
> >>> > >> > > > > >       artifacts such as pip wheels and source packages
> >>> for the
> >>> > >> > > > community
> >>> > >> > > > > to test.
> >>> > >> > > > > >       1.
> >>> > >> > > > > >
> >>> > >> > > > > >          Automated releases will happen on
> >>> > >> > > http://jenkins-master-elb-
> >>> > >> > > > > >          1979848568.us-east-1.elb.amazonaws.com/
> >>> > >> > > > > >          <http://jenkins-master-elb-197
> >>> 9848568.us-east-1.elb.
> >>> > >> > > > > amazonaws.com/>
> >>> > >> > > > > >          as Apache’s build doesn’t support key storage.
> >>> > >> > > > > >          11.
> >>> > >> > > > > >
> >>> > >> > > > > >    Is mxnet.io moving too?
> >>> > >> > > > > >    1.
> >>> > >> > > > > >
> >>> > >> > > > > >       For some time we will have both mxnet.apache.org
> >>> and
> >>> > >> > mxnet.io
> >>> > >> > > > > >       hosting the docs. When we are confident that
> >>> > >> > mxnet.apache.org
> >>> > >> > > is
> >>> > >> > > > > >       stable, we will redirect mxnet.io to there.
> >>> > >> > > > > >
> >>> > >> > > > > >
> >>> > >> > > > > > Link on GitHub repo transfers: https://help.github.com/
> >>> > >> > > > > > articles/about-repository-transfers/
> >>> > >> > > > > >
> >>> > >> > > > > > Feel free to ask any other questions.
> >>> > >> > > > > >
> >>> > >> > > > > >
> >>> > >> > > > > >
> >>> > >> > > > > > On Wed, Jun 7, 2017 at 12:53 PM, Ly Nguyen <
> >>> > nguyen...@gmail.com>
> >>> > >> > > > wrote:
> >>> > >> > > > > >
> >>> > >> > > > > >> I’ve documented the detailed steps below on the process
> >>> of
> >>> > >> > migrating
> >>> > >> > > > > >> MXNet -> Apache for open feedback and discussion.
> >>> > >> > > > > >>
> >>> > >> > > > > >> Essentially Amazon will be providing the GPU build
> >>> slaves to
> >>> > be
> >>> > >> > > hooked
> >>> > >> > > > > >> into Apache’s Jenkins build Master. We’ll first make
> sure
> >>> > that
> >>> > >> > > Apache
> >>> > >> > > > > can
> >>> > >> > > > > >> build a fork of MXNet, before officially transferring
> >>> > ownership
> >>> > >> > > > > >> of
> >>> > >> > > the
> >>> > >> > > > > >> MXNet repo.
> >>> > >> > > > > >>
> >>> > >> > > > > >> Steps to migration:
> >>> > >> > > > > >> 1.      Provide Apache with Linux slaves & slave tags
> >>> > >> > > > > >> a.      Provide Apache with slave configuration (tags,
> >>> remote
> >>> > >> > > > > >> root
> >>> > >> > > > dir,
> >>> > >> > > > > >> etc.)
> >>> > >> > > > > >> b.      Spin up 6 slaves
> >>> > >> > > > > >> c.       Launch connection via JNLP
> >>> > >> > > > > >> 2.      Apache forks MXNet repo and makes sure builds
> are
> >>> > >> > successful
> >>> > >> > > > on
> >>> > >> > > > > >> their build set up
> >>> > >> > > > > >> a.      Ask Apache to give me committer rights
> >>> > >> > > > > >> b.      I remove the Windows jobs until a later time
> >>> > >> > > > > >> c.       Apache sets up Jenkins jobs and Github
> webhooks
> >>> > >> > > > > >>
> >>> > >> > > > > >> i.
> >>> > >> > > > > >> Build every commit and origin/fork PR’s without merge
> >>> (main
> >>> > >> > > > Jenkinsfile)
> >>> > >> > > > > >>
> >>> >  ii.
> >>> > >> > > > > >> Nightly job (nightly Jenkins file, will start with a
> >>> dummy
> >>> > one
> >>> > >> > > > > >> and
> >>> > >> > > add
> >>> > >> > > > > more
> >>> > >> > > > > >> configurations later)
> >>> > >> > > > > >> d.      If Windows slave setup is available, provide it
> >>> to
> >>> > >> > > > > >> Apache
> >>> > >> > > and
> >>> > >> > > > > >> enable the jobs again
> >>> > >> > > > > >> 3.      Transfer the repo and point the build set up
> >>> there
> >>> > >> > > > > >> 4.      Apache deploys the docs to their website
> >>> > >> > > > > >>
> >>> > >> > > > > >> Open security questions:
> >>> > >> > > > > >> 1.      How can we ensure that our slaves are not used
> by
> >>> > other
> >>> > >> > > > > projects?
> >>> > >> > > > > >> a.      It’s not, it’s a social contract.
> >>> > >> > > > > >> 2.      To protect the slave hosts, would running
> Jenkins
> >>> > slave
> >>> > >> > > > inside a
> >>> > >> > > > > >> Docker container be a solution, or is there a
> recommended
> >>> > best
> >>> > >> > > > practice?
> >>> > >> > > > > >> a.      Run slave behind a NAT gateway and launch via
> >>> JNLP
> >>> > >> > > > > >> 3.      Does Apache place SSH key inside the build host
> >>> for
> >>> > >> > > > > >> Docs
> >>> > >> > > > > >> deployment to the website? Are there security concerns
> >>> there?
> >>> > >> > > > > >> a.      The only slaves that are allowed to deploy docs
> >>> are
> >>> > >> > > > > >> ASF-controlled. Just provide the build command.
> >>> > >> > > > > >>
> >>> > >> > > > > >
> >>> > >> > > > > >
> >>> > >> > > > >
> >>> > >> > > >
> >>> > >> > >
> >>> > >> >
> >>> > >
> >>> > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > >
> >>> > >
> >>> > > Dominic Divakaruni
> >>> > > 206.475.9200 Cell
> >>> >
> >>>
> >>
> >>
> >
>

Reply via email to