Re: MXNet -> Apache Migration proposal

[Ly Nguyen]

Also, DON'T mark the branch as "*Require status checks to pass before
merging*" since builds may be unstable at first. The existing configuration
should be preserved after the transfer but we just have to validate that.

https://help.github.com/articles/enabling-required-status-checks/

On Mon, Jul 10, 2017 at 12:36 PM, Ly Nguyen <nguyen...@gmail.com> wrote:

> From Mu:
> Please preserve the following hooks after transferring:
> -http://mxnet.io:8080/github-webhook/  (docs deployment)
> -https://ci.appveyor.com/api/github/webhook
> all other hooks can be removed
>
> On Mon, Jul 10, 2017 at 12:00 PM, Ly Nguyen <nguyen...@gmail.com> wrote:
>
>> @Suneel I don't think so. We'll just be transferring the organization of
>> the GitHub repo from DMLC -> Apache.
>>
>>
>> Pono has agreed to help with those steps from 2-3:30pm PDT today. Pending
>> Mu adding Pono as an owner.
>>
>> On Sat, Jul 8, 2017 at 11:33 AM, Suneel Marthi <smar...@apache.org>
>> wrote:
>>
>>> I suppose we'll be migrating to apache Gitbox, correct ??
>>>
>>> On Sat, Jul 8, 2017 at 1:48 PM, Li, Mu <m...@amazon.com> wrote:
>>>
>>> > Can we schedule a time on monday to finish the transfer together?
>>> > ____________
>>> > From: Daniel Pono Takamori <p...@apache.org>
>>> > Sent: Friday, July 7, 2017 5:22 PM
>>> > To: dev@mxnet.incubator.apache.org
>>> > Subject: Re: MXNet -> Apache Migration proposal
>>> >
>>> > This looks great Ly.  On Monday we can transfer the main repo over and
>>> > then you can change the builds to point to the official repo.  I'll
>>> > make a clone of the website next week so we can do a similar thing to
>>> > test building the website and then we can publish it to
>>> > mxnet.incubator.apache.org.  Once we have that worked out we can move
>>> > forward with transferring the domain to our control.
>>> >
>>> > Have a great weekend everyone!
>>> >
>>> > On Fri, Jul 7, 2017 at 6:07 PM, Dominic Divakaruni
>>> > <dominic.divakar...@gmail.com> wrote:
>>> > > great stuff!! glad to see this getting close!
>>> > >
>>> > > On Fri, Jul 7, 2017 at 3:47 PM, Ly Nguyen <nguyen...@gmail.com>
>>> wrote:
>>> > >>
>>> > >> We have successfully validated that merges and pull requests
>>> against an
>>> > >> Apache fork of MXNet runs successfully on builds.apache.org:
>>> > >>
>>> > >> https://builds.apache.org/blue/organizations/jenkins/
>>> > incubator-mxnet-master2/detail/master/13/pipeline
>>> > >>
>>> > >> https://builds.apache.org/blue/organizations/jenkins/
>>> > incubator-mxnet-master2/detail/PR-3/1/pipeline
>>> > >>
>>> > >> We have also added a dummy nightly run to be populated with builds
>>> and
>>> > >> test
>>> > >> cases after migration, as discussed. We can now move forward with
>>> the
>>> > >> migration to Apache and I recommend the following steps:
>>> > >> - [ ] Add Pono as owner
>>> > >> - [ ] Pono adds Apache git hooks to MXNet repo
>>> > >> - [ ] Change source control of Apache Jenkins jobs to point to MXNet
>>> > repo,
>>> > >> verify a run is successful
>>> > >> - [ ] Change MXNet org to Apache, verify a run is successful,
>>> mxnet.io
>>> > >> still building
>>> > >> - [ ] Start docs build to mxnet.apache.org
>>> > >> Note that one kink to iron out is that PR build statuses aren’t
>>> being
>>> > >> updated. Here’s a ticket to follow:
>>> > >>
>>> > >> https://issues.apache.org/jira/secure/RapidBoard.jspa?
>>> > rapidView=25=INFRA=detail=INFRA-14540
>>> > >>
>>> > >>
>>> > >> On Sat, Jul 1, 2017 at 9:15 PM, shiwen hu <yajiedes...@gmail.com>
>>> > wrote:
>>> > >>
>>> > >> > 1. The `mxnet directory` is a directory on the current CI server.
>>> He
>>> > >> > contains the necessary files, including library dependencies, data
>>> > files
>>> > >> > needed for testing, compiling scripts needed, and so on.You can
>>> find
>>> > Mu
>&

Re: MXNet -> Apache Migration proposal

[Daniel Pono Takamori]

This looks great Ly.  On Monday we can transfer the main repo over and
then you can change the builds to point to the official repo.  I'll
make a clone of the website next week so we can do a similar thing to
test building the website and then we can publish it to
mxnet.incubator.apache.org.  Once we have that worked out we can move
forward with transferring the domain to our control.

Have a great weekend everyone!

On Fri, Jul 7, 2017 at 6:07 PM, Dominic Divakaruni
 wrote:
> great stuff!! glad to see this getting close!
>
> On Fri, Jul 7, 2017 at 3:47 PM, Ly Nguyen  wrote:
>>
>> We have successfully validated that merges and pull requests against an
>> Apache fork of MXNet runs successfully on builds.apache.org:
>>
>> https://builds.apache.org/blue/organizations/jenkins/incubator-mxnet-master2/detail/master/13/pipeline
>>
>> https://builds.apache.org/blue/organizations/jenkins/incubator-mxnet-master2/detail/PR-3/1/pipeline
>>
>> We have also added a dummy nightly run to be populated with builds and
>> test
>> cases after migration, as discussed. We can now move forward with the
>> migration to Apache and I recommend the following steps:
>> - [ ] Add Pono as owner
>> - [ ] Pono adds Apache git hooks to MXNet repo
>> - [ ] Change source control of Apache Jenkins jobs to point to MXNet repo,
>> verify a run is successful
>> - [ ] Change MXNet org to Apache, verify a run is successful, mxnet.io
>> still building
>> - [ ] Start docs build to mxnet.apache.org
>> Note that one kink to iron out is that PR build statuses aren’t being
>> updated. Here’s a ticket to follow:
>>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=25=INFRA=detail=INFRA-14540
>>
>>
>> On Sat, Jul 1, 2017 at 9:15 PM, shiwen hu  wrote:
>>
>> > 1. The `mxnet directory` is a directory on the current CI server. He
>> > contains the necessary files, including library dependencies, data files
>> > needed for testing, compiling scripts needed, and so on.You can find Mu
>> > Li
>> > and ask him to copy from the current Ci
>> > 2.Graphics Driver downlaod from
>> > http://www.nvidia.com/Download/index.aspx
>> > 3.Luanch is a small program. As long as you run it, you should be able
>> > to
>> > see what to do at a glance
>> >
>> > 2017-07-02 10:03 GMT+08:00 Naveen Swamy :
>> >
>> > > @yajiedesign
>> > > we are building a new slave to be used in Apache Infra, the
>> > > instructions
>> > > here
>> > > https://gist.github.com/yajiedesign/40b3809b51a1706d353e9129071b14
>> > fb
>> > > to setup a new slave from scratch is insufficient(probably outdated),
>> > > we
>> > > ran into quite a bit of problem setting up OpenBlas and OpenCV(those
>> > > instructions were missing) though we were able to get through these
>> > > problems we anticipate further problems.
>> > > since we want to move our Infrastructure to Apache by the end of next
>> > week
>> > > we have paused the effort of setting up a Windows slave and testing
>> > > the
>> > > Linux slaves that are already setup.
>> > >
>> > > Is it possible for you to update those instructions? meanwhile, we
>> > > have
>> > > requested Mu Li to create an AMI out of the existing slave.
>> > >
>> > > Can I also request you to provide instructions on how to create pip
>> > package
>> > > for Windows? currently, 0.10 version does not have windows pip
>> > > package?
>> > >
>> > > Thanks, Naveen
>> > >
>> > >
>> > >
>> > > On Sat, Jul 1, 2017 at 12:14 AM, shiwen hu 
>> > wrote:
>> > >
>> > > > what problem with windows ci?
>> > > >
>> > > > 2017-07-01 9:06 GMT+08:00 Ly Nguyen :
>> > > >
>> > > > > This week's summary:
>> > > > > 1. Wrote FAQ and publicized CI wiki
>> > > > > 2. Plan was to complete migration by end of next week
>> > > > > 1. Spent 1.5 days trying to set up Windows slave - was not
>> > > successful
>> > > > > and would find it more productive to create an AMI from currently
>> > > running
>> > > > > slaves. Mu says a running Windows slave is not necessary for
>> > migration
>> > > > but
>> > > > > that means we would be losing Windows coverage.
>> > > > > 2. The goal for this week was to ensure that PRs, merges,
>> > nightlies
>> > > > > against the fork trigger builds that pass. There were a lot of
>> > hurdles.
>> > > > > Many items had to happen in sequence and depended on others’
>> > schedules.
>> > > > > Namely,
>> > > > > 1. accepted invitation to be committer on Monday morning
>> > > > > 2. received Apache account Tuesday morning
>> > > > > 3. got access to Jenkins & repo Wednesday morning
>> > > > > 4. filed tickets for the Infra team to add webhooks which
>> > > > > was
>> > > > > addressed this morning https://issues.apache.org/
>> > > jira/browse/INFRA-14472
>> > > > > 5. Apache builds of all projects including MXNet’s were
>> > > > > not
>> > > > > happening because of some infra issue 

Re: MXNet -> Apache Migration proposal

[Naveen Swamy]

great job! Ly. Thanks for all your help pono@.

On Fri, Jul 7, 2017 at 5:07 PM, Dominic Divakaruni <
dominic.divakar...@gmail.com> wrote:

> great stuff!! glad to see this getting close!
>
> On Fri, Jul 7, 2017 at 3:47 PM, Ly Nguyen  wrote:
>
> > We have successfully validated that merges and pull requests against an
> > Apache fork of MXNet runs successfully on builds.apache.org:
> > https://builds.apache.org/blue/organizations/jenkins/
> > incubator-mxnet-master2/detail/master/13/pipeline
> > https://builds.apache.org/blue/organizations/jenkins/
> > incubator-mxnet-master2/detail/PR-3/1/pipeline
> >
> > We have also added a dummy nightly run to be populated with builds and
> test
> > cases after migration, as discussed. We can now move forward with the
> > migration to Apache and I recommend the following steps:
> > - [ ] Add Pono as owner
> > - [ ] Pono adds Apache git hooks to MXNet repo
> > - [ ] Change source control of Apache Jenkins jobs to point to MXNet
> repo,
> > verify a run is successful
> > - [ ] Change MXNet org to Apache, verify a run is successful, mxnet.io
> > still building
> > - [ ] Start docs build to mxnet.apache.org
> > Note that one kink to iron out is that PR build statuses aren’t being
> > updated. Here’s a ticket to follow:
> > https://issues.apache.org/jira/secure/RapidBoard.jspa?
> > rapidView=25=INFRA=detail=INFRA-14540
> >
> >
> > On Sat, Jul 1, 2017 at 9:15 PM, shiwen hu  wrote:
> >
> > > 1. The `mxnet directory` is a directory on the current CI server. He
> > > contains the necessary files, including library dependencies, data
> files
> > > needed for testing, compiling scripts needed, and so on.You can find Mu
> > Li
> > > and ask him to copy from the current Ci
> > > 2.Graphics Driver downlaod from http://www.nvidia.com/
> > Download/index.aspx
> > > 3.Luanch is a small program. As long as you run it, you should be able
> to
> > > see what to do at a glance
> > >
> > > 2017-07-02 10:03 GMT+08:00 Naveen Swamy :
> > >
> > > > @yajiedesign
> > > > we are building a new slave to be used in Apache Infra, the
> > instructions
> > > > here https://gist.github.com/yajiedesign/
> > 40b3809b51a1706d353e9129071b14
> > > fb
> > > > to setup a new slave from scratch is insufficient(probably outdated),
> > we
> > > > ran into quite a bit of problem setting up OpenBlas and OpenCV(those
> > > > instructions were missing) though we were able to get through these
> > > > problems we anticipate further problems.
> > > > since we want to move our Infrastructure to Apache by the end of next
> > > week
> > > > we have paused the effort of setting up a Windows slave and testing
> the
> > > > Linux slaves that are already setup.
> > > >
> > > > Is it possible for you to update those instructions? meanwhile, we
> have
> > > > requested Mu Li to create an AMI out of the existing slave.
> > > >
> > > > Can I also request you to provide instructions on how to create pip
> > > package
> > > > for Windows? currently, 0.10 version does not have windows pip
> package?
> > > >
> > > > Thanks, Naveen
> > > >
> > > >
> > > >
> > > > On Sat, Jul 1, 2017 at 12:14 AM, shiwen hu 
> > > wrote:
> > > >
> > > > > what problem with windows ci?
> > > > >
> > > > > 2017-07-01 9:06 GMT+08:00 Ly Nguyen :
> > > > >
> > > > > > This week's summary:
> > > > > > 1. Wrote FAQ and publicized CI wiki
> > > > > > 2. Plan was to complete migration by end of next week
> > > > > > 1. Spent 1.5 days trying to set up Windows slave - was not
> > > > successful
> > > > > > and would find it more productive to create an AMI from currently
> > > > running
> > > > > > slaves. Mu says a running Windows slave is not necessary for
> > > migration
> > > > > but
> > > > > > that means we would be losing Windows coverage.
> > > > > > 2. The goal for this week was to ensure that PRs, merges,
> > > nightlies
> > > > > > against the fork trigger builds that pass. There were a lot of
> > > hurdles.
> > > > > > Many items had to happen in sequence and depended on others’
> > > schedules.
> > > > > > Namely,
> > > > > > 1. accepted invitation to be committer on Monday morning
> > > > > > 2. received Apache account Tuesday morning
> > > > > > 3. got access to Jenkins & repo Wednesday morning
> > > > > > 4. filed tickets for the Infra team to add webhooks which
> > was
> > > > > > addressed this morning https://issues.apache.org/
> > > > jira/browse/INFRA-14472
> > > > > > 5. Apache builds of all projects including MXNet’s were
> not
> > > > > > happening because of some infra issue so there was not much
> > traction
> > > > > today
> > > > > > https://issues.apache.org/jira/browse/INFRA-14476
> > > > > > 3. Filed a ticket for support on building docs website
> > > > > > https://issues.apache.org/jira/browse/INFRA-14479
> > > > > > 4. Filed a ticket to reconfigure donated linux slaves
> 

Re: MXNet -> Apache Migration proposal

[Dominic Divakaruni]

great stuff!! glad to see this getting close!

On Fri, Jul 7, 2017 at 3:47 PM, Ly Nguyen  wrote:

> We have successfully validated that merges and pull requests against an
> Apache fork of MXNet runs successfully on builds.apache.org:
> https://builds.apache.org/blue/organizations/jenkins/
> incubator-mxnet-master2/detail/master/13/pipeline
> https://builds.apache.org/blue/organizations/jenkins/
> incubator-mxnet-master2/detail/PR-3/1/pipeline
>
> We have also added a dummy nightly run to be populated with builds and test
> cases after migration, as discussed. We can now move forward with the
> migration to Apache and I recommend the following steps:
> - [ ] Add Pono as owner
> - [ ] Pono adds Apache git hooks to MXNet repo
> - [ ] Change source control of Apache Jenkins jobs to point to MXNet repo,
> verify a run is successful
> - [ ] Change MXNet org to Apache, verify a run is successful, mxnet.io
> still building
> - [ ] Start docs build to mxnet.apache.org
> Note that one kink to iron out is that PR build statuses aren’t being
> updated. Here’s a ticket to follow:
> https://issues.apache.org/jira/secure/RapidBoard.jspa?
> rapidView=25=INFRA=detail=INFRA-14540
>
>
> On Sat, Jul 1, 2017 at 9:15 PM, shiwen hu  wrote:
>
> > 1. The `mxnet directory` is a directory on the current CI server. He
> > contains the necessary files, including library dependencies, data files
> > needed for testing, compiling scripts needed, and so on.You can find Mu
> Li
> > and ask him to copy from the current Ci
> > 2.Graphics Driver downlaod from http://www.nvidia.com/
> Download/index.aspx
> > 3.Luanch is a small program. As long as you run it, you should be able to
> > see what to do at a glance
> >
> > 2017-07-02 10:03 GMT+08:00 Naveen Swamy :
> >
> > > @yajiedesign
> > > we are building a new slave to be used in Apache Infra, the
> instructions
> > > here https://gist.github.com/yajiedesign/
> 40b3809b51a1706d353e9129071b14
> > fb
> > > to setup a new slave from scratch is insufficient(probably outdated),
> we
> > > ran into quite a bit of problem setting up OpenBlas and OpenCV(those
> > > instructions were missing) though we were able to get through these
> > > problems we anticipate further problems.
> > > since we want to move our Infrastructure to Apache by the end of next
> > week
> > > we have paused the effort of setting up a Windows slave and testing the
> > > Linux slaves that are already setup.
> > >
> > > Is it possible for you to update those instructions? meanwhile, we have
> > > requested Mu Li to create an AMI out of the existing slave.
> > >
> > > Can I also request you to provide instructions on how to create pip
> > package
> > > for Windows? currently, 0.10 version does not have windows pip package?
> > >
> > > Thanks, Naveen
> > >
> > >
> > >
> > > On Sat, Jul 1, 2017 at 12:14 AM, shiwen hu 
> > wrote:
> > >
> > > > what problem with windows ci?
> > > >
> > > > 2017-07-01 9:06 GMT+08:00 Ly Nguyen :
> > > >
> > > > > This week's summary:
> > > > > 1. Wrote FAQ and publicized CI wiki
> > > > > 2. Plan was to complete migration by end of next week
> > > > > 1. Spent 1.5 days trying to set up Windows slave - was not
> > > successful
> > > > > and would find it more productive to create an AMI from currently
> > > running
> > > > > slaves. Mu says a running Windows slave is not necessary for
> > migration
> > > > but
> > > > > that means we would be losing Windows coverage.
> > > > > 2. The goal for this week was to ensure that PRs, merges,
> > nightlies
> > > > > against the fork trigger builds that pass. There were a lot of
> > hurdles.
> > > > > Many items had to happen in sequence and depended on others’
> > schedules.
> > > > > Namely,
> > > > > 1. accepted invitation to be committer on Monday morning
> > > > > 2. received Apache account Tuesday morning
> > > > > 3. got access to Jenkins & repo Wednesday morning
> > > > > 4. filed tickets for the Infra team to add webhooks which
> was
> > > > > addressed this morning https://issues.apache.org/
> > > jira/browse/INFRA-14472
> > > > > 5. Apache builds of all projects including MXNet’s were not
> > > > > happening because of some infra issue so there was not much
> traction
> > > > today
> > > > > https://issues.apache.org/jira/browse/INFRA-14476
> > > > > 3. Filed a ticket for support on building docs website
> > > > > https://issues.apache.org/jira/browse/INFRA-14479
> > > > > 4. Filed a ticket to reconfigure donated linux slaves
> > > > > https://issues.apache.org/jira/browse/INFRA-14478
> > > > >
> > > > > On Tue, Jun 27, 2017 at 1:10 PM, Ly Nguyen 
> > > wrote:
> > > > >
> > > > > > We are aiming to complete migration of MXNet to Apache by July
> 10.
> > > This
> > > > > > involves transferring the GitHub repo ownership to Apache.
> > > > > >
> > > > > > Migration is