Re: [Proposal] Stabilizing Apache MXNet CI build system

[sandeep krishnamurthy]

Good work Meghna and thanks to community members for participating in the
discussion and providing valuable inputs.
Yes please share the document again and ask for vote and more broader
inputs.

On Wed, Nov 8, 2017 at 2:43 PM, Chris Olivier  wrote:

> +1
>
> On Wed, Nov 8, 2017 at 2:40 PM Meghna Baijal 
> wrote:
>
> > Thanks for the active discussion on the document for the new CI for
> MXNet.
> > Now that many of you have reviewed it, do you think I should start a vote
> > on which framework the community wants to move forward with ?
> >
> > Thanks,
> > Meghna
> >
> > On Mon, Nov 6, 2017 at 6:59 PM, Chris Olivier 
> > wrote:
> >
> > > After a decision is reached, i am willing to add tasks to Apache MXNet
> > JIRA
> > >
> > > On Mon, Nov 6, 2017 at 6:15 AM, Pedro Larroy <
> > pedro.larroy.li...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Thanks for setting up the document guys, looks like a solid basis to
> > > > start to work on!
> > > >
> > > > Marco, Kellen and I have already added some comments.
> > > >
> > > > Pedro
> > > >
> > > >
> > > > On Sun, Nov 5, 2017 at 3:43 AM, Meghna Baijal
> > > >  wrote:
> > > > > Kellen, Thank you for your comments in the doc.
> > > > > Sure Steffen, I will continue to merge everyone’s comments into the
> > doc
> > > > and
> > > > > work with Pedro to finalize it.
> > > > > And then we can vote on the options.
> > > > >
> > > > > Thanks,
> > > > > Meghna Baijal
> > > > >
> > > > >
> > > > > On Sat, Nov 4, 2017 at 6:34 AM, Steffen Rochel <
> > > steffenroc...@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Sandeep and Meghna have been working in background collecting
> input
> > > and
> > > > >> preparing a doc. I suggest to drive discussion forward and would
> > like
> > > to
> > > > >> ask everybody to contribute to
> > > > >> https://docs.google.com/document/d/17PEasQ2VWrXi2Cf7IGZSWGZMawxDk
> > > > >> dlavUDASzUmLjk/edit?usp=sharing
> > > > >>
> > > > >> Lets converge on requirements and architecture, so we can move
> > forward
> > > > with
> > > > >> implementation.
> > > > >>
> > > > >> I would like to suggest for Pedro  and Meghna to lead the
> discussion
> > > and
> > > > >> help to resolve suggestions.
> > > > >>
> > > > >> I assume we need a vote once we are converged on a good draft to
> > call
> > > > it a
> > > > >> plan and move forward with implementation. As we all are unhappy
> > with
> > > > the
> > > > >> current CI situation I would also suggest a phased approach, so we
> > can
> > > > get
> > > > >> back to reliable and efficient basic CI quickly and add advanced
> > > > >> capabilities over time.
> > > > >>
> > > > >> Steffen
> > > > >>
> > > > >> On Wed, Nov 1, 2017 at 1:14 PM kellen sunderland <
> > > > >> kellen.sunderl...@gmail.com> wrote:
> > > > >>
> > > > >> > Hey Henri, I think that's what a few of us are advocating.
> > Running
> > > a
> > > > set
> > > > >> > of quick tests as part of the PR process, and then a more
> detailed
> > > > >> > regression test suite periodically (say every 4 hours). This
> fits
> > > > nicely
> > > > >> > into a tagging or 2 branch development system.  Commits will be
> > > tagged
> > > > >> (or
> > > > >> > merged into a stable branch) as soon as they pass the detailed
> > > > regression
> > > > >> > testing.
> > > > >> >
> > > > >> > On Wed, Nov 1, 2017 at 9:07 PM, Hen  wrote:
> > > > >> >
> > > > >> > > Random question - can the CI be split such that the Apache CI
> is
> > > > doing
> > > > >> a
> > > > >> > > basic set of checks on that hardware, and is hooked to a PR,
> > while
> > > > >> there
> > > > >> > is
> > > > >> > > a larger "Is trunk good for release?" test that is running
> > > > periodically
> > > > >> > > rather than on every PR?
> > > > >> > >
> > > > >> > > ie: do we need each PR to be run on varied hardware, or can we
> > > have
> > > > >> this
> > > > >> > > two tier approach?
> > > > >> > >
> > > > >> > > Hen
> > > > >> > >
> > > > >> > > On Fri, Oct 20, 2017 at 1:01 PM, sandeep krishnamurthy <
> > > > >> > > sandeep.krishn...@gmail.com> wrote:
> > > > >> > >
> > > > >> > > > Hello all,
> > > > >> > > >
> > > > >> > > > I am hereby opening up a discussion thread on how we can
> > > stabilize
> > > > >> > Apache
> > > > >> > > > MXNet CI build system.
> > > > >> > > >
> > > > >> > > > Problems:
> > > > >> > > >
> > > > >> > > > 
> > > > >> > > >
> > > > >> > > > Recently, we have seen following issues with Apache MXNet CI
> > > build
> > > > >> > > systems:
> > > > >> > > >
> > > > >> > > >1. Apache Jenkins master is overloaded and we see issues
> > > like -
> > > > >> > unable
> > > > >> > > >to trigger builds, difficult to load and view the blue
> > ocean
> > > > and
> > > > >> > other
> > > > >> > > >Jenkins build status page.
> > > > >> > > >2. We are generating too many request/interaction on
> Apache
> > > > Infra
> > > > >> > > team.
> > > 

[BUILD FAILED] Branch master build 606

[Apache Jenkins Server]

Build for MXNet branch master has broken. Please view the build at 
https://builds.apache.org/job/incubator-mxnet/job/master/606/

Re: [Proposal] Stabilizing Apache MXNet CI build system

[Chris Olivier]

+1

On Wed, Nov 8, 2017 at 2:40 PM Meghna Baijal 
wrote:

> Thanks for the active discussion on the document for the new CI for MXNet.
> Now that many of you have reviewed it, do you think I should start a vote
> on which framework the community wants to move forward with ?
>
> Thanks,
> Meghna
>
> On Mon, Nov 6, 2017 at 6:59 PM, Chris Olivier 
> wrote:
>
> > After a decision is reached, i am willing to add tasks to Apache MXNet
> JIRA
> >
> > On Mon, Nov 6, 2017 at 6:15 AM, Pedro Larroy <
> pedro.larroy.li...@gmail.com
> > >
> > wrote:
> >
> > > Thanks for setting up the document guys, looks like a solid basis to
> > > start to work on!
> > >
> > > Marco, Kellen and I have already added some comments.
> > >
> > > Pedro
> > >
> > >
> > > On Sun, Nov 5, 2017 at 3:43 AM, Meghna Baijal
> > >  wrote:
> > > > Kellen, Thank you for your comments in the doc.
> > > > Sure Steffen, I will continue to merge everyone’s comments into the
> doc
> > > and
> > > > work with Pedro to finalize it.
> > > > And then we can vote on the options.
> > > >
> > > > Thanks,
> > > > Meghna Baijal
> > > >
> > > >
> > > > On Sat, Nov 4, 2017 at 6:34 AM, Steffen Rochel <
> > steffenroc...@gmail.com>
> > > > wrote:
> > > >
> > > >> Sandeep and Meghna have been working in background collecting input
> > and
> > > >> preparing a doc. I suggest to drive discussion forward and would
> like
> > to
> > > >> ask everybody to contribute to
> > > >> https://docs.google.com/document/d/17PEasQ2VWrXi2Cf7IGZSWGZMawxDk
> > > >> dlavUDASzUmLjk/edit?usp=sharing
> > > >>
> > > >> Lets converge on requirements and architecture, so we can move
> forward
> > > with
> > > >> implementation.
> > > >>
> > > >> I would like to suggest for Pedro  and Meghna to lead the discussion
> > and
> > > >> help to resolve suggestions.
> > > >>
> > > >> I assume we need a vote once we are converged on a good draft to
> call
> > > it a
> > > >> plan and move forward with implementation. As we all are unhappy
> with
> > > the
> > > >> current CI situation I would also suggest a phased approach, so we
> can
> > > get
> > > >> back to reliable and efficient basic CI quickly and add advanced
> > > >> capabilities over time.
> > > >>
> > > >> Steffen
> > > >>
> > > >> On Wed, Nov 1, 2017 at 1:14 PM kellen sunderland <
> > > >> kellen.sunderl...@gmail.com> wrote:
> > > >>
> > > >> > Hey Henri, I think that's what a few of us are advocating.
> Running
> > a
> > > set
> > > >> > of quick tests as part of the PR process, and then a more detailed
> > > >> > regression test suite periodically (say every 4 hours). This fits
> > > nicely
> > > >> > into a tagging or 2 branch development system.  Commits will be
> > tagged
> > > >> (or
> > > >> > merged into a stable branch) as soon as they pass the detailed
> > > regression
> > > >> > testing.
> > > >> >
> > > >> > On Wed, Nov 1, 2017 at 9:07 PM, Hen  wrote:
> > > >> >
> > > >> > > Random question - can the CI be split such that the Apache CI is
> > > doing
> > > >> a
> > > >> > > basic set of checks on that hardware, and is hooked to a PR,
> while
> > > >> there
> > > >> > is
> > > >> > > a larger "Is trunk good for release?" test that is running
> > > periodically
> > > >> > > rather than on every PR?
> > > >> > >
> > > >> > > ie: do we need each PR to be run on varied hardware, or can we
> > have
> > > >> this
> > > >> > > two tier approach?
> > > >> > >
> > > >> > > Hen
> > > >> > >
> > > >> > > On Fri, Oct 20, 2017 at 1:01 PM, sandeep krishnamurthy <
> > > >> > > sandeep.krishn...@gmail.com> wrote:
> > > >> > >
> > > >> > > > Hello all,
> > > >> > > >
> > > >> > > > I am hereby opening up a discussion thread on how we can
> > stabilize
> > > >> > Apache
> > > >> > > > MXNet CI build system.
> > > >> > > >
> > > >> > > > Problems:
> > > >> > > >
> > > >> > > > 
> > > >> > > >
> > > >> > > > Recently, we have seen following issues with Apache MXNet CI
> > build
> > > >> > > systems:
> > > >> > > >
> > > >> > > >1. Apache Jenkins master is overloaded and we see issues
> > like -
> > > >> > unable
> > > >> > > >to trigger builds, difficult to load and view the blue
> ocean
> > > and
> > > >> > other
> > > >> > > >Jenkins build status page.
> > > >> > > >2. We are generating too many request/interaction on Apache
> > > Infra
> > > >> > > team.
> > > >> > > >   1. Addition/deletion of new slave: Caused from scaling
> > > >> activity,
> > > >> > > >   recycling, troubleshooting or any actions leading to
> > change
> > > of
> > > >> > > slave
> > > >> > > >   machines.
> > > >> > > >   2. Plugins / other Jenkins Master configurations.
> > > >> > > >   3. Experimentation on CI pipelines.
> > > >> > > >3. Harder to debug and resolve issues - Since access to
> > master
> > > and
> > > >> > > slave
> > > >> > > >is not with the same community, it requires Infra and
> > > community to
> > > >> > 

Re: [Proposal] Stabilizing Apache MXNet CI build system

[Meghna Baijal]

Thanks for the active discussion on the document for the new CI for MXNet.
Now that many of you have reviewed it, do you think I should start a vote
on which framework the community wants to move forward with ?

Thanks,
Meghna

On Mon, Nov 6, 2017 at 6:59 PM, Chris Olivier  wrote:

> After a decision is reached, i am willing to add tasks to Apache MXNet JIRA
>
> On Mon, Nov 6, 2017 at 6:15 AM, Pedro Larroy  >
> wrote:
>
> > Thanks for setting up the document guys, looks like a solid basis to
> > start to work on!
> >
> > Marco, Kellen and I have already added some comments.
> >
> > Pedro
> >
> >
> > On Sun, Nov 5, 2017 at 3:43 AM, Meghna Baijal
> >  wrote:
> > > Kellen, Thank you for your comments in the doc.
> > > Sure Steffen, I will continue to merge everyone’s comments into the doc
> > and
> > > work with Pedro to finalize it.
> > > And then we can vote on the options.
> > >
> > > Thanks,
> > > Meghna Baijal
> > >
> > >
> > > On Sat, Nov 4, 2017 at 6:34 AM, Steffen Rochel <
> steffenroc...@gmail.com>
> > > wrote:
> > >
> > >> Sandeep and Meghna have been working in background collecting input
> and
> > >> preparing a doc. I suggest to drive discussion forward and would like
> to
> > >> ask everybody to contribute to
> > >> https://docs.google.com/document/d/17PEasQ2VWrXi2Cf7IGZSWGZMawxDk
> > >> dlavUDASzUmLjk/edit?usp=sharing
> > >>
> > >> Lets converge on requirements and architecture, so we can move forward
> > with
> > >> implementation.
> > >>
> > >> I would like to suggest for Pedro  and Meghna to lead the discussion
> and
> > >> help to resolve suggestions.
> > >>
> > >> I assume we need a vote once we are converged on a good draft to call
> > it a
> > >> plan and move forward with implementation. As we all are unhappy with
> > the
> > >> current CI situation I would also suggest a phased approach, so we can
> > get
> > >> back to reliable and efficient basic CI quickly and add advanced
> > >> capabilities over time.
> > >>
> > >> Steffen
> > >>
> > >> On Wed, Nov 1, 2017 at 1:14 PM kellen sunderland <
> > >> kellen.sunderl...@gmail.com> wrote:
> > >>
> > >> > Hey Henri, I think that's what a few of us are advocating.  Running
> a
> > set
> > >> > of quick tests as part of the PR process, and then a more detailed
> > >> > regression test suite periodically (say every 4 hours). This fits
> > nicely
> > >> > into a tagging or 2 branch development system.  Commits will be
> tagged
> > >> (or
> > >> > merged into a stable branch) as soon as they pass the detailed
> > regression
> > >> > testing.
> > >> >
> > >> > On Wed, Nov 1, 2017 at 9:07 PM, Hen  wrote:
> > >> >
> > >> > > Random question - can the CI be split such that the Apache CI is
> > doing
> > >> a
> > >> > > basic set of checks on that hardware, and is hooked to a PR, while
> > >> there
> > >> > is
> > >> > > a larger "Is trunk good for release?" test that is running
> > periodically
> > >> > > rather than on every PR?
> > >> > >
> > >> > > ie: do we need each PR to be run on varied hardware, or can we
> have
> > >> this
> > >> > > two tier approach?
> > >> > >
> > >> > > Hen
> > >> > >
> > >> > > On Fri, Oct 20, 2017 at 1:01 PM, sandeep krishnamurthy <
> > >> > > sandeep.krishn...@gmail.com> wrote:
> > >> > >
> > >> > > > Hello all,
> > >> > > >
> > >> > > > I am hereby opening up a discussion thread on how we can
> stabilize
> > >> > Apache
> > >> > > > MXNet CI build system.
> > >> > > >
> > >> > > > Problems:
> > >> > > >
> > >> > > > 
> > >> > > >
> > >> > > > Recently, we have seen following issues with Apache MXNet CI
> build
> > >> > > systems:
> > >> > > >
> > >> > > >1. Apache Jenkins master is overloaded and we see issues
> like -
> > >> > unable
> > >> > > >to trigger builds, difficult to load and view the blue ocean
> > and
> > >> > other
> > >> > > >Jenkins build status page.
> > >> > > >2. We are generating too many request/interaction on Apache
> > Infra
> > >> > > team.
> > >> > > >   1. Addition/deletion of new slave: Caused from scaling
> > >> activity,
> > >> > > >   recycling, troubleshooting or any actions leading to
> change
> > of
> > >> > > slave
> > >> > > >   machines.
> > >> > > >   2. Plugins / other Jenkins Master configurations.
> > >> > > >   3. Experimentation on CI pipelines.
> > >> > > >3. Harder to debug and resolve issues - Since access to
> master
> > and
> > >> > > slave
> > >> > > >is not with the same community, it requires Infra and
> > community to
> > >> > > dive
> > >> > > >deep together on all action items.
> > >> > > >
> > >> > > > Possible Solutions:
> > >> > > >
> > >> > > > ==
> > >> > > >
> > >> > > >1. Can we set up a separate Jenkins CI build system for
> Apache
> > >> MXNet
> > >> > > >outside Apache Infra?
> > >> > > >2. Can we have a separate Jenkins Master in Apache Infra for
> > >> MXNet?
> > >> > > >

Re: [VOTE] Release Apache MXNet(incubating) version 0.12.1.rc0

[Indhu]

+1 (binding)

Built from source and ran some example code.


On Tue, Nov 7, 2017 at 7:48 PM Meghna Baijal 
wrote:

> This is the vote to release Apache MXNet (incubating) version 0.12.1.
>
> Voting will start now (Tuesday, November 7, 2017) and
>
> close Friday, November 10, 2017
>
>
> Link to release notes:
>
>
> https://cwiki.apache.org/confluence/display/MXNET/MXNet+0.12.1+Release+Notes
>
>
> Link to release candidate 0.12.1.rc0:
>
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/0.12.1.rc0/
>
>
> View this page and scroll down to “Build from Source” to build this
> project:
>
> https://mxnet.incubator.apache.org/install/index.html
>
>
> The release tag can be found here:
>
> https://github.com/apache/incubator-mxnet/tree/0.12.1.rc0
>
> (Note: The README.md points to the 0.12.1 tag and does not work at the
> moment.)
>
>
>
> Please make sure you TEST before you vote accordingly:
>
>
> +1 = approve
>
>
> +0 = no opinion
>
>
> -1 = disapprove (provide reason)
>
>
>
> Thanks,
>
> Meghna Baijal
>