Hi everyone, Just some status update regarding the release:
1. More license fixes by @mbaijal are merged. https://github.com/apache/incubator-mxnet/pull/9554 for the perl package https://github.com/apache/incubator-mxnet/pull/9556 for the docs folder https://github.com/apache/incubator-mxnet/pull/9559 for the ci_build folder 2. Two tests failed intermittently, reported by @marcoabreu in another thread. We(@anirudh2290 and I) are looking into them. - test_operator_gpu.test_correlation <http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-9560/1/pipeline/> There were no changes for the test and implementation of this operator for the past two months. Is anyone using this operator? We are currently trying to reproduce the error. - test_forward.test_consistency <http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-9522/1/pipeline/> It looks like the downloaded numpy file is truncated. We'll make a PR to update the test. 3. More API changes since 1.0.0 reported by @szha - https://github.com/apache/incubator-mxnet/pull/9040 <https://github.com/apache/incubator-mxnet/pull/9040> - https://github.com/apache/incubator-mxnet/pull/9420 <https://github.com/apache/incubator-mxnet/pull/9420> - https://github.com/apache/incubator-mxnet/pull/9306 Best, Haibin On Thu, Jan 25, 2018 at 8:08 AM, Steffen Rochel <steffenroc...@gmail.com> wrote: > I support the proposal from Nan - this is a practical and productive way. I > include Nan's description into > https://cwiki.apache.org/confluence/display/MXNET/Release+ > Versioning+and+Branching > > > We should make API changes very carefully and need to depend on the > community to flag any changes until we have better test coverage. > > Steffen > > On Thu, Jan 25, 2018 at 7:30 AM kellen sunderland < > kellen.sunderl...@gmail.com> wrote: > > > @Marco: Ok, well then it sounds like a good time for the community to > > re-think how we look for API changes ;-). We can continue the chat in > the > > semver proposal, but maybe we could create a collection of APIs we > consider > > to be semver'd and review those interfaces each release. Spending > reviewer > > and contributor time each PR on something that we ultimately ignore > doesn't > > seem productive. > > > > On Thu, Jan 25, 2018 at 4:29 PM, kellen sunderland < > > kellen.sunderl...@gmail.com> wrote: > > > > > Yes this is also what I'd suggest Nan, sorry if I wasn't clear. My > > > comment was referring to 2. So as an example for our current release > we > > > could cut a new minor release including new features such as the text > > api, > > > scala rename, but we could cherry-pick the important bug fixes and > apply > > > them to the 1.0.x branch. > > > > > > On Thu, Jan 25, 2018 at 4:22 PM, Nan Zhu <zhunanmcg...@gmail.com> > wrote: > > > > > >> regarding "I'd agree that we should apply most of the fixes to the > > >> previous > > >> release branch and build from there." > > >> > > >> my suggestion is actually a bit different with this, my idea is that > > >> > > >> 1. we always work with master branch, and when there is a date for > > >> releasing a new version (minor or major) one, we cut a new branch and > > >> announce code freeze for that version (of course, there is some > > exception > > >> to merge the previously-ignored blockers to the newly cut branch) > > >> > > >> 2. after the release, we still work in master for the next (at least > > minor > > >> version) and cautiously backport the patches to the last cut branch > > >> (assuming that we always maintain only one previous minor version) > > >> > > >> with this model, what we would do now is cut a new 1.1 branch for the > > >> coming release, if it is necessary to have a maintenance version, we > > would > > >> cherry-pick the important and backward-compatible fixes to 1.0.x > branch > > >> (though ideally this should be done when merging fixes to master ) > > >> > > >> Best, > > >> > > >> Nan > > >> > > >> > > >> > > >> On Thu, Jan 25, 2018 at 2:01 AM, kellen sunderland < > > >> kellen.sunderl...@gmail.com> wrote: > > >> > > >> > -.5 (non-binding) to releasing as a minor release. If we don't have > > any > > >> > breaking API changes, and we haven't added any major features I > would > > >> tend > > >> > to release this as a patch release. The reason being that > > organizations > > >> > and users will know that they can apply this release without making > > >> major > > >> > changes to their dependencies. It also helps set expectations > around > > >> the > > >> > degree of regression testing you'd expect to do on a release > > (typically > > >> > patch releases would require less testing). For that reason if we > > >> release > > >> > as a patch release I think we could expect better adoption rates in > > the > > >> > community and within large tech orgs. If we release as a minor > > release > > >> we > > >> > should expect that many customers may take a long time to update, > and > > >> as a > > >> > community we will be forced to provide support for bugs which have > > >> already > > >> > been fixed. > > >> > > > >> > +1 (non-binding) In terms of branching I'd agree that we should > apply > > >> most > > >> > of the fixes to the previous release branch and build from there. > > >> Happy to > > >> > help with this if needed. > > >> > > > >> > On Thu, Jan 25, 2018 at 6:19 AM, Nan Zhu <zhunanmcg...@gmail.com> > > >> wrote: > > >> > > > >> > > +1 and suggest consolidating all maintenance releases under the > same > > >> > > major.minor version into a single branch > > >> > > > > >> > > On Wed, Jan 24, 2018 at 9:06 PM, Meghna Baijal < > > >> > meghnabaijal2...@gmail.com > > >> > > > > > >> > > wrote: > > >> > > > > >> > > > I agree. If the release candidate is being cut from the master > > >> branch, > > >> > it > > >> > > > should be considered a minor release. > > >> > > > > > >> > > > Anyway the effort involved in the release process is exactly the > > >> same > > >> > in > > >> > > > either case. > > >> > > > > > >> > > > Thanks, > > >> > > > Meghna > > >> > > > > > >> > > > On Jan 24, 2018 8:56 PM, "Marco de Abreu" < > > >> > marco.g.ab...@googlemail.com> > > >> > > > wrote: > > >> > > > > > >> > > > > Are there any particular reasons why we are classifying this > > >> release > > >> > as > > >> > > > > patch instead of minor release? As far as I know, we don't > have > > >> any > > >> > > tests > > >> > > > > in place to determine API changes and thus can't guarantee > that > > >> this > > >> > is > > >> > > > an > > >> > > > > actual patch release. Considering the fact that PRs have been > > >> merged > > >> > > > > without having semantic versioning in place, this could be > quite > > >> > risky. > > >> > > > > > > >> > > > > Instead, I'd rather propose to make a minor release 1.1 > instead > > of > > >> > > patch > > >> > > > > release 1.0.1. > > >> > > > > > > >> > > > > -Marco > > >> > > > > > > >> > > > > Am 24.01.2018 7:20 nachm. schrieb "Zha, Sheng" < > > >> zhash...@amazon.com > > >> > >: > > >> > > > > > > >> > > > > > There’s an experimental API for text data indexing and > > >> embedding in > > >> > > > > > mx.contrib.text. > > >> > > > > > > > >> > > > > > - Sent by my thumb > > >> > > > > > > > >> > > > > > > On Jan 24, 2018, at 7:08 PM, Chris Olivier < > > >> > cjolivie...@gmail.com> > > >> > > > > > wrote: > > >> > > > > > > > > >> > > > > > > the profiling PR contains a small breaking change, but i > > don’t > > >> > > think > > >> > > > > it’s > > >> > > > > > > going into 1.0.1 > > >> > > > > > > > > >> > > > > > >> On Wed, Jan 24, 2018 at 6:48 PM Haibin Lin < > > >> > > > haibin.lin....@gmail.com> > > >> > > > > > wrote: > > >> > > > > > >> > > >> > > > > > >> Hi everyone, > > >> > > > > > >> > > >> > > > > > >> Since the plan was to cut a branch from the master > branch, > > >> the > > >> > > code > > >> > > > > will > > >> > > > > > >> include changes other than the bug fix PRs noted in the > > >> release > > >> > > > note. > > >> > > > > Is > > >> > > > > > >> anyone aware of any API changes in the current MXNet > master > > >> > > branch? > > >> > > > In > > >> > > > > > >> particular, are there backward incompatible ones? > > >> > > > > > >> > > >> > > > > > >> Best, > > >> > > > > > >> Haibin > > >> > > > > > >> > > >> > > > > > >> On Tue, Jan 23, 2018 at 11:25 AM, Haibin Lin < > > >> > > > > haibin.lin....@gmail.com> > > >> > > > > > >> wrote: > > >> > > > > > >> > > >> > > > > > >>> Hi Sheng, > > >> > > > > > >>> > > >> > > > > > >>> 1. I've been following the discussion on the branching & > > >> > > versioning > > >> > > > > > >>> thread. Features like MKLDNN integration should not go > to > > >> patch > > >> > > > > release > > >> > > > > > >>> 1.0.1, and it's risky to merge large PRs right before > the > > >> > > release. > > >> > > > > I've > > >> > > > > > >>> removed the MKLDNN section from the release note. > > >> > > > > https://cwiki.apache > > >> > > > > > . > > >> > > > > > >>> > > org/confluence/display/MXNET/Apache+MXNet+%28incubating%29+ > > >> > > > > > >>> 1.0.1+Release+Notes > > >> > > > > > >>> > > >> > > > > > >>> 2. I agree that we should aim for better test coverage & > > >> stable > > >> > > CI, > > >> > > > > and > > >> > > > > > >>> get those disabled/flaky tests fixed eventually. Fixing > > >> these > > >> > > > > requires > > >> > > > > > >>> efforts from the community and I strongly encourage > > >> > contributors > > >> > > to > > >> > > > > > help. > > >> > > > > > >>> Removing the corresponding feature from the release > > doesn't > > >> > sound > > >> > > > > > >> practical > > >> > > > > > >>> since users might be already using some of those. I > > suggest > > >> > that > > >> > > we > > >> > > > > > keep > > >> > > > > > >>> track of these tests on Apache Wiki and make sure they > are > > >> > > > addressed > > >> > > > > > for > > >> > > > > > >>> the release after 1.0.1. > > >> > > > > > >>> > > >> > > > > > >>> Hi everyone, > > >> > > > > > >>> > > >> > > > > > >>> In terms of the current status for this release, all > > >> critical > > >> > bug > > >> > > > > fixes > > >> > > > > > >>> are merged (to the best of my knowledge) and we have > made > > >> good > > >> > > > > progress > > >> > > > > > >>> fixing license issues. As Meghna mentioned, a list of > open > > >> > > > questions > > >> > > > > > >>> regarding license is at > > >> > > > > > >> https://cwiki.apache.org/confluence/display/MXNET/ > > >> > > > > > >>> MXNet+Source+Licenses section D - it would be great if > we > > >> can > > >> > get > > >> > > > > more > > >> > > > > > >>> clarification/help/feedback from Apache mentors. > > >> > > > > > >>> > > >> > > > > > >>> I suggest that we shoot for code freeze for 1.0.1 rc0 > this > > >> > > > Wednesday. > > >> > > > > > >> Does > > >> > > > > > >>> anyone have concern or objection on this? > > >> > > > > > >>> > > >> > > > > > >>> Best, > > >> > > > > > >>> Haibin > > >> > > > > > >>> > > >> > > > > > >>> On Tue, Jan 23, 2018 at 7:51 AM, Steffen Rochel < > > >> > > > > > steffenroc...@gmail.com > > >> > > > > > >>> > > >> > > > > > >>> wrote: > > >> > > > > > >>> > > >> > > > > > >>>> Hi Sheng - > > >> > > > > > >>>> 1. branch usage and versioning - lets converge our > > >> discussion > > >> > > and > > >> > > > > > >> document > > >> > > > > > >>>> the agreement on wiki. I started a draft summarizing my > > >> > > > > understanding > > >> > > > > > of > > >> > > > > > >>>> the proposal at > > >> > > > > > >>>> > > https://cwiki.apache.org/confluence/display/MXNET/Release+ > > >> > > > > > >>>> Versioning+and+Branching. > > >> > > > > > >>>> Lets work together to refine and clarify the draft, so > we > > >> have > > >> > > > > clarity > > >> > > > > > >>>> going forward. I'm inviting everyone to contribute to > > this > > >> > > > > discussion. > > >> > > > > > >>>> As MKLDNN integration is not ready yet and we want to > > >> release > > >> > > all > > >> > > > > the > > >> > > > > > >> good > > >> > > > > > >>>> improvements including updates in tutorials and > > >> documentation > > >> > I > > >> > > > > > suggest > > >> > > > > > >> we > > >> > > > > > >>>> move forward with the release asap. As we don't have > > major > > >> > > > features > > >> > > > > or > > >> > > > > > >>>> non-compatible API changes (to best of my knowledge) I > > >> think > > >> > it > > >> > > is > > >> > > > > > >>>> appropriate to label the release as 1.0.1. > > >> > > > > > >>>> Note: This label indicates a patch release. Patch > > releases > > >> > > should > > >> > > > be > > >> > > > > > >>>> created from the related release branch. As we didn't > > plan > > >> for > > >> > > it > > >> > > > > and > > >> > > > > > to > > >> > > > > > >>>> minimize overhead I suggest we make a one time > exception > > to > > >> > cut > > >> > > > the > > >> > > > > > >> 1.0.1 > > >> > > > > > >>>> release from master branch and clearly communicate in > > >> release > > >> > > > notes. > > >> > > > > > >> Going > > >> > > > > > >>>> forward we should follow the methodology for versioning > > and > > >> > > > > branching > > >> > > > > > to > > >> > > > > > >>>> whatever we agree on. > > >> > > > > > >>>> 2. Disabled tests: I agree with your concerns that we > had > > >> to > > >> > > > disable > > >> > > > > > 13 > > >> > > > > > >>>> tests due to non-deterministic behavior (see issues > > >> > > > > > >>>> <https://github.com/apache/inc > ubator-mxnet/labels/Flaky > > >). > > >> > I'm > > >> > > > > > calling > > >> > > > > > >> on > > >> > > > > > >>>> all contributors to help to resolve the > non-deterministic > > >> > > > behavior, > > >> > > > > so > > >> > > > > > >> we > > >> > > > > > >>>> can improve our test coverage. As we discussed offline, > > >> lets > > >> > > tests > > >> > > > > > >>>> manually > > >> > > > > > >>>> short term, document the known issue in the release > notes > > >> and > > >> > > > > > prioritize > > >> > > > > > >>>> efforts post 1.0.1 release. > > >> > > > > > >>>> > > >> > > > > > >>>> Regards, > > >> > > > > > >>>> Steffen > > >> > > > > > >>>> > > >> > > > > > >>>>> On Wed, Jan 17, 2018 at 5:05 PM Sheng Zha < > > >> > zhash...@apache.org > > >> > > > > > >> > > > > > wrote: > > >> > > > > > >>>>> > > >> > > > > > >>>>> Hi Haibin, > > >> > > > > > >>>>> > > >> > > > > > >>>>> Thanks for leading this. I suggest that we hold onto > > this > > >> > > release > > >> > > > > > >> until > > >> > > > > > >>>> we > > >> > > > > > >>>>> have clarity on the following items. > > >> > > > > > >>>>> > > >> > > > > > >>>>> 1. branch usage and versioning > > >> > > > > > >>>>> Given that we are past 1.0 and we're changing APIs, > I'd > > >> like > > >> > to > > >> > > > > > >> suggest > > >> > > > > > >>>>> that we first agree on how > > >> > > > > > >>>>> versioning works in mxnet. If we follow semantic > > >> versioning, > > >> > it > > >> > > > > would > > >> > > > > > >>>>> suggest that features like > > >> > > > > > >>>>> MKL-DNN should go at least into 1.1 (minor version > > change) > > >> > > > instead > > >> > > > > of > > >> > > > > > >>>>> 1.0.1 (patch release). > > >> > > > > > >>>>> Also, assuming that new release will come from a new > > >> forked > > >> > > > > branch, I > > >> > > > > > >>>>> suggest that we clarify on how to > > >> > > > > > >>>>> name the branches too. > > >> > > > > > >>>>> You can find relevant thread at > > >> > > > > > >>>>> https://lists.apache.org/threa > > >> d.html/c52f8353f63c1e63b2646ec > > >> > > > > > >>>> 3b08d9a8180a1381787d777b41b8ac69f@% > > 3Cdev.mxnet.apache.org > > >> %3E > > >> > > > > > >>>>> > > >> > > > > > >>>>> 2. disabled tests > > >> > > > > > >>>>> For the purpose of stabilizing test automation system, > > >> many > > >> > > tests > > >> > > > > > were > > >> > > > > > >>>>> disabled. In order to avoid > > >> > > > > > >>>>> releasing untested features, we should mitigate the > > >> situation > > >> > > of > > >> > > > > > >> having > > >> > > > > > >>>>> disabled tests. > > >> > > > > > >>>>> That means we can fix the tests before the release, or > > >> remove > > >> > > the > > >> > > > > > >>>>> corresponding feature from release > > >> > > > > > >>>>> (might be hard to do, e.g. for optimizer). Otherwise, > we > > >> must > > >> > > > > > >>>> collectively > > >> > > > > > >>>>> decide that a feature is > > >> > > > > > >>>>> OK to release without tests. > > >> > > > > > >>>>> The thread on this topic can be found at > > >> > > > > > >>>>> https://lists.apache.org/threa > > >> d.html/addab1937bfcf09b5dfa15c > > >> > > > > > >>>> 1149ddcebd084f1c4bf4e10a73770fb35@% > > 3Cdev.mxnet.apache.org > > >> %3E > > >> > > > > > >>>>> > > >> > > > > > >>>>> We can proceed on the release with more confidence > once > > we > > >> > have > > >> > > > > > >> clarity. > > >> > > > > > >>>>> > > >> > > > > > >>>>> Best regards, > > >> > > > > > >>>>> -sz > > >> > > > > > >>>>> > > >> > > > > > >>>>>> On 2018-01-10 15:33, Haibin Lin < > > >> haibin.lin....@gmail.com> > > >> > > > wrote: > > >> > > > > > >>>>>> I am starting the process to prepare for MXNET 1.0.1 > > >> > release. > > >> > > I > > >> > > > > have > > >> > > > > > >>>>>> drafted release notes > > >> > > > > > >>>>>> (* > > >> > > > > > >>>>> > > https://cwiki.apache.org/confluence/display/MXNET/Apache+ > > >> > > > > > >>>> MXNet+%28incubating%29+1.0.1+Release+Notes > > >> > > > > > >>>>>> < > > >> > > > > > >>>>> > > https://cwiki.apache.org/confluence/display/MXNET/Apache+ > > >> > > > > > >>>> MXNet+%28incubating%29+1.0.1+Release+Notes > > >> > > > > > >>>>>> *) > > >> > > > > > >>>>>> to cover the tasks under this release. > > >> > > > > > >>>>>> > > >> > > > > > >>>>>> A release candidate will be cut on Monday 22nd Jan, > > 2018 > > >> and > > >> > > > > voting > > >> > > > > > >>>> will > > >> > > > > > >>>>>> commence from then till Thursday 25th Jan, 2018. If > you > > >> have > > >> > > any > > >> > > > > > >>>>> additional > > >> > > > > > >>>>>> features in progress and would like to include it in > > this > > >> > > > release, > > >> > > > > > >>>> please > > >> > > > > > >>>>>> assure they have been merged by Thursday 18th Jan, > 2018 > > >> with > > >> > > > > comment > > >> > > > > > >>>> so I > > >> > > > > > >>>>>> may update the release notes. > > >> > > > > > >>>>>> > > >> > > > > > >>>>>> Feel free to add any other comments/suggestions. > > >> > > > > > >>>>>> > > >> > > > > > >>>>>> Thanks, > > >> > > > > > >>>>>> Haibin > > >> > > > > > >>>>>> > > >> > > > > > >>>>> > > >> > > > > > >>>> > > >> > > > > > >>> > > >> > > > > > >>> > > >> > > > > > >> > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > >
