Hi everyone, Just some status update:
1. The two tests reported in #9553 are both fixed by @anirudh2290. 2. Many broken website links are fixed in #9575 by @thinksanky. 3. I made some updates to the release note. Please help edit the note and so that it reflects all changes in MXNet including new functionalities in the contrib namespace. Best, Haibin On Thu, Jan 25, 2018 at 6:26 PM, Haibin Lin <haibin.lin....@gmail.com> wrote: > Hi everyone, > > Just some status update regarding the release: > > 1. More license fixes by @mbaijal are merged. > https://github.com/apache/incubator-mxnet/pull/9554 for the perl package > https://github.com/apache/incubator-mxnet/pull/9556 for the docs folder > https://github.com/apache/incubator-mxnet/pull/9559 for the ci_build > folder > > 2. Two tests failed intermittently, reported by @marcoabreu in another > thread. We(@anirudh2290 and I) are looking into them. > - test_operator_gpu.test_correlation > <http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-9560/1/pipeline/> > There were no changes for the test and implementation of this operator for > the past two months. Is anyone using this operator? We are currently trying > to reproduce the error. > - test_forward.test_consistency > <http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-9522/1/pipeline/> > It looks like the downloaded numpy file is truncated. We'll make a PR to > update the test. > > 3. More API changes since 1.0.0 reported by @szha > - https://github.com/apache/incubator-mxnet/pull/9040 > <https://github.com/apache/incubator-mxnet/pull/9040> > - https://github.com/apache/incubator-mxnet/pull/9420 > <https://github.com/apache/incubator-mxnet/pull/9420> > - https://github.com/apache/incubator-mxnet/pull/9306 > > Best, > Haibin > > On Thu, Jan 25, 2018 at 8:08 AM, Steffen Rochel <steffenroc...@gmail.com> > wrote: > >> I support the proposal from Nan - this is a practical and productive way. >> I >> include Nan's description into >> https://cwiki.apache.org/confluence/display/MXNET/Release+Ve >> rsioning+and+Branching >> >> >> We should make API changes very carefully and need to depend on the >> community to flag any changes until we have better test coverage. >> >> Steffen >> >> On Thu, Jan 25, 2018 at 7:30 AM kellen sunderland < >> kellen.sunderl...@gmail.com> wrote: >> >> > @Marco: Ok, well then it sounds like a good time for the community to >> > re-think how we look for API changes ;-). We can continue the chat in >> the >> > semver proposal, but maybe we could create a collection of APIs we >> consider >> > to be semver'd and review those interfaces each release. Spending >> reviewer >> > and contributor time each PR on something that we ultimately ignore >> doesn't >> > seem productive. >> > >> > On Thu, Jan 25, 2018 at 4:29 PM, kellen sunderland < >> > kellen.sunderl...@gmail.com> wrote: >> > >> > > Yes this is also what I'd suggest Nan, sorry if I wasn't clear. My >> > > comment was referring to 2. So as an example for our current release >> we >> > > could cut a new minor release including new features such as the text >> > api, >> > > scala rename, but we could cherry-pick the important bug fixes and >> apply >> > > them to the 1.0.x branch. >> > > >> > > On Thu, Jan 25, 2018 at 4:22 PM, Nan Zhu <zhunanmcg...@gmail.com> >> wrote: >> > > >> > >> regarding "I'd agree that we should apply most of the fixes to the >> > >> previous >> > >> release branch and build from there." >> > >> >> > >> my suggestion is actually a bit different with this, my idea is that >> > >> >> > >> 1. we always work with master branch, and when there is a date for >> > >> releasing a new version (minor or major) one, we cut a new branch and >> > >> announce code freeze for that version (of course, there is some >> > exception >> > >> to merge the previously-ignored blockers to the newly cut branch) >> > >> >> > >> 2. after the release, we still work in master for the next (at least >> > minor >> > >> version) and cautiously backport the patches to the last cut branch >> > >> (assuming that we always maintain only one previous minor version) >> > >> >> > >> with this model, what we would do now is cut a new 1.1 branch for the >> > >> coming release, if it is necessary to have a maintenance version, we >> > would >> > >> cherry-pick the important and backward-compatible fixes to 1.0.x >> branch >> > >> (though ideally this should be done when merging fixes to master ) >> > >> >> > >> Best, >> > >> >> > >> Nan >> > >> >> > >> >> > >> >> > >> On Thu, Jan 25, 2018 at 2:01 AM, kellen sunderland < >> > >> kellen.sunderl...@gmail.com> wrote: >> > >> >> > >> > -.5 (non-binding) to releasing as a minor release. If we don't >> have >> > any >> > >> > breaking API changes, and we haven't added any major features I >> would >> > >> tend >> > >> > to release this as a patch release. The reason being that >> > organizations >> > >> > and users will know that they can apply this release without making >> > >> major >> > >> > changes to their dependencies. It also helps set expectations >> around >> > >> the >> > >> > degree of regression testing you'd expect to do on a release >> > (typically >> > >> > patch releases would require less testing). For that reason if we >> > >> release >> > >> > as a patch release I think we could expect better adoption rates in >> > the >> > >> > community and within large tech orgs. If we release as a minor >> > release >> > >> we >> > >> > should expect that many customers may take a long time to update, >> and >> > >> as a >> > >> > community we will be forced to provide support for bugs which have >> > >> already >> > >> > been fixed. >> > >> > >> > >> > +1 (non-binding) In terms of branching I'd agree that we should >> apply >> > >> most >> > >> > of the fixes to the previous release branch and build from there. >> > >> Happy to >> > >> > help with this if needed. >> > >> > >> > >> > On Thu, Jan 25, 2018 at 6:19 AM, Nan Zhu <zhunanmcg...@gmail.com> >> > >> wrote: >> > >> > >> > >> > > +1 and suggest consolidating all maintenance releases under the >> same >> > >> > > major.minor version into a single branch >> > >> > > >> > >> > > On Wed, Jan 24, 2018 at 9:06 PM, Meghna Baijal < >> > >> > meghnabaijal2...@gmail.com >> > >> > > > >> > >> > > wrote: >> > >> > > >> > >> > > > I agree. If the release candidate is being cut from the master >> > >> branch, >> > >> > it >> > >> > > > should be considered a minor release. >> > >> > > > >> > >> > > > Anyway the effort involved in the release process is exactly >> the >> > >> same >> > >> > in >> > >> > > > either case. >> > >> > > > >> > >> > > > Thanks, >> > >> > > > Meghna >> > >> > > > >> > >> > > > On Jan 24, 2018 8:56 PM, "Marco de Abreu" < >> > >> > marco.g.ab...@googlemail.com> >> > >> > > > wrote: >> > >> > > > >> > >> > > > > Are there any particular reasons why we are classifying this >> > >> release >> > >> > as >> > >> > > > > patch instead of minor release? As far as I know, we don't >> have >> > >> any >> > >> > > tests >> > >> > > > > in place to determine API changes and thus can't guarantee >> that >> > >> this >> > >> > is >> > >> > > > an >> > >> > > > > actual patch release. Considering the fact that PRs have been >> > >> merged >> > >> > > > > without having semantic versioning in place, this could be >> quite >> > >> > risky. >> > >> > > > > >> > >> > > > > Instead, I'd rather propose to make a minor release 1.1 >> instead >> > of >> > >> > > patch >> > >> > > > > release 1.0.1. >> > >> > > > > >> > >> > > > > -Marco >> > >> > > > > >> > >> > > > > Am 24.01.2018 7:20 nachm. schrieb "Zha, Sheng" < >> > >> zhash...@amazon.com >> > >> > >: >> > >> > > > > >> > >> > > > > > There’s an experimental API for text data indexing and >> > >> embedding in >> > >> > > > > > mx.contrib.text. >> > >> > > > > > >> > >> > > > > > - Sent by my thumb >> > >> > > > > > >> > >> > > > > > > On Jan 24, 2018, at 7:08 PM, Chris Olivier < >> > >> > cjolivie...@gmail.com> >> > >> > > > > > wrote: >> > >> > > > > > > >> > >> > > > > > > the profiling PR contains a small breaking change, but i >> > don’t >> > >> > > think >> > >> > > > > it’s >> > >> > > > > > > going into 1.0.1 >> > >> > > > > > > >> > >> > > > > > >> On Wed, Jan 24, 2018 at 6:48 PM Haibin Lin < >> > >> > > > haibin.lin....@gmail.com> >> > >> > > > > > wrote: >> > >> > > > > > >> >> > >> > > > > > >> Hi everyone, >> > >> > > > > > >> >> > >> > > > > > >> Since the plan was to cut a branch from the master >> branch, >> > >> the >> > >> > > code >> > >> > > > > will >> > >> > > > > > >> include changes other than the bug fix PRs noted in the >> > >> release >> > >> > > > note. >> > >> > > > > Is >> > >> > > > > > >> anyone aware of any API changes in the current MXNet >> master >> > >> > > branch? >> > >> > > > In >> > >> > > > > > >> particular, are there backward incompatible ones? >> > >> > > > > > >> >> > >> > > > > > >> Best, >> > >> > > > > > >> Haibin >> > >> > > > > > >> >> > >> > > > > > >> On Tue, Jan 23, 2018 at 11:25 AM, Haibin Lin < >> > >> > > > > haibin.lin....@gmail.com> >> > >> > > > > > >> wrote: >> > >> > > > > > >> >> > >> > > > > > >>> Hi Sheng, >> > >> > > > > > >>> >> > >> > > > > > >>> 1. I've been following the discussion on the branching >> & >> > >> > > versioning >> > >> > > > > > >>> thread. Features like MKLDNN integration should not go >> to >> > >> patch >> > >> > > > > release >> > >> > > > > > >>> 1.0.1, and it's risky to merge large PRs right before >> the >> > >> > > release. >> > >> > > > > I've >> > >> > > > > > >>> removed the MKLDNN section from the release note. >> > >> > > > > https://cwiki.apache >> > >> > > > > > . >> > >> > > > > > >>> >> > org/confluence/display/MXNET/Apache+MXNet+%28incubating%29+ >> > >> > > > > > >>> 1.0.1+Release+Notes >> > >> > > > > > >>> >> > >> > > > > > >>> 2. I agree that we should aim for better test coverage >> & >> > >> stable >> > >> > > CI, >> > >> > > > > and >> > >> > > > > > >>> get those disabled/flaky tests fixed eventually. Fixing >> > >> these >> > >> > > > > requires >> > >> > > > > > >>> efforts from the community and I strongly encourage >> > >> > contributors >> > >> > > to >> > >> > > > > > help. >> > >> > > > > > >>> Removing the corresponding feature from the release >> > doesn't >> > >> > sound >> > >> > > > > > >> practical >> > >> > > > > > >>> since users might be already using some of those. I >> > suggest >> > >> > that >> > >> > > we >> > >> > > > > > keep >> > >> > > > > > >>> track of these tests on Apache Wiki and make sure they >> are >> > >> > > > addressed >> > >> > > > > > for >> > >> > > > > > >>> the release after 1.0.1. >> > >> > > > > > >>> >> > >> > > > > > >>> Hi everyone, >> > >> > > > > > >>> >> > >> > > > > > >>> In terms of the current status for this release, all >> > >> critical >> > >> > bug >> > >> > > > > fixes >> > >> > > > > > >>> are merged (to the best of my knowledge) and we have >> made >> > >> good >> > >> > > > > progress >> > >> > > > > > >>> fixing license issues. As Meghna mentioned, a list of >> open >> > >> > > > questions >> > >> > > > > > >>> regarding license is at >> > >> > > > > > >> https://cwiki.apache.org/confluence/display/MXNET/ >> > >> > > > > > >>> MXNet+Source+Licenses section D - it would be great if >> we >> > >> can >> > >> > get >> > >> > > > > more >> > >> > > > > > >>> clarification/help/feedback from Apache mentors. >> > >> > > > > > >>> >> > >> > > > > > >>> I suggest that we shoot for code freeze for 1.0.1 rc0 >> this >> > >> > > > Wednesday. >> > >> > > > > > >> Does >> > >> > > > > > >>> anyone have concern or objection on this? >> > >> > > > > > >>> >> > >> > > > > > >>> Best, >> > >> > > > > > >>> Haibin >> > >> > > > > > >>> >> > >> > > > > > >>> On Tue, Jan 23, 2018 at 7:51 AM, Steffen Rochel < >> > >> > > > > > steffenroc...@gmail.com >> > >> > > > > > >>> >> > >> > > > > > >>> wrote: >> > >> > > > > > >>> >> > >> > > > > > >>>> Hi Sheng - >> > >> > > > > > >>>> 1. branch usage and versioning - lets converge our >> > >> discussion >> > >> > > and >> > >> > > > > > >> document >> > >> > > > > > >>>> the agreement on wiki. I started a draft summarizing >> my >> > >> > > > > understanding >> > >> > > > > > of >> > >> > > > > > >>>> the proposal at >> > >> > > > > > >>>> >> > https://cwiki.apache.org/confluence/display/MXNET/Release+ >> > >> > > > > > >>>> Versioning+and+Branching. >> > >> > > > > > >>>> Lets work together to refine and clarify the draft, >> so we >> > >> have >> > >> > > > > clarity >> > >> > > > > > >>>> going forward. I'm inviting everyone to contribute to >> > this >> > >> > > > > discussion. >> > >> > > > > > >>>> As MKLDNN integration is not ready yet and we want to >> > >> release >> > >> > > all >> > >> > > > > the >> > >> > > > > > >> good >> > >> > > > > > >>>> improvements including updates in tutorials and >> > >> documentation >> > >> > I >> > >> > > > > > suggest >> > >> > > > > > >> we >> > >> > > > > > >>>> move forward with the release asap. As we don't have >> > major >> > >> > > > features >> > >> > > > > or >> > >> > > > > > >>>> non-compatible API changes (to best of my knowledge) I >> > >> think >> > >> > it >> > >> > > is >> > >> > > > > > >>>> appropriate to label the release as 1.0.1. >> > >> > > > > > >>>> Note: This label indicates a patch release. Patch >> > releases >> > >> > > should >> > >> > > > be >> > >> > > > > > >>>> created from the related release branch. As we didn't >> > plan >> > >> for >> > >> > > it >> > >> > > > > and >> > >> > > > > > to >> > >> > > > > > >>>> minimize overhead I suggest we make a one time >> exception >> > to >> > >> > cut >> > >> > > > the >> > >> > > > > > >> 1.0.1 >> > >> > > > > > >>>> release from master branch and clearly communicate in >> > >> release >> > >> > > > notes. >> > >> > > > > > >> Going >> > >> > > > > > >>>> forward we should follow the methodology for >> versioning >> > and >> > >> > > > > branching >> > >> > > > > > to >> > >> > > > > > >>>> whatever we agree on. >> > >> > > > > > >>>> 2. Disabled tests: I agree with your concerns that we >> had >> > >> to >> > >> > > > disable >> > >> > > > > > 13 >> > >> > > > > > >>>> tests due to non-deterministic behavior (see issues >> > >> > > > > > >>>> <https://github.com/apache/inc >> ubator-mxnet/labels/Flaky >> > >). >> > >> > I'm >> > >> > > > > > calling >> > >> > > > > > >> on >> > >> > > > > > >>>> all contributors to help to resolve the >> non-deterministic >> > >> > > > behavior, >> > >> > > > > so >> > >> > > > > > >> we >> > >> > > > > > >>>> can improve our test coverage. As we discussed >> offline, >> > >> lets >> > >> > > tests >> > >> > > > > > >>>> manually >> > >> > > > > > >>>> short term, document the known issue in the release >> notes >> > >> and >> > >> > > > > > prioritize >> > >> > > > > > >>>> efforts post 1.0.1 release. >> > >> > > > > > >>>> >> > >> > > > > > >>>> Regards, >> > >> > > > > > >>>> Steffen >> > >> > > > > > >>>> >> > >> > > > > > >>>>> On Wed, Jan 17, 2018 at 5:05 PM Sheng Zha < >> > >> > zhash...@apache.org >> > >> > > > >> > >> > > > > > wrote: >> > >> > > > > > >>>>> >> > >> > > > > > >>>>> Hi Haibin, >> > >> > > > > > >>>>> >> > >> > > > > > >>>>> Thanks for leading this. I suggest that we hold onto >> > this >> > >> > > release >> > >> > > > > > >> until >> > >> > > > > > >>>> we >> > >> > > > > > >>>>> have clarity on the following items. >> > >> > > > > > >>>>> >> > >> > > > > > >>>>> 1. branch usage and versioning >> > >> > > > > > >>>>> Given that we are past 1.0 and we're changing APIs, >> I'd >> > >> like >> > >> > to >> > >> > > > > > >> suggest >> > >> > > > > > >>>>> that we first agree on how >> > >> > > > > > >>>>> versioning works in mxnet. If we follow semantic >> > >> versioning, >> > >> > it >> > >> > > > > would >> > >> > > > > > >>>>> suggest that features like >> > >> > > > > > >>>>> MKL-DNN should go at least into 1.1 (minor version >> > change) >> > >> > > > instead >> > >> > > > > of >> > >> > > > > > >>>>> 1.0.1 (patch release). >> > >> > > > > > >>>>> Also, assuming that new release will come from a new >> > >> forked >> > >> > > > > branch, I >> > >> > > > > > >>>>> suggest that we clarify on how to >> > >> > > > > > >>>>> name the branches too. >> > >> > > > > > >>>>> You can find relevant thread at >> > >> > > > > > >>>>> https://lists.apache.org/threa >> > >> d.html/c52f8353f63c1e63b2646ec >> > >> > > > > > >>>> 3b08d9a8180a1381787d777b41b8ac69f@% >> > 3Cdev.mxnet.apache.org >> > >> %3E >> > >> > > > > > >>>>> >> > >> > > > > > >>>>> 2. disabled tests >> > >> > > > > > >>>>> For the purpose of stabilizing test automation >> system, >> > >> many >> > >> > > tests >> > >> > > > > > were >> > >> > > > > > >>>>> disabled. In order to avoid >> > >> > > > > > >>>>> releasing untested features, we should mitigate the >> > >> situation >> > >> > > of >> > >> > > > > > >> having >> > >> > > > > > >>>>> disabled tests. >> > >> > > > > > >>>>> That means we can fix the tests before the release, >> or >> > >> remove >> > >> > > the >> > >> > > > > > >>>>> corresponding feature from release >> > >> > > > > > >>>>> (might be hard to do, e.g. for optimizer). >> Otherwise, we >> > >> must >> > >> > > > > > >>>> collectively >> > >> > > > > > >>>>> decide that a feature is >> > >> > > > > > >>>>> OK to release without tests. >> > >> > > > > > >>>>> The thread on this topic can be found at >> > >> > > > > > >>>>> https://lists.apache.org/threa >> > >> d.html/addab1937bfcf09b5dfa15c >> > >> > > > > > >>>> 1149ddcebd084f1c4bf4e10a73770fb35@% >> > 3Cdev.mxnet.apache.org >> > >> %3E >> > >> > > > > > >>>>> >> > >> > > > > > >>>>> We can proceed on the release with more confidence >> once >> > we >> > >> > have >> > >> > > > > > >> clarity. >> > >> > > > > > >>>>> >> > >> > > > > > >>>>> Best regards, >> > >> > > > > > >>>>> -sz >> > >> > > > > > >>>>> >> > >> > > > > > >>>>>> On 2018-01-10 15:33, Haibin Lin < >> > >> haibin.lin....@gmail.com> >> > >> > > > wrote: >> > >> > > > > > >>>>>> I am starting the process to prepare for MXNET 1.0.1 >> > >> > release. >> > >> > > I >> > >> > > > > have >> > >> > > > > > >>>>>> drafted release notes >> > >> > > > > > >>>>>> (* >> > >> > > > > > >>>>> >> > https://cwiki.apache.org/confluence/display/MXNET/Apache+ >> > >> > > > > > >>>> MXNet+%28incubating%29+1.0.1+Release+Notes >> > >> > > > > > >>>>>> < >> > >> > > > > > >>>>> >> > https://cwiki.apache.org/confluence/display/MXNET/Apache+ >> > >> > > > > > >>>> MXNet+%28incubating%29+1.0.1+Release+Notes >> > >> > > > > > >>>>>> *) >> > >> > > > > > >>>>>> to cover the tasks under this release. >> > >> > > > > > >>>>>> >> > >> > > > > > >>>>>> A release candidate will be cut on Monday 22nd Jan, >> > 2018 >> > >> and >> > >> > > > > voting >> > >> > > > > > >>>> will >> > >> > > > > > >>>>>> commence from then till Thursday 25th Jan, 2018. If >> you >> > >> have >> > >> > > any >> > >> > > > > > >>>>> additional >> > >> > > > > > >>>>>> features in progress and would like to include it in >> > this >> > >> > > > release, >> > >> > > > > > >>>> please >> > >> > > > > > >>>>>> assure they have been merged by Thursday 18th Jan, >> 2018 >> > >> with >> > >> > > > > comment >> > >> > > > > > >>>> so I >> > >> > > > > > >>>>>> may update the release notes. >> > >> > > > > > >>>>>> >> > >> > > > > > >>>>>> Feel free to add any other comments/suggestions. >> > >> > > > > > >>>>>> >> > >> > > > > > >>>>>> Thanks, >> > >> > > > > > >>>>>> Haibin >> > >> > > > > > >>>>>> >> > >> > > > > > >>>>> >> > >> > > > > > >>>> >> > >> > > > > > >>> >> > >> > > > > > >>> >> > >> > > > > > >> >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > > >> > > >> > >> > >
