Hi Roshani, Good notes :)
Several items about the performance and MKL-DNN in the below, please help take a review. @Da, Alex, if anything about MKL-DNN is missed, feel free to add. *Performance improvement +Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on CPU https://github.com/apache/incubator-mxnet/pull/11113 +Performance improvement for Batch Dot on CPU from mshadow https://github.com/dmlc/mshadow/pull/342 -Fix the topk regression issue (#12197) This is the bugfix rather than performance improvements *MKL-DNN More functionality supports: +Support more activation functions, "sigmoid", "tanh", "softrelu" https://github.com/apache/incubator-mxnet/pull/10336 Debugging functionality: +Result check https://github.com/apache/incubator-mxnet/pull/12069 +Backend switch https://github.com/apache/incubator-mxnet/pull/12058 Thanks, --Patric > -----Original Message----- > From: Roshani Nagmote [mailto:[email protected]] > Sent: Wednesday, August 22, 2018 1:53 AM > To: [email protected] > Subject: Re: Release plan - MXNET 1.3 > > Hi, > > Thank you everyone for helping to clear release blockers. CI tests were > failing > so we delayed RC by some time. But now the tests are passing and we are > ready to cut the release branch. > > I have drafted release notes here: > https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+%28in > cubating%29+1.3.0+Release+Notes > > > Please take a look and update if I have missed anything. I will be cutting > RC0 tomorrow. > > Thanks, > Roshani > > On Thu, Aug 16, 2018 at 2:28 PM Roshani Nagmote > <[email protected]> > wrote: > > > Sure will do. thanks. > > > > -Roshani > > > > On Thu, Aug 16, 2018 at 11:53 AM Afrooze, Sina <[email protected]> > wrote: > > > >> Hi Roshani - Can you please make sure that this fix (which is already > >> merged to master) is also merged to the stable branch for 1.3.0: > >> https://github.com/apache/incubator-mxnet/pull/11493 - Thanks, Sina > >> > >> > >> On 8/16/18, 10:51 AM, "Roshani Nagmote" > <[email protected]> > >> wrote: > >> > >> Hi all, > >> > >> Release status: > >> > >> Currently, for release 1.3.0 there are a couple of issues open > >> which needs > >> to be resolved before cutting RC. > >> > >> The current date we are looking at for cutting RC0 is 08/17(Friday). > >> > >> > >> > >> Open issues which need to be looked at before cutting RC: > >> > >> 1. Topk regression issue > >> <https://github.com/apache/incubator-mxnet/issues/12197> - > >> #12202 PR > >> with fix <https://github.com/apache/incubator-mxnet/pull/12202> > >> 2. Excessive memory allocation issue > >> <https://github.com/apache/incubator-mxnet/issues/12116> - > >> #12184 PR > >> with fix <https://github.com/apache/incubator-mxnet/pull/12184> > >> 3. Test_io.test_csvIter breaks on CentOS > >> <https://github.com/apache/incubator-mxnet/issues/12139> - > >> #12189 PR > >> with fix > >> <https://github.com/apache/incubator-mxnet/pull/12189> > >> > >> > >> > >> @committers, could you please help review these PRs and get them > >> merged? > >> > >> > >> > >> Thanks, > >> > >> Roshani > >> > >> On Tue, Aug 14, 2018 at 12:46 PM Roshani Nagmote < > >> [email protected]> > >> wrote: > >> > >> > Talked to the person who ran resnet50 benchmarks offline. Build > >> flag was > >> > not properly set so there was a difference in performance > >> numbers observed. > >> > There is no issue caught and he was able to get the same results as > >> > mentioned here https://mxnet.incubator.apache.org/faq/perf.html > >> > <https://mxnet.incubator.apache.org/faq/perf.html#scoring-results> > >> > > >> > We are good here. > >> > > >> > Thanks, > >> > Roshani > >> > > >> > On Mon, Aug 13, 2018 at 4:08 PM Roshani Nagmote < > >> [email protected]> > >> > wrote: > >> > > >> >> Hi Dom, > >> >> > >> >> I verified resnet50 run on MXNet master branch. Checked on > >> single gpu > >> >> machine. Numbers match. I didn't see any performance degradation. > >> >> https://mxnet.incubator.apache.org/faq/perf.html#scoring-results > >> >> > >> >> Can you please give me more details on the instance type and > >> script you > >> >> ran exactly so that I can try to reproduce it again? > >> >> > >> >> Thanks, > >> >> Roshani > >> >> > >> >> > >> >> On Mon, Aug 13, 2018 at 12:31 PM Roshani Nagmote < > >> >> [email protected]> wrote: > >> >> > >> >>> This is not a major feature. I meant other new feature > >> requests PR won't > >> >>> be accepted in 1.3 release now. > >> >>> Bug fixes will be accepted. I will be trying to reproduce the > >> regression > >> >>> Dom mentioned today. :) > >> >>> > >> >>> Thanks, > >> >>> Roshani > >> >>> > >> >>> On Mon, Aug 13, 2018 at 12:06 PM Naveen Swamy > >> <[email protected] > >> > > >> >>> wrote: > >> >>> > >> >>>> Is this is a major feature? This is a regression that Dom is > >> reporting > >> >>>> wrt > >> >>>> to performance > >> >>>> > >> >>>> On Mon, Aug 13, 2018 at 11:38 AM, Roshani Nagmote < > >> >>>> [email protected] > >> >>>> > wrote: > >> >>>> > >> >>>> > Thanks for reporting this issue Dom. > >> >>>> > 08/10 (Frida)y was the major feature freeze date. We won't be > >> >>>> accepting any > >> >>>> > new features now for MXNet 1.3 release. > >> >>>> > RC0 will be cut on 08/17(Friday). > >> >>>> > > >> >>>> > Will be verifying the performance degradation issue mentioned. > >> >>>> > > >> >>>> > Thanks, > >> >>>> > Roshani > >> >>>> > > >> >>>> > On Mon, Aug 13, 2018 at 8:45 AM Divakaruni, Dominic > >> >>>> > <[email protected]> wrote: > >> >>>> > > >> >>>> > > Hi all, We tested resnet50 on MXNet built from master > >> branch on > >> >>>> Friday > >> >>>> > and > >> >>>> > > were seeing degraded performance on GPU - about 50% > >> slower compared > >> >>>> to > >> >>>> > > these values here > >> https://mxnet.incubator.apache.org/faq/perf.html. > >> >>>> FWIW > >> >>>> > > this slowdown was seen for both MXNet as well as the TRT > >> integrated > >> >>>> > MXNet. > >> >>>> > > > >> >>>> > > Something for you all to verify before or after you cut > >> the RC. > >> >>>> > > > >> >>>> > > Thx! > >> >>>> > > > >> >>>> > > On 8/13/18, 4:34 AM, "kellen sunderland" < > >> >>>> [email protected]> > >> >>>> > > wrote: > >> >>>> > > > >> >>>> > > Hey Roshani, > >> >>>> > > > >> >>>> > > Has a RC branch already been cut? If so, a quick heads > >> up that > >> >>>> I > >> >>>> > think > >> >>>> > > this commit should probably get into RC0 for 1.3. > >> >>>> > > > >> >>>> > > https://github.com/apache/incubator-mxnet/commit/ > >> >>>> > ee8755a2531b322fec29c9c3d2aa3b8738da41f3 > >> >>>> > > > >> >>>> > > It won't cause issues for users, but from a versioning > >> >>>> compatibility > >> >>>> > > perspective it's probably better that we remove these > >> functions > >> >>>> in > >> >>>> > this > >> >>>> > > release. This way we don't have to worry about major > >> bumps in > >> >>>> the > >> >>>> > next > >> >>>> > > release if they're removed. > >> >>>> > > > >> >>>> > > -Kellen > >> >>>> > > > >> >>>> > > > >> >>>> > > On Fri, Aug 10, 2018 at 7:24 PM Roshani Nagmote < > >> >>>> > > [email protected]> > >> >>>> > > wrote: > >> >>>> > > > >> >>>> > > > Thanks Kellen and everyone else for working to get > >> TensorRT PR > >> >>>> > > merged! > >> >>>> > > > @Sina, I will be keeping track of that issue and fixes > >> to get > >> >>>> in > >> >>>> > the > >> >>>> > > > release. > >> >>>> > > > > >> >>>> > > > We are starting code freeze for 1.3 release today. A > >> release > >> >>>> > > candidate will > >> >>>> > > > be cut on 08/17. > >> >>>> > > > Feel free to add any other comments/suggestions. > >> >>>> > > > > >> >>>> > > > Thanks, > >> >>>> > > > Roshani > >> >>>> > > > > >> >>>> > > > On Fri, Aug 10, 2018 at 5:39 AM kellen sunderland < > >> >>>> > > > [email protected]> wrote: > >> >>>> > > > > >> >>>> > > > > All merged and ready to go from my side Roshani (the > >> >>>> TensorRT > >> >>>> > PR). > >> >>>> > > > > > >> >>>> > > > > I agree with Sina that issue 12116 looks it's a > >> blocker. > >> >>>> I'll > >> >>>> > try > >> >>>> > > and > >> >>>> > > > > reproduce it locally to get another datapoint. > >> >>>> > > > > > >> >>>> > > > > On Fri, Aug 10, 2018 at 3:15 AM Afrooze, Sina < > >> >>>> > [email protected]> > >> >>>> > > > wrote: > >> >>>> > > > > > >> >>>> > > > > > Hi Roshani - I think this regression issue is a > >> release > >> >>>> > blocker: > >> >>>> > > > > > > >> https://github.com/apache/incubator-mxnet/issues/12116 > >> >>>> - Sina > >> >>>> > > > > > > >> >>>> > > > > > > >> >>>> > > > > > On 8/8/18, 12:40 PM, "Roshani Nagmote" < > >> >>>> > > [email protected]> > >> >>>> > > > > wrote: > >> >>>> > > > > > > >> >>>> > > > > > Thanks, Kellen for letting me know. > >> >>>> > > > > > > >> >>>> > > > > > On Wed, Aug 8, 2018 at 12:09 PM kellen > >> sunderland < > >> >>>> > > > > > [email protected]> wrote: > >> >>>> > > > > > > >> >>>> > > > > > > Hey Roshani, I think it should be ready by > >> Friday. > >> >>>> > > > > > > > >> >>>> > > > > > > On Tue, Aug 7, 2018, 10:20 PM Roshani > >> Nagmote < > >> >>>> > > > > > [email protected]> > >> >>>> > > > > > > wrote: > >> >>>> > > > > > > > >> >>>> > > > > > > > Thanks Kellen. Yes, we were treating this > >> PR as a > >> >>>> > release > >> >>>> > > > > blocker. > >> >>>> > > > > > Do you > >> >>>> > > > > > > > have any ETA by which it will be completed? > >> >>>> Approximate > >> >>>> > > time > >> >>>> > > > will > >> >>>> > > > > > also > >> >>>> > > > > > > > work. > >> >>>> > > > > > > > @zhi, Thanks for bringing this PR into > >> notice. I > >> >>>> will > >> >>>> > > keep a > >> >>>> > > > > track > >> >>>> > > > > > of it. > >> >>>> > > > > > > > > >> >>>> > > > > > > > -Roshani > >> >>>> > > > > > > > > >> >>>> > > > > > > > On Tue, Aug 7, 2018 at 11:30 AM Joshua Z. > >> Zhang < > >> >>>> > > > > > [email protected]> > >> >>>> > > > > > > > wrote: > >> >>>> > > > > > > > > >> >>>> > > > > > > > > I strongly suggest to track this PR > >> >>>> > > > > > > > > > >> >>>> https://github.com/apache/incubator-mxnet/pull/11908 > >> >>>> > < > >> >>>> > > > > > > > > > >> >>>> https://github.com/apache/incubator-mxnet/pull/11908 > >> >>>> > > > >> >>>> > > in 1.3 > >> >>>> > > > > > release > >> >>>> > > > > > > > > which fixed the usability issue for > >> lower end > >> >>>> > machines > >> >>>> > > that > >> >>>> > > > > > don’t have > >> >>>> > > > > > > as > >> >>>> > > > > > > > > large shared memory space as ec2 > >> instances. > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > Best, > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > - Zhi > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > > On Aug 7, 2018, at 9:05 AM, Roshani > >> Nagmote < > >> >>>> > > > > > > [email protected] > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > wrote: > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > Hi all, > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > Right now, we are delaying MXNet 1.3 > >> release > >> >>>> for > >> >>>> > > pending > >> >>>> > > > > > TensorRT PR > >> >>>> > > > > > > ( > >> >>>> > > > > > > > > > https://github.com/apache/ > >> >>>> > incubator-mxnet/pull/11325 > >> >>>> > > ). > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > I wanted to ask everyone for their > >> opinions > >> >>>> if we > >> >>>> > > should > >> >>>> > > > > delay > >> >>>> > > > > > the > >> >>>> > > > > > > > > release > >> >>>> > > > > > > > > > to get tensorRT integration in or we > >> should go > >> >>>> > ahead > >> >>>> > > with > >> >>>> > > > the > >> >>>> > > > > > release > >> >>>> > > > > > > > and > >> >>>> > > > > > > > > > include tensorRT in next release. > >> Please > >> >>>> provide > >> >>>> > > > suggestions. > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > Thanks, > >> >>>> > > > > > > > > > Roshani > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > On Mon, Aug 6, 2018 at 12:45 AM Hagay > >> Lupesko > >> >>>> < > >> >>>> > > > > > [email protected]> > >> >>>> > > > > > > > wrote: > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > >> Some thoughts: why not keep it out of > >> 1.3, > >> >>>> and > >> >>>> > > merge it > >> >>>> > > > into > >> >>>> > > > > > master > >> >>>> > > > > > > so > >> >>>> > > > > > > > > it > >> >>>> > > > > > > > > >> can go out with 1.4 instead? > >> >>>> > > > > > > > > >> Pros: > >> >>>> > > > > > > > > >> - Reduce quality risks for 1.3 > >> >>>> > > > > > > > > >> - More time to test and get feedback > >> before > >> >>>> > release > >> >>>> > > > > > > > > >> - Avoid further delays in 1.3 release > >> (lots > >> >>>> of > >> >>>> > good > >> >>>> > > stuff > >> >>>> > > > > > there > >> >>>> > > > > > > > already > >> >>>> > > > > > > > > for > >> >>>> > > > > > > > > >> users) > >> >>>> > > > > > > > > >> Cons: > >> >>>> > > > > > > > > >> - People will need to get master to > >> >>>> experiment > >> >>>> > with > >> >>>> > > TRT > >> >>>> > > > (not > >> >>>> > > > > > a major > >> >>>> > > > > > > > > issue > >> >>>> > > > > > > > > >> IMO) > >> >>>> > > > > > > > > >> > >> >>>> > > > > > > > > >> Besides, TRT requires a build flag > >> anyway, so > >> >>>> > MXNet > >> >>>> > > users > >> >>>> > > > > > consuming > >> >>>> > > > > > > > > built > >> >>>> > > > > > > > > >> packages (PyPi, Scala) will anyway > >> not be > >> >>>> able to > >> >>>> > > try it > >> >>>> > > > out > >> >>>> > > > > > unless > >> >>>> > > > > > > > > >> building from source... > >> >>>> > > > > > > > > >> > >> >>>> > > > > > > > > >> Thoughts? > >> >>>> > > > > > > > > >> > >> >>>> > > > > > > > > >> On Sun, Aug 5, 2018 at 10:38 PM > >> Steffen > >> >>>> Rochel < > >> >>>> > > > > > > > [email protected] > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > >> wrote: > >> >>>> > > > > > > > > >> > >> >>>> > > > > > > > > >>> Marek, Kellen, Jun, Da, Eric, myself > >> and a > >> >>>> few > >> >>>> > > other > >> >>>> > > > people > >> >>>> > > > > > > discussed > >> >>>> > > > > > > > > >>> offline about TensorRT integration > >> PR ( > >> >>>> > > > > > > > > >>> > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11325 ). > >> >>>> > > > We > >> >>>> > > > > > do > >> >>>> > > > > > > agree > >> >>>> > > > > > > > > that > >> >>>> > > > > > > > > >>> it > >> >>>> > > > > > > > > >>> would be good to include the PR into > >> >>>> upcoming 1.3 > >> >>>> > > > release, > >> >>>> > > > > > but are > >> >>>> > > > > > > > all > >> >>>> > > > > > > > > >>> concerned about the risk involved > >> and the > >> >>>> > breaking > >> >>>> > > API > >> >>>> > > > > > change. The > >> >>>> > > > > > > > > >>> discussion converged to following > >> proposal. > >> >>>> (1) > >> >>>> > > change to > >> >>>> > > > > > contrib > >> >>>> > > > > > > PR > >> >>>> > > > > > > > > and > >> >>>> > > > > > > > > >>> (2) define a different top level API > >> to > >> >>>> indicate > >> >>>> > > that the > >> >>>> > > > > > package > >> >>>> > > > > > > is > >> >>>> > > > > > > > > part > >> >>>> > > > > > > > > >>> of contrib and experimental (details > >> of API > >> >>>> TBD > >> >>>> > > between > >> >>>> > > > > > Marek, > >> >>>> > > > > > > Kellen > >> >>>> > > > > > > > > and > >> >>>> > > > > > > > > >>> Eric). This change would allow to > >> include > >> >>>> TRT > >> >>>> > > integration > >> >>>> > > > > > with v1.3 > >> >>>> > > > > > > > to > >> >>>> > > > > > > > > >>> enable users to try TRT with MXNet, > >> >>>> minimize the > >> >>>> > > risk and > >> >>>> > > > > > avoid > >> >>>> > > > > > > > > breaking > >> >>>> > > > > > > > > >>> API change. > >> >>>> > > > > > > > > >>> To accommodate the change the > >> request is to > >> >>>> delay > >> >>>> > > RC for > >> >>>> > > > a > >> >>>> > > > > > few > >> >>>> > > > > > > days. > >> >>>> > > > > > > > > >>> > >> >>>> > > > > > > > > >>> Regards, > >> >>>> > > > > > > > > >>> Steffen > >> >>>> > > > > > > > > >>> > >> >>>> > > > > > > > > >>> On Tue, Jul 31, 2018 at 5:08 PM > >> Roshani > >> >>>> Nagmote < > >> >>>> > > > > > > > > >> [email protected] > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>> wrote: > >> >>>> > > > > > > > > >>> > >> >>>> > > > > > > > > >>>> Hi, > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>>> I have created a wiki for tracking > >> MXNet > >> >>>> 1.3 > >> >>>> > > release > >> >>>> > > > with > >> >>>> > > > > > the > >> >>>> > > > > > > > > timeline. > >> >>>> > > > > > > > > >>>> Please take a look here: > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>> > >> >>>> > > > > > > > > >> > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > >> >>>> > > > > > >> >>>> > > > > >> >>>> > > https://cwiki.apache.org/confluence/display/MXNET/ > >> >>>> > Apache+MXNet+%28incubating%29+1.3.0+Release+Status > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>>> I am still waiting for following 2 > >> PRs to > >> >>>> get > >> >>>> > > merged: > >> >>>> > > > > > > > > >>>> TRT integration: > >> >>>> > > > > > > > > >> >>>> https://github.com/apache/incubator-mxnet/pull/11325 > >> >>>> > > > > > > > > >>>> Gluon RNN: > >> >>>> > > > > > > >> https://github.com/apache/incubator-mxnet/pull/11482 > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>>> *Code freeze date is > >> 08/02(Thursday).* > >> >>>> Kindly > >> >>>> > try > >> >>>> > > to > >> >>>> > > > > > complete > >> >>>> > > > > > > > ongoing > >> >>>> > > > > > > > > >>> work > >> >>>> > > > > > > > > >>>> and get these PRs merged. > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>>> Thanks, > >> >>>> > > > > > > > > >>>> Roshani > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>>> On Mon, Jul 30, 2018 at 1:02 PM > >> Roshani > >> >>>> Nagmote > >> >>>> > < > >> >>>> > > > > > > > > >>> [email protected] > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>> wrote: > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>>>> Hi all, > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> Here is an update on MXNet 1.3 > >> release: > >> >>>> > > > > > > > > >>>>> I am still waiting for following > >> PRs to > >> >>>> get > >> >>>> > > merged: > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> TRT integration: > >> >>>> > > > > > > > > >> > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11325 > >> >>>> > > > > > > > > >>>>> Gluon RNN: > >> >>>> > > > > > > >> https://github.com/apache/incubator-mxnet/pull/11482 > >> >>>> > > > > > > > > >>>>> Scala examples: > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11753 > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11621 > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> *New code freeze date is: 08/03* > >> Please > >> >>>> try to > >> >>>> > > get > >> >>>> > > > your > >> >>>> > > > > > ongoing > >> >>>> > > > > > > > PRs > >> >>>> > > > > > > > > >>>>> merged by then. > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> @Pedro, I didn't include your PRs > >> in > >> >>>> tracking > >> >>>> > > list as > >> >>>> > > > you > >> >>>> > > > > > said > >> >>>> > > > > > > > those > >> >>>> > > > > > > > > >>> are > >> >>>> > > > > > > > > >>>>> not critical for now. Please let > >> me know > >> >>>> if > >> >>>> > > those needs > >> >>>> > > > > to > >> >>>> > > > > > be > >> >>>> > > > > > > > > >> included. > >> >>>> > > > > > > > > >>>>> > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11636 > >> >>>> > > > > > > > > >>>>> > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11562 > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> I also have updated project > >> proposal > >> >>>> cwiki page > >> >>>> > > to > >> >>>> > > > update > >> >>>> > > > > > the > >> >>>> > > > > > > > status > >> >>>> > > > > > > > > >> of > >> >>>> > > > > > > > > >>>>> PRs. > >> >>>> > > > > > > > > >>>>> < > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>> > >> >>>> > > > > > > > > >> > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > >> >>>> > > > > > >> >>>> > > > > >> >>>> > > https://cwiki.apache.org/confluence/display/MXNET/ > >> >>>> > Project+Proposals+for+next+MXNet+Release > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> Please let me know if I am missing > >> >>>> something. > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> Thanks, > >> >>>> > > > > > > > > >>>>> Roshani > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>> On Thu, Jul 26, 2018 at 1:34 PM > >> Pedro > >> >>>> Larroy < > >> >>>> > > > > > > > > >>>> [email protected]> > >> >>>> > > > > > > > > >>>>> wrote: > >> >>>> > > > > > > > > >>>>> > >> >>>> > > > > > > > > >>>>>> I would like to get these PR > >> merged: > >> >>>> > > > > > > > > >>>>>> > >> >>>> > > > > > > > > >>>>>> > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11636 > >> >>>> > > > > > > > > >>>>>> > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11562 > >> >>>> > > > > > > > > >>>>>> > >> >>>> > > > > > > > > >>>>>> How much longer until the code > >> freeze? > >> >>>> > > > > > > > > >>>>>> > >> >>>> > > > > > > > > >>>>>> On Thu, Jul 26, 2018 at 1:44 AM > >> Roshani > >> >>>> > Nagmote > >> >>>> > > < > >> >>>> > > > > > > > > >>>>>> [email protected]> > >> >>>> > > > > > > > > >>>>>> wrote: > >> >>>> > > > > > > > > >>>>>> > >> >>>> > > > > > > > > >>>>>>> Hi all, > >> >>>> > > > > > > > > >>>>>>> > >> >>>> > > > > > > > > >>>>>>> PRs waiting to be merged for 1.3 > >> >>>> release: > >> >>>> > > > > > > > > >>>>>>> > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11325 > >> >>>> > > > > > > > > >>>>>>> > >> >>>> > > > > > > > > >>>>>>> Are there any other PRs waiting > >> to get > >> >>>> > merged? > >> >>>> > > Please > >> >>>> > > > > > let me > >> >>>> > > > > > > > know. > >> >>>> > > > > > > > > >>>>>>> > >> >>>> > > > > > > > > >>>>>>> Release blocker issue: > >> >>>> > > > > > > > > >>>>>>> > >> >>>> > > > https://github.com/apache/incubator-mxnet/issues/11853 > >> >>>> > > > > > > > > >>>>>>> > >> >>>> > > > > > > > > >>>>>>> @Marco, @Kellen, Thanks for > >> bringing up > >> >>>> the > >> >>>> > > important > >> >>>> > > > > > topic. I > >> >>>> > > > > > > > > >> agree > >> >>>> > > > > > > > > >>>>>> with > >> >>>> > > > > > > > > >>>>>>> you and we(internal Amazon team) > >> will be > >> >>>> > > working on > >> >>>> > > > > > fixing the > >> >>>> > > > > > > > > >>>> disabled > >> >>>> > > > > > > > > >>>>>>> tests. > >> >>>> > > > > > > > > >>>>>>> Currently, my colleague, Hao Jin > >> is > >> >>>> working > >> >>>> > on > >> >>>> > > > > compiling > >> >>>> > > > > > the > >> >>>> > > > > > > list > >> >>>> > > > > > > > > >> of > >> >>>> > > > > > > > > >>>>>>> disabled tests and leading the > >> effort > >> >>>> to fix > >> >>>> > > them in > >> >>>> > > > > the > >> >>>> > > > > > next > >> >>>> > > > > > > few > >> >>>> > > > > > > > > >>>> days. > >> >>>> > > > > > > > > >>>>>>> > >> >>>> > > > > > > > > >>>>>>> Thanks, > >> >>>> > > > > > > > > >>>>>>> Roshani > >> >>>> > > > > > > > > >>>>>>> > >> >>>> > > > > > > > > >>>>>>> On Mon, Jul 23, 2018 at 6:39 PM > >> kellen > >> >>>> > > sunderland < > >> >>>> > > > > > > > > >>>>>>> [email protected]> > >> wrote: > >> >>>> > > > > > > > > >>>>>>> > >> >>>> > > > > > > > > >>>>>>>> Thanks again for organizing > >> Roshani. I > >> >>>> > > believe the > >> >>>> > > > > > TensorRT > >> >>>> > > > > > > > > >> work > >> >>>> > > > > > > > > >>> is > >> >>>> > > > > > > > > >>>>>>> ready > >> >>>> > > > > > > > > >>>>>>>> for a merge. Thanks to Marek > >> and all > >> >>>> the > >> >>>> > > NVIDIA > >> >>>> > > > > people > >> >>>> > > > > > for > >> >>>> > > > > > > > > >>>> iterating > >> >>>> > > > > > > > > >>>>>> on > >> >>>> > > > > > > > > >>>>>>>> it. If possible could a > >> committer > >> >>>> review, > >> >>>> > > make sure > >> >>>> > > > > it > >> >>>> > > > > > meets > >> >>>> > > > > > > > > >>> their > >> >>>> > > > > > > > > >>>>>>>> expectations and then merge? > >> PR is > >> >>>> here: > >> >>>> > > > > > > > > >>>>>>>> > >> >>>> > > > https://github.com/apache/incubator-mxnet/pull/11325 > >> >>>> > > > > > > > > >>>>>>>> > >> >>>> > > > > > > > > >>>>>>>> To Marco's point. I'd > >> recommend we > >> >>>> review > >> >>>> > > some of > >> >>>> > > > > those > >> >>>> > > > > > > > > >> disabled > >> >>>> > > > > > > > > >>>>>> tests > >> >>>> > > > > > > > > >>>>>>> and > >> >>>> > > > > > > > > >>>>>>>> see how likely they are to > >> affect users > >> >>>> > > before we > >> >>>> > > > cut > >> >>>> > > > > a > >> >>>> > > > > > > release. > >> >>>> > > > > > > > > >>>>>> Many of > >> >>>> > > > > > > > > >>>>>>>> them are obviously not too > >> important > >> >>>> from a > >> >>>> > > user's > >> >>>> > > > > > point of > >> >>>> > > > > > > view > >> >>>> > > > > > > > > >>>> (e.g. > >> >>>> > > > > > > > > >>>>>>>> downloading a sometimes-offline > >> image > >> >>>> in a > >> >>>> > > test). > >> >>>> > > > One > >> >>>> > > > > > idea > >> >>>> > > > > > > > > >> would > >> >>>> > > > > > > > > >>> be > >> >>>> > > > > > > > > >>>>>> to > >> >>>> > > > > > > > > >>>>>>> try > >> >>>> > > > > > > > > >>>>>>>> and address as many of the > >> customer > >> >>>> > impacting > >> >>>> > > issues > >> >>>> > > > > as > >> >>>> > > > > > > possible > >> >>>> > > > > > > > > >>>>>> between > >> >>>> > > > > > > > > >>>>>>>> code freeze and the RC0 vote. > >> >>>> > > > > > > > > >>>>>>>> > >> >>>> > > > > > > > > >>>>>>>> On Mon, Jul 23, 2018 at 1:23 PM > >> Marco > >> >>>> de > >> >>>> > Abreu > >> >>>> > > > > > > > > >>>>>>>> <[email protected] > >> .invalid> > >> >>>> > wrote: > >> >>>> > > > > > > > > >>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>> Hello Roshani, > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>> frequent releases are good and > >> I'm > >> >>>> > > supportive for > >> >>>> > > > > this > >> >>>> > > > > > in > >> >>>> > > > > > > > > >>> general > >> >>>> > > > > > > > > >>>> in > >> >>>> > > > > > > > > >>>>>>>> order > >> >>>> > > > > > > > > >>>>>>>>> to provide our users with the > >> latest > >> >>>> > > features and > >> >>>> > > > > > > > > >> improvements. > >> >>>> > > > > > > > > >>>> But > >> >>>> > > > > > > > > >>>>>> at > >> >>>> > > > > > > > > >>>>>>>> the > >> >>>> > > > > > > > > >>>>>>>>> moment, I'm slightly concerned > >> about > >> >>>> the > >> >>>> > test > >> >>>> > > > > coverage > >> >>>> > > > > > due to > >> >>>> > > > > > > > > >>>> [1]. I > >> >>>> > > > > > > > > >>>>>>> want > >> >>>> > > > > > > > > >>>>>>>>> us to be conscious about > >> cutting a > >> >>>> release > >> >>>> > > even > >> >>>> > > > > though > >> >>>> > > > > > not > >> >>>> > > > > > > all > >> >>>> > > > > > > > > >>>> tests > >> >>>> > > > > > > > > >>>>>>> are > >> >>>> > > > > > > > > >>>>>>>>> enabled (29 disabled tests [2] > >> as of > >> >>>> > today). > >> >>>> > > > > However, I > >> >>>> > > > > > > > > >>>> acknowledge > >> >>>> > > > > > > > > >>>>>>> that > >> >>>> > > > > > > > > >>>>>>>> we > >> >>>> > > > > > > > > >>>>>>>>> have improved by a lot lately > >> thanks > >> >>>> to > >> >>>> > > everybody > >> >>>> > > > > > > > > >> participating > >> >>>> > > > > > > > > >>>> and > >> >>>> > > > > > > > > >>>>>>>> leading > >> >>>> > > > > > > > > >>>>>>>>> the efforts around improving > >> flaky > >> >>>> tests. > >> >>>> > > From a > >> >>>> > > > > > > retrospective > >> >>>> > > > > > > > > >>>>>> point of > >> >>>> > > > > > > > > >>>>>>>>> view, we could say that these > >> efforts > >> >>>> have > >> >>>> > > actually > >> >>>> > > > > > revealed > >> >>>> > > > > > > > > >>> some > >> >>>> > > > > > > > > >>>>>> quite > >> >>>> > > > > > > > > >>>>>>>>> interesting bugs and thus the > >> time > >> >>>> was well > >> >>>> > > spent > >> >>>> > > > and > >> >>>> > > > > > yielded > >> >>>> > > > > > > > > >>> good > >> >>>> > > > > > > > > >>>>>>>> results. > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>> What does the community think > >> about > >> >>>> making > >> >>>> > > another > >> >>>> > > > > > sprint of > >> >>>> > > > > > > > > >>>>>>> improvements > >> >>>> > > > > > > > > >>>>>>>>> around tests followed up by a > >> period > >> >>>> of 1-2 > >> >>>> > > weeks > >> >>>> > > > > > during > >> >>>> > > > > > > which > >> >>>> > > > > > > > > >>> we > >> >>>> > > > > > > > > >>>>>>> observe > >> >>>> > > > > > > > > >>>>>>>>> the failures closely to ensure > >> that no > >> >>>> > > critical > >> >>>> > > > paths > >> >>>> > > > > > are > >> >>>> > > > > > > > > >>>> impacted? > >> >>>> > > > > > > > > >>>>>> If > >> >>>> > > > > > > > > >>>>>>> we > >> >>>> > > > > > > > > >>>>>>>>> are in a good shape by then, > >> we could > >> >>>> > > continue the > >> >>>> > > > > > release > >> >>>> > > > > > > > > >>> process > >> >>>> > > > > > > > > >>>>>> and > >> >>>> > > > > > > > > >>>>>>> at > >> >>>> > > > > > > > > >>>>>>>>> the same time have the > >> advantage of > >> >>>> giving > >> >>>> > > > > > contributors more > >> >>>> > > > > > > > > >>> lead > >> >>>> > > > > > > > > >>>>>> time > >> >>>> > > > > > > > > >>>>>>> to > >> >>>> > > > > > > > > >>>>>>>>> finish their work to ensure it > >> gets > >> >>>> into > >> >>>> > the > >> >>>> > > > release > >> >>>> > > > > > in the > >> >>>> > > > > > > > > >>>> desired > >> >>>> > > > > > > > > >>>>>>>>> quality. > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>> Again, thanks to everybody for > >> their > >> >>>> > efforts > >> >>>> > > during > >> >>>> > > > > > the last > >> >>>> > > > > > > > > >>> weeks > >> >>>> > > > > > > > > >>>>>> to > >> >>>> > > > > > > > > >>>>>>>>> improve the usability and > >> stability of > >> >>>> > > MXNet. This > >> >>>> > > > is > >> >>>> > > > > > great > >> >>>> > > > > > > > > >>>>>> community > >> >>>> > > > > > > > > >>>>>>>>> effort and a good example of a > >> >>>> community > >> >>>> > > working > >> >>>> > > > > > together > >> >>>> > > > > > > > > >>> towards > >> >>>> > > > > > > > > >>>> a > >> >>>> > > > > > > > > >>>>>>>> unified > >> >>>> > > > > > > > > >>>>>>>>> goal! > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>> Best regards, > >> >>>> > > > > > > > > >>>>>>>>> Marco > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>> [1]: > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>> > >> >>>> > > > > > > > > >>>>>>> > >> >>>> > > > > > > > > >>>>>> > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>> > >> >>>> > > > > > > > > >> > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > >> >>>> > > > > > >> >>>> > > > > >> >>>> > > > >> https://lists.apache.org/thread.html/d6d81401de796a96677a112d6cd0b0 > >> >>>> > 74b01f46564194ea89b86c6a3e@%3Cdev.mxnet.apache.org%3E > >> >>>> > > > > > > > > >>>>>>>>> [2]: > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>> > >> >>>> > > > > > > > > >>>>>>> > >> >>>> > > > > > > > > >>>>>> > >> >>>> > > > > > > > > >>>> > >> >>>> > > > > > > > > >>> > >> >>>> > > > > > > > > >> > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > >> >>>> > > > > > >> >>>> > > > > >> >>>> > > https://github.com/apache/incubator-mxnet/issues?q=is% > >> >>>> > 3Aopen+is%3Aissue+label%3A%22Disabled+test%22 > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>> On Mon, Jul 23, 2018 at 8:34 PM > >> >>>> Roshani > >> >>>> > > Nagmote < > >> >>>> > > > > > > > > >>>>>>>> [email protected] > >> >>>> > > > > > > > > >>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>> wrote: > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>> Hi all, > >> >>>> > > > > > > > > >>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>> As mentioned before, code > >> freeze > >> >>>> date is > >> >>>> > > today > >> >>>> > > > July > >> >>>> > > > > > 23rd. > >> >>>> > > > > > > > > >>> Please > >> >>>> > > > > > > > > >>>>>> try > >> >>>> > > > > > > > > >>>>>>> to > >> >>>> > > > > > > > > >>>>>>>>> get > >> >>>> > > > > > > > > >>>>>>>>>> your ongoing PRs merged by > >> today. > >> >>>> > > > > > > > > >>>>>>>>>> Please let me know if there > >> are any > >> >>>> > > concerns or > >> >>>> > > > need > >> >>>> > > > > > more > >> >>>> > > > > > > > > >>> time. > >> >>>> > > > > > > > > >>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>> Thanks, > >> >>>> > > > > > > > > >>>>>>>>>> Roshani > >> >>>> > > > > > > > > >>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>> On Thu, Jul 19, 2018 at 1:16 > >> PM > >> >>>> Anirudh > >> >>>> > > Acharya < > >> >>>> > > > > > > > > >>>>>>> [email protected] > >> >>>> > > > > > > > > >>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>> wrote: > >> >>>> > > > > > > > > >>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>>> @sandeep krishnamurthy < > >> >>>> > > > > [email protected]> > >> >>>> > > > > > the > >> >>>> > > > > > > > > >>> bug > >> >>>> > > > > > > > > >>>>>> fixes > >> >>>> > > > > > > > > >>>>>>>> in > >> >>>> > > > > > > > > >>>>>>>>>> the > >> >>>> > > > > > > > > >>>>>>>>>>> R-package is something we > >> have just > >> >>>> > begun, > >> >>>> > > there > >> >>>> > > > > > will not > >> >>>> > > > > > > > > >> be > >> >>>> > > > > > > > > >>>>>>> anything > >> >>>> > > > > > > > > >>>>>>>>>>> significant to announce > >> before the > >> >>>> v1.3 > >> >>>> > > code > >> >>>> > > > > freeze. > >> >>>> > > > > > > > > >>>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>>> On Wed, Jul 18, 2018 at > >> 10:08 PM > >> >>>> Steffen > >> >>>> > > Rochel < > >> >>>> > > > > > > > > >>>>>>>>> [email protected] > >> >>>> > > > > > > > > >>>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>>> wrote: > >> >>>> > > > > > > > > >>>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>>>> To make it easier to find > >> the > >> >>>> > discussions > >> >>>> > > > related > >> >>>> > > > > to > >> >>>> > > > > > > > > >>> project > >> >>>> > > > > > > > > >>>>>>>>> proposals > >> >>>> > > > > > > > > >>>>>>>>>> I > >> >>>> > > > > > > > > >>>>>>>>>>>> added a column with a link > >> to the > >> >>>> thread > >> >>>> > > on dev@ > >> >>>> > > > > > for > >> >>>> > > > > > > > > >> most > >> >>>> > > > > > > > > >>>>>>>> projects. > >> >>>> > > > > > > > > >>>>>>>>>>>> Appreciate for the project > >> owners > >> >>>> to > >> >>>> > fill > >> >>>> > > in the > >> >>>> > > > > > blanks > >> >>>> > > > > > > > > >>> and > >> >>>> > > > > > > > > >>>> to > >> >>>> > > > > > > > > >>>>>>>> check > >> >>>> > > > > > > > > >>>>>>>>>>> that I > >> >>>> > > > > > > > > >>>>>>>>>>>> got the right threads. > >> >>>> > > > > > > > > >>>>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>>>> Regards, > >> >>>> > > > > > > > > >>>>>>>>>>>> Steffen > >> >>>> > > > > > > > > >>>>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>>>> On Wed, Jul 18, 2018 at > >> 7:11 PM > >> >>>> Roshani > >> >>>> > > Nagmote > >> >>>> > > > < > >> >>>> > > > > > > > > >>>>>>>>>>> [email protected] > >> >>>> > > > > > > > > >>>>>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>>>> wrote: > >> >>>> > > > > > > > > >>>>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>>>>> Hi Kellen, > >> >>>> > > > > > > > > >>>>>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>>>>> Sure. I will update the > >> notes > >> >>>> with the > >> >>>> > > > > information. > >> >>>> > > > > > > > > >>>>>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>>>>> Thanks, > >> >>>> > > > > > > > > >>>>>>>>>>>>> Roshani > >> >>>> > > > > > > > > >>>>>>>>>>>>> > >> >>>> > > > > > > > > >>>>>>>>>>>>> On Wed, Jul 18, 2018 at > >> 3:01 PM > >> > > > >
