Thanks Patric for reviewing the notes. Updated the doc with MKL-DNN points you mentioned accordingly.
Regards, Roshani On Tue, Aug 21, 2018 at 8:03 PM Zhao, Patric <[email protected]> wrote: > Hi Roshani, > > Good notes :) > > Several items about the performance and MKL-DNN in the below, please help > take a review. > > @Da, Alex, if anything about MKL-DNN is missed, feel free to add. > > *Performance improvement > +Support for dot(dns, csr) = dns and dot(dns, csr.T) = dns on CPU > https://github.com/apache/incubator-mxnet/pull/11113 > +Performance improvement for Batch Dot on CPU from mshadow > https://github.com/dmlc/mshadow/pull/342 > -Fix the topk regression issue (#12197) > This is the bugfix rather than performance improvements > > > *MKL-DNN > More functionality supports: > +Support more activation functions, "sigmoid", "tanh", "softrelu" > https://github.com/apache/incubator-mxnet/pull/10336 > > Debugging functionality: > +Result check > https://github.com/apache/incubator-mxnet/pull/12069 > +Backend switch > https://github.com/apache/incubator-mxnet/pull/12058 > > Thanks, > > --Patric > > > -----Original Message----- > > From: Roshani Nagmote [mailto:[email protected]] > > Sent: Wednesday, August 22, 2018 1:53 AM > > To: [email protected] > > Subject: Re: Release plan - MXNET 1.3 > > > > Hi, > > > > Thank you everyone for helping to clear release blockers. CI tests were > failing > > so we delayed RC by some time. But now the tests are passing and we are > > ready to cut the release branch. > > > > I have drafted release notes here: > > https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+%28in > > cubating%29+1.3.0+Release+Notes > > > > > > Please take a look and update if I have missed anything. I will be > cutting > > RC0 tomorrow. > > > > Thanks, > > Roshani > > > > On Thu, Aug 16, 2018 at 2:28 PM Roshani Nagmote > > <[email protected]> > > wrote: > > > > > Sure will do. thanks. > > > > > > -Roshani > > > > > > On Thu, Aug 16, 2018 at 11:53 AM Afrooze, Sina <[email protected]> > > wrote: > > > > > >> Hi Roshani - Can you please make sure that this fix (which is already > > >> merged to master) is also merged to the stable branch for 1.3.0: > > >> https://github.com/apache/incubator-mxnet/pull/11493 - Thanks, Sina > > >> > > >> > > >> On 8/16/18, 10:51 AM, "Roshani Nagmote" > > <[email protected]> > > >> wrote: > > >> > > >> Hi all, > > >> > > >> Release status: > > >> > > >> Currently, for release 1.3.0 there are a couple of issues open > > >> which needs > > >> to be resolved before cutting RC. > > >> > > >> The current date we are looking at for cutting RC0 is > 08/17(Friday). > > >> > > >> > > >> > > >> Open issues which need to be looked at before cutting RC: > > >> > > >> 1. Topk regression issue > > >> <https://github.com/apache/incubator-mxnet/issues/12197> - > > >> #12202 PR > > >> with fix <https://github.com/apache/incubator-mxnet/pull/12202 > > > > >> 2. Excessive memory allocation issue > > >> <https://github.com/apache/incubator-mxnet/issues/12116> - > > >> #12184 PR > > >> with fix <https://github.com/apache/incubator-mxnet/pull/12184 > > > > >> 3. Test_io.test_csvIter breaks on CentOS > > >> <https://github.com/apache/incubator-mxnet/issues/12139> - > > >> #12189 PR > > >> with fix > > >> <https://github.com/apache/incubator-mxnet/pull/12189> > > >> > > >> > > >> > > >> @committers, could you please help review these PRs and get them > > >> merged? > > >> > > >> > > >> > > >> Thanks, > > >> > > >> Roshani > > >> > > >> On Tue, Aug 14, 2018 at 12:46 PM Roshani Nagmote < > > >> [email protected]> > > >> wrote: > > >> > > >> > Talked to the person who ran resnet50 benchmarks offline. Build > > >> flag was > > >> > not properly set so there was a difference in performance > > >> numbers observed. > > >> > There is no issue caught and he was able to get the same > results as > > >> > mentioned here https://mxnet.incubator.apache.org/faq/perf.html > > >> > < > https://mxnet.incubator.apache.org/faq/perf.html#scoring-results> > > >> > > > >> > We are good here. > > >> > > > >> > Thanks, > > >> > Roshani > > >> > > > >> > On Mon, Aug 13, 2018 at 4:08 PM Roshani Nagmote < > > >> [email protected]> > > >> > wrote: > > >> > > > >> >> Hi Dom, > > >> >> > > >> >> I verified resnet50 run on MXNet master branch. Checked on > > >> single gpu > > >> >> machine. Numbers match. I didn't see any performance > degradation. > > >> >> > https://mxnet.incubator.apache.org/faq/perf.html#scoring-results > > >> >> > > >> >> Can you please give me more details on the instance type and > > >> script you > > >> >> ran exactly so that I can try to reproduce it again? > > >> >> > > >> >> Thanks, > > >> >> Roshani > > >> >> > > >> >> > > >> >> On Mon, Aug 13, 2018 at 12:31 PM Roshani Nagmote < > > >> >> [email protected]> wrote: > > >> >> > > >> >>> This is not a major feature. I meant other new feature > > >> requests PR won't > > >> >>> be accepted in 1.3 release now. > > >> >>> Bug fixes will be accepted. I will be trying to reproduce the > > >> regression > > >> >>> Dom mentioned today. :) > > >> >>> > > >> >>> Thanks, > > >> >>> Roshani > > >> >>> > > >> >>> On Mon, Aug 13, 2018 at 12:06 PM Naveen Swamy > > >> <[email protected] > > >> > > > >> >>> wrote: > > >> >>> > > >> >>>> Is this is a major feature? This is a regression that Dom is > > >> reporting > > >> >>>> wrt > > >> >>>> to performance > > >> >>>> > > >> >>>> On Mon, Aug 13, 2018 at 11:38 AM, Roshani Nagmote < > > >> >>>> [email protected] > > >> >>>> > wrote: > > >> >>>> > > >> >>>> > Thanks for reporting this issue Dom. > > >> >>>> > 08/10 (Frida)y was the major feature freeze date. We won't > be > > >> >>>> accepting any > > >> >>>> > new features now for MXNet 1.3 release. > > >> >>>> > RC0 will be cut on 08/17(Friday). > > >> >>>> > > > >> >>>> > Will be verifying the performance degradation issue > mentioned. > > >> >>>> > > > >> >>>> > Thanks, > > >> >>>> > Roshani > > >> >>>> > > > >> >>>> > On Mon, Aug 13, 2018 at 8:45 AM Divakaruni, Dominic > > >> >>>> > <[email protected]> wrote: > > >> >>>> > > > >> >>>> > > Hi all, We tested resnet50 on MXNet built from master > > >> branch on > > >> >>>> Friday > > >> >>>> > and > > >> >>>> > > were seeing degraded performance on GPU - about 50% > > >> slower compared > > >> >>>> to > > >> >>>> > > these values here > > >> https://mxnet.incubator.apache.org/faq/perf.html. > > >> >>>> FWIW > > >> >>>> > > this slowdown was seen for both MXNet as well as the TRT > > >> integrated > > >> >>>> > MXNet. > > >> >>>> > > > > >> >>>> > > Something for you all to verify before or after you cut > > >> the RC. > > >> >>>> > > > > >> >>>> > > Thx! > > >> >>>> > > > > >> >>>> > > On 8/13/18, 4:34 AM, "kellen sunderland" < > > >> >>>> [email protected]> > > >> >>>> > > wrote: > > >> >>>> > > > > >> >>>> > > Hey Roshani, > > >> >>>> > > > > >> >>>> > > Has a RC branch already been cut? If so, a quick > heads > > >> up that > > >> >>>> I > > >> >>>> > think > > >> >>>> > > this commit should probably get into RC0 for 1.3. > > >> >>>> > > > > >> >>>> > > https://github.com/apache/incubator-mxnet/commit/ > > >> >>>> > ee8755a2531b322fec29c9c3d2aa3b8738da41f3 > > >> >>>> > > > > >> >>>> > > It won't cause issues for users, but from a > versioning > > >> >>>> compatibility > > >> >>>> > > perspective it's probably better that we remove these > > >> functions > > >> >>>> in > > >> >>>> > this > > >> >>>> > > release. This way we don't have to worry about major > > >> bumps in > > >> >>>> the > > >> >>>> > next > > >> >>>> > > release if they're removed. > > >> >>>> > > > > >> >>>> > > -Kellen > > >> >>>> > > > > >> >>>> > > > > >> >>>> > > On Fri, Aug 10, 2018 at 7:24 PM Roshani Nagmote < > > >> >>>> > > [email protected]> > > >> >>>> > > wrote: > > >> >>>> > > > > >> >>>> > > > Thanks Kellen and everyone else for working to get > > >> TensorRT PR > > >> >>>> > > merged! > > >> >>>> > > > @Sina, I will be keeping track of that issue and > fixes > > >> to get > > >> >>>> in > > >> >>>> > the > > >> >>>> > > > release. > > >> >>>> > > > > > >> >>>> > > > We are starting code freeze for 1.3 release today. > A > > >> release > > >> >>>> > > candidate will > > >> >>>> > > > be cut on 08/17. > > >> >>>> > > > Feel free to add any other comments/suggestions. > > >> >>>> > > > > > >> >>>> > > > Thanks, > > >> >>>> > > > Roshani > > >> >>>> > > > > > >> >>>> > > > On Fri, Aug 10, 2018 at 5:39 AM kellen sunderland < > > >> >>>> > > > [email protected]> wrote: > > >> >>>> > > > > > >> >>>> > > > > All merged and ready to go from my side Roshani > (the > > >> >>>> TensorRT > > >> >>>> > PR). > > >> >>>> > > > > > > >> >>>> > > > > I agree with Sina that issue 12116 looks it's a > > >> blocker. > > >> >>>> I'll > > >> >>>> > try > > >> >>>> > > and > > >> >>>> > > > > reproduce it locally to get another datapoint. > > >> >>>> > > > > > > >> >>>> > > > > On Fri, Aug 10, 2018 at 3:15 AM Afrooze, Sina < > > >> >>>> > [email protected]> > > >> >>>> > > > wrote: > > >> >>>> > > > > > > >> >>>> > > > > > Hi Roshani - I think this regression issue is a > > >> release > > >> >>>> > blocker: > > >> >>>> > > > > > > > >> https://github.com/apache/incubator-mxnet/issues/12116 > > >> >>>> - Sina > > >> >>>> > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > On 8/8/18, 12:40 PM, "Roshani Nagmote" < > > >> >>>> > > [email protected]> > > >> >>>> > > > > wrote: > > >> >>>> > > > > > > > >> >>>> > > > > > Thanks, Kellen for letting me know. > > >> >>>> > > > > > > > >> >>>> > > > > > On Wed, Aug 8, 2018 at 12:09 PM kellen > > >> sunderland < > > >> >>>> > > > > > [email protected]> wrote: > > >> >>>> > > > > > > > >> >>>> > > > > > > Hey Roshani, I think it should be ready > by > > >> Friday. > > >> >>>> > > > > > > > > >> >>>> > > > > > > On Tue, Aug 7, 2018, 10:20 PM Roshani > > >> Nagmote < > > >> >>>> > > > > > [email protected]> > > >> >>>> > > > > > > wrote: > > >> >>>> > > > > > > > > >> >>>> > > > > > > > Thanks Kellen. Yes, we were treating > this > > >> PR as a > > >> >>>> > release > > >> >>>> > > > > blocker. > > >> >>>> > > > > > Do you > > >> >>>> > > > > > > > have any ETA by which it will be > completed? > > >> >>>> Approximate > > >> >>>> > > time > > >> >>>> > > > will > > >> >>>> > > > > > also > > >> >>>> > > > > > > > work. > > >> >>>> > > > > > > > @zhi, Thanks for bringing this PR into > > >> notice. I > > >> >>>> will > > >> >>>> > > keep a > > >> >>>> > > > > track > > >> >>>> > > > > > of it. > > >> >>>> > > > > > > > > > >> >>>> > > > > > > > -Roshani > > >> >>>> > > > > > > > > > >> >>>> > > > > > > > On Tue, Aug 7, 2018 at 11:30 AM Joshua > Z. > > >> Zhang < > > >> >>>> > > > > > [email protected]> > > >> >>>> > > > > > > > wrote: > > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > I strongly suggest to track this PR > > >> >>>> > > > > > > > > > > >> >>>> https://github.com/apache/incubator-mxnet/pull/11908 > > >> >>>> > < > > >> >>>> > > > > > > > > > > >> >>>> https://github.com/apache/incubator-mxnet/pull/11908 > > >> >>>> > > > > >> >>>> > > in 1.3 > > >> >>>> > > > > > release > > >> >>>> > > > > > > > > which fixed the usability issue for > > >> lower end > > >> >>>> > machines > > >> >>>> > > that > > >> >>>> > > > > > don’t have > > >> >>>> > > > > > > as > > >> >>>> > > > > > > > > large shared memory space as ec2 > > >> instances. > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > Best, > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > - Zhi > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > On Aug 7, 2018, at 9:05 AM, Roshani > > >> Nagmote < > > >> >>>> > > > > > > [email protected] > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > wrote: > > >> >>>> > > > > > > > > > > > >> >>>> > > > > > > > > > Hi all, > > >> >>>> > > > > > > > > > > > >> >>>> > > > > > > > > > Right now, we are delaying MXNet > 1.3 > > >> release > > >> >>>> for > > >> >>>> > > pending > > >> >>>> > > > > > TensorRT PR > > >> >>>> > > > > > > ( > > >> >>>> > > > > > > > > > https://github.com/apache/ > > >> >>>> > incubator-mxnet/pull/11325 > > >> >>>> > > ). > > >> >>>> > > > > > > > > > > > >> >>>> > > > > > > > > > I wanted to ask everyone for their > > >> opinions > > >> >>>> if we > > >> >>>> > > should > > >> >>>> > > > > delay > > >> >>>> > > > > > the > > >> >>>> > > > > > > > > release > > >> >>>> > > > > > > > > > to get tensorRT integration in or > we > > >> should go > > >> >>>> > ahead > > >> >>>> > > with > > >> >>>> > > > the > > >> >>>> > > > > > release > > >> >>>> > > > > > > > and > > >> >>>> > > > > > > > > > include tensorRT in next release. > > >> Please > > >> >>>> provide > > >> >>>> > > > suggestions. > > >> >>>> > > > > > > > > > > > >> >>>> > > > > > > > > > Thanks, > > >> >>>> > > > > > > > > > Roshani > > >> >>>> > > > > > > > > > > > >> >>>> > > > > > > > > > On Mon, Aug 6, 2018 at 12:45 AM > Hagay > > >> Lupesko > > >> >>>> < > > >> >>>> > > > > > [email protected]> > > >> >>>> > > > > > > > wrote: > > >> >>>> > > > > > > > > > > > >> >>>> > > > > > > > > >> Some thoughts: why not keep it > out of > > >> 1.3, > > >> >>>> and > > >> >>>> > > merge it > > >> >>>> > > > into > > >> >>>> > > > > > master > > >> >>>> > > > > > > so > > >> >>>> > > > > > > > > it > > >> >>>> > > > > > > > > >> can go out with 1.4 instead? > > >> >>>> > > > > > > > > >> Pros: > > >> >>>> > > > > > > > > >> - Reduce quality risks for 1.3 > > >> >>>> > > > > > > > > >> - More time to test and get > feedback > > >> before > > >> >>>> > release > > >> >>>> > > > > > > > > >> - Avoid further delays in 1.3 > release > > >> (lots > > >> >>>> of > > >> >>>> > good > > >> >>>> > > stuff > > >> >>>> > > > > > there > > >> >>>> > > > > > > > already > > >> >>>> > > > > > > > > for > > >> >>>> > > > > > > > > >> users) > > >> >>>> > > > > > > > > >> Cons: > > >> >>>> > > > > > > > > >> - People will need to get master > to > > >> >>>> experiment > > >> >>>> > with > > >> >>>> > > TRT > > >> >>>> > > > (not > > >> >>>> > > > > > a major > > >> >>>> > > > > > > > > issue > > >> >>>> > > > > > > > > >> IMO) > > >> >>>> > > > > > > > > >> > > >> >>>> > > > > > > > > >> Besides, TRT requires a build flag > > >> anyway, so > > >> >>>> > MXNet > > >> >>>> > > users > > >> >>>> > > > > > consuming > > >> >>>> > > > > > > > > built > > >> >>>> > > > > > > > > >> packages (PyPi, Scala) will anyway > > >> not be > > >> >>>> able to > > >> >>>> > > try it > > >> >>>> > > > out > > >> >>>> > > > > > unless > > >> >>>> > > > > > > > > >> building from source... > > >> >>>> > > > > > > > > >> > > >> >>>> > > > > > > > > >> Thoughts? > > >> >>>> > > > > > > > > >> > > >> >>>> > > > > > > > > >> On Sun, Aug 5, 2018 at 10:38 PM > > >> Steffen > > >> >>>> Rochel < > > >> >>>> > > > > > > > [email protected] > > >> >>>> > > > > > > > > > > > >> >>>> > > > > > > > > >> wrote: > > >> >>>> > > > > > > > > >> > > >> >>>> > > > > > > > > >>> Marek, Kellen, Jun, Da, Eric, > myself > > >> and a > > >> >>>> few > > >> >>>> > > other > > >> >>>> > > > people > > >> >>>> > > > > > > discussed > > >> >>>> > > > > > > > > >>> offline about TensorRT > integration > > >> PR ( > > >> >>>> > > > > > > > > >>> > > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11325 ). > > >> >>>> > > > We > > >> >>>> > > > > > do > > >> >>>> > > > > > > agree > > >> >>>> > > > > > > > > that > > >> >>>> > > > > > > > > >>> it > > >> >>>> > > > > > > > > >>> would be good to include the PR > into > > >> >>>> upcoming 1.3 > > >> >>>> > > > release, > > >> >>>> > > > > > but are > > >> >>>> > > > > > > > all > > >> >>>> > > > > > > > > >>> concerned about the risk involved > > >> and the > > >> >>>> > breaking > > >> >>>> > > API > > >> >>>> > > > > > change. The > > >> >>>> > > > > > > > > >>> discussion converged to following > > >> proposal. > > >> >>>> (1) > > >> >>>> > > change to > > >> >>>> > > > > > contrib > > >> >>>> > > > > > > PR > > >> >>>> > > > > > > > > and > > >> >>>> > > > > > > > > >>> (2) define a different top level > API > > >> to > > >> >>>> indicate > > >> >>>> > > that the > > >> >>>> > > > > > package > > >> >>>> > > > > > > is > > >> >>>> > > > > > > > > part > > >> >>>> > > > > > > > > >>> of contrib and experimental > (details > > >> of API > > >> >>>> TBD > > >> >>>> > > between > > >> >>>> > > > > > Marek, > > >> >>>> > > > > > > Kellen > > >> >>>> > > > > > > > > and > > >> >>>> > > > > > > > > >>> Eric). This change would allow to > > >> include > > >> >>>> TRT > > >> >>>> > > integration > > >> >>>> > > > > > with v1.3 > > >> >>>> > > > > > > > to > > >> >>>> > > > > > > > > >>> enable users to try TRT with > MXNet, > > >> >>>> minimize the > > >> >>>> > > risk and > > >> >>>> > > > > > avoid > > >> >>>> > > > > > > > > breaking > > >> >>>> > > > > > > > > >>> API change. > > >> >>>> > > > > > > > > >>> To accommodate the change the > > >> request is to > > >> >>>> delay > > >> >>>> > > RC for > > >> >>>> > > > a > > >> >>>> > > > > > few > > >> >>>> > > > > > > days. > > >> >>>> > > > > > > > > >>> > > >> >>>> > > > > > > > > >>> Regards, > > >> >>>> > > > > > > > > >>> Steffen > > >> >>>> > > > > > > > > >>> > > >> >>>> > > > > > > > > >>> On Tue, Jul 31, 2018 at 5:08 PM > > >> Roshani > > >> >>>> Nagmote < > > >> >>>> > > > > > > > > >> [email protected] > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>> wrote: > > >> >>>> > > > > > > > > >>> > > >> >>>> > > > > > > > > >>>> Hi, > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>>> I have created a wiki for > tracking > > >> MXNet > > >> >>>> 1.3 > > >> >>>> > > release > > >> >>>> > > > with > > >> >>>> > > > > > the > > >> >>>> > > > > > > > > timeline. > > >> >>>> > > > > > > > > >>>> Please take a look here: > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>> > > >> >>>> > > > > > > > > >> > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > >> >>>> > > > > > >> >>>> > > https://cwiki.apache.org/confluence/display/MXNET/ > > >> >>>> > Apache+MXNet+%28incubating%29+1.3.0+Release+Status > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>>> I am still waiting for > following 2 > > >> PRs to > > >> >>>> get > > >> >>>> > > merged: > > >> >>>> > > > > > > > > >>>> TRT integration: > > >> >>>> > > > > > > > > > >> >>>> https://github.com/apache/incubator-mxnet/pull/11325 > > >> >>>> > > > > > > > > >>>> Gluon RNN: > > >> >>>> > > > > > > > >> https://github.com/apache/incubator-mxnet/pull/11482 > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>>> *Code freeze date is > > >> 08/02(Thursday).* > > >> >>>> Kindly > > >> >>>> > try > > >> >>>> > > to > > >> >>>> > > > > > complete > > >> >>>> > > > > > > > ongoing > > >> >>>> > > > > > > > > >>> work > > >> >>>> > > > > > > > > >>>> and get these PRs merged. > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>>> Thanks, > > >> >>>> > > > > > > > > >>>> Roshani > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>>> On Mon, Jul 30, 2018 at 1:02 PM > > >> Roshani > > >> >>>> Nagmote > > >> >>>> > < > > >> >>>> > > > > > > > > >>> [email protected] > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>> wrote: > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>>>> Hi all, > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> Here is an update on MXNet 1.3 > > >> release: > > >> >>>> > > > > > > > > >>>>> I am still waiting for > following > > >> PRs to > > >> >>>> get > > >> >>>> > > merged: > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> TRT integration: > > >> >>>> > > > > > > > > >> > > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11325 > > >> >>>> > > > > > > > > >>>>> Gluon RNN: > > >> >>>> > > > > > > > >> https://github.com/apache/incubator-mxnet/pull/11482 > > >> >>>> > > > > > > > > >>>>> Scala examples: > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11753 > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11621 > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> *New code freeze date is: > 08/03* > > >> Please > > >> >>>> try to > > >> >>>> > > get > > >> >>>> > > > your > > >> >>>> > > > > > ongoing > > >> >>>> > > > > > > > PRs > > >> >>>> > > > > > > > > >>>>> merged by then. > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> @Pedro, I didn't include your > PRs > > >> in > > >> >>>> tracking > > >> >>>> > > list as > > >> >>>> > > > you > > >> >>>> > > > > > said > > >> >>>> > > > > > > > those > > >> >>>> > > > > > > > > >>> are > > >> >>>> > > > > > > > > >>>>> not critical for now. Please > let > > >> me know > > >> >>>> if > > >> >>>> > > those needs > > >> >>>> > > > > to > > >> >>>> > > > > > be > > >> >>>> > > > > > > > > >> included. > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11636 > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11562 > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> I also have updated project > > >> proposal > > >> >>>> cwiki page > > >> >>>> > > to > > >> >>>> > > > update > > >> >>>> > > > > > the > > >> >>>> > > > > > > > status > > >> >>>> > > > > > > > > >> of > > >> >>>> > > > > > > > > >>>>> PRs. > > >> >>>> > > > > > > > > >>>>> < > > >> >>>> > > > > > > > > >>>> > > >> >>>> > > > > > > > > >>> > > >> >>>> > > > > > > > > >> > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > >> >>>> > > > > > >> >>>> > > https://cwiki.apache.org/confluence/display/MXNET/ > > >> >>>> > Project+Proposals+for+next+MXNet+Release > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> Please let me know if I am > missing > > >> >>>> something. > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> Thanks, > > >> >>>> > > > > > > > > >>>>> Roshani > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>> On Thu, Jul 26, 2018 at 1:34 PM > > >> Pedro > > >> >>>> Larroy < > > >> >>>> > > > > > > > > >>>> [email protected]> > > >> >>>> > > > > > > > > >>>>> wrote: > > >> >>>> > > > > > > > > >>>>> > > >> >>>> > > > > > > > > >>>>>> I would like to get these PR > > >> merged: > > >> >>>> > > > > > > > > >>>>>> > > >> >>>> > > > > > > > > >>>>>> > > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11636 > > >> >>>> > > > > > > > > >>>>>> > > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11562 > > >> >>>> > > > > > > > > >>>>>> > > >> >>>> > > > > > > > > >>>>>> How much longer until the code > > >> freeze? > > >> >>>> > > > > > > > > >>>>>> > > >> >>>> > > > > > > > > >>>>>> On Thu, Jul 26, 2018 at 1:44 > AM > > >> Roshani > > >> >>>> > Nagmote > > >> >>>> > > < > > >> >>>> > > > > > > > > >>>>>> [email protected]> > > >> >>>> > > > > > > > > >>>>>> wrote: > > >> >>>> > > > > > > > > >>>>>> > > >> >>>> > > > > > > > > >>>>>>> Hi all, > > >> >>>> > > > > > > > > >>>>>>> > > >> >>>> > > > > > > > > >>>>>>> PRs waiting to be merged for > 1.3 > > >> >>>> release: > > >> >>>> > > > > > > > > >>>>>>> > > >> >>>> > > https://github.com/apache/incubator-mxnet/pull/11325 > > >> >>>> > > > > > > > > >>>>>>> > > >> >>>> > > > > > > > > >>>>>>> Are there any other PRs > waiting > > >> to get > > >> >>>> > merged? > > >> >>>> > > Please > > >> >>>> > > > > > let me > > >> >>>> > > > > > > > know. > > >> >>>> > > > > > > > > >>>>>>> > > >> >>>> > > > > > > > > >>>>>>> Release blocker issue: > > >> >>>> > > > > > > > > >>>>>>> > > >> >>>> > > > > https://github.com/apache/incubator-mxnet/issues/11853 > > >> >>>> > > > > > > > > >>>>>>> > > >> >>>> > > > > > > > > >>>>>>> @Marco, @Kellen, Thanks for > > >> bringing up > > >> >>>> the > > >> >>>> > > important > > >> >>>> > > > > > topic. I > > >> >>>> > > > > > > > > >> agree > > >> >>>> > > > > > > > > >>>>>> with > > >> >>>> > > > > > > > > >>>>>>> you and we(internal Amazon > team) > > >> will be > > >> >>>> > > working on > > >> >>>> > > > > > fixing the > > >> >>>> > > > > > > > > >>>> disabled > > >> >>>> > > > > > > > > >>>>>>> tests. > > >> >>>> > > > > > > > > >>>>>>> Currently, my colleague, Hao > Jin > > >> is > > >> >>>> working > > >> >>>> > on > > >> >>>> > > > > compiling > > >> >>>> > > > > > the > > >> >>>> > > > > > > list > > >> >>>> > > > > > > > > >> of > > >> >>>> > > > > > > > > >>>>>>> disabled tests and leading > the > > >> effort > > >> >>>> to fix > > >> >>>> > > them in > > >> >>>> > > > > the > > >> >>>> > > > > > next > > >> >>>> > > > > > > few > > >> >>>> > > > > > > > > >>>> days. > > >> >>>> > > > > > > > > >>>>>>> > > >> >>>> > > > > > > > > >>>>>>> Thanks, > > >> >>>> > > > > > > > > >>>>>>> Roshani > > >> >>>> > > > > > > > > >>>>>>> > > >> >>>> > > > > > > > > >>>>>>> On Mon, Jul 23, 2018 at 6:39 > PM > > >> kellen > > >> >>>> > > sunderland < > > >> >>>> > > > > > > > > >>>>>>> [email protected]> > > >> wrote: > > >> >>>> > > > > > > > > >>>>>>> > > >> >>>> > > > > > > > > >>>>>>>> Thanks again for organizing > > >> Roshani. I > > >> >>>> > > believe the > > >> >>>> > > > > > TensorRT > > >> >>>> > > > > > > > > >> work > > >> >>>> > > > > > > > > >>> is > > >> >>>> > > > > > > > > >>>>>>> ready > > >> >>>> > > > > > > > > >>>>>>>> for a merge. Thanks to > Marek > > >> and all > > >> >>>> the > > >> >>>> > > NVIDIA > > >> >>>> > > > > people > > >> >>>> > > > > > for > > >> >>>> > > > > > > > > >>>> iterating > > >> >>>> > > > > > > > > >>>>>> on > > >> >>>> > > > > > > > > >>>>>>>> it. If possible could a > > >> committer > > >> >>>> review, > > >> >>>> > > make sure > > >> >>>> > > > > it > > >> >>>> > > > > > meets > > >> >>>> > > > > > > > > >>> their > > >> >>>> > > > > > > > > >>>>>>>> expectations and then merge? > > >> PR is > > >> >>>> here: > > >> >>>> > > > > > > > > >>>>>>>> > > >> >>>> > > > > https://github.com/apache/incubator-mxnet/pull/11325 > > >> >>>> > > > > > > > > >>>>>>>> > > >> >>>> > > > > > > > > >>>>>>>> To Marco's point. I'd > > >> recommend we > > >> >>>> review > > >> >>>> > > some of > > >> >>>> > > > > those > > >> >>>> > > > > > > > > >> disabled > > >> >>>> > > > > > > > > >>>>>> tests > > >> >>>> > > > > > > > > >>>>>>> and > > >> >>>> > > > > > > > > >>>>>>>> see how likely they are to > > >> affect users > > >> >>>> > > before we > > >> >>>> > > > cut > > >> >>>> > > > > a > > >> >>>> > > > > > > release. > > >> >>>> > > > > > > > > >>>>>> Many of > > >> >>>> > > > > > > > > >>>>>>>> them are obviously not too > > >> important > > >> >>>> from a > > >> >>>> > > user's > > >> >>>> > > > > > point of > > >> >>>> > > > > > > view > > >> >>>> > > > > > > > > >>>> (e.g. > > >> >>>> > > > > > > > > >>>>>>>> downloading a > sometimes-offline > > >> image > > >> >>>> in a > > >> >>>> > > test). > > >> >>>> > > > One > > >> >>>> > > > > > idea > > >> >>>> > > > > > > > > >> would > > >> >>>> > > > > > > > > >>> be > > >> >>>> > > > > > > > > >>>>>> to > > >> >>>> > > > > > > > > >>>>>>> try > > >> >>>> > > > > > > > > >>>>>>>> and address as many of the > > >> customer > > >> >>>> > impacting > > >> >>>> > > issues > > >> >>>> > > > > as > > >> >>>> > > > > > > possible > > >> >>>> > > > > > > > > >>>>>> between > > >> >>>> > > > > > > > > >>>>>>>> code freeze and the RC0 > vote. > > >> >>>> > > > > > > > > >>>>>>>> > > >> >>>> > > > > > > > > >>>>>>>> On Mon, Jul 23, 2018 at > 1:23 PM > > >> Marco > > >> >>>> de > > >> >>>> > Abreu > > >> >>>> > > > > > > > > >>>>>>>> < > [email protected] > > >> .invalid> > > >> >>>> > wrote: > > >> >>>> > > > > > > > > >>>>>>>> > > >> >>>> > > > > > > > > >>>>>>>>> Hello Roshani, > > >> >>>> > > > > > > > > >>>>>>>>> > > >> >>>> > > > > > > > > >>>>>>>>> frequent releases are good > and > > >> I'm > > >> >>>> > > supportive for > > >> >>>> > > > > this > > >> >>>> > > > > > in > > >> >>>> > > > > > > > > >>> general > > >> >>>> > > > > > > > > >>>> in > > >> >>>> > > > > > > > > >>>>>>>> order > > >> >>>> > > > > > > > > >>>>>>>>> to provide our users with > the > > >> latest > > >> >>>> > > features and > > >> >>>> > > > > > > > > >> improvements. > > >> >>>> > > > > > > > > >>>> But > > >> >>>> > > > > > > > > >>>>>> at > > >> >>>> > > > > > > > > >>>>>>>> the > > >> >>>> > > > > > > > > >>>>>>>>> moment, I'm slightly > concerned > > >> about > > >> >>>> the > > >> >>>> > test > > >> >>>> > > > > coverage > > >> >>>> > > > > > due to > > >> >>>> > > > > > > > > >>>> [1]. I > > >> >>>> > > > > > > > > >>>>>>> want > > >> >>>> > > > > > > > > >>>>>>>>> us to be conscious about > > >> cutting a > > >> >>>> release > > >> >>>> > > even > > >> >>>> > > > > though > > >> >>>> > > > > > not > > >> >>>> > > > > > > all > > >> >>>> > > > > > > > > >>>> tests > > >> >>>> > > > > > > > > >>>>>>> are > > >> >>>> > > > > > > > > >>>>>>>>> enabled (29 disabled tests > [2] > > >> as of > > >> >>>> > today). > > >> >>>> > > > > However, I > > >> >>>> > > > > > > > > >>>> acknowledge > > >> >>>> > > > > > > > > >>>>>>> that > > >> >>>> > > > > > > > > >>>>>>>> we > > >> >>>> > > > > > > > > >>>>>>>>> have improved by a lot > lately > > >> thanks > > >> >>>> to > > >> >>>> > > everybody > > >> >>>> > > > > > > > > >> participating > > >> >>>> > > > > > > > > >>>> and > > >> >>>> > > > > > > > > >>>>>>>> leading > > >> >>>> > > > > > > > > >>>>>>>>> the efforts around > improving > > >> flaky > > >> >>>> tests. > > >> >>>> > > From a > > >> >>>> > > > > > > retrospective > > >> >>>> > > > > > > > > >>>>>> point of > > >> >>>> > > > > > > > > >>>>>>>>> view, we could say that > these > > >> efforts > > >> >>>> have > > >> >>>> > > actually > > >> >>>> > > > > > revealed > > >> >>>> > > > > > > > > >>> some > > >> >>>> > > > > > > > > >>>>>> quite > > >> >>>> > > > > > > > > >>>>>>>>> interesting bugs and thus > the > > >> time > > >> >>>> was well > > >> >>>> > > spent > > >> >>>> > > > and > > >> >>>> > > > > > yielded > > >> >>>> > > > > > > > > >>> good > > >> >>>> > > > > > > > > >>>>>>>> results. > > >> >>>> > > > > > > > > >>>>>>>>> > > >> >>>> > > > > > > > > >>>>>>>>> What does the community > think > > >> about > > >> >>>> making > > >> >>>> > > another > > >> >>>> > > > > > sprint of > > >> >>>> > > > > > > > > >>>>>>> improvements > > >> >>>> > > > > > > > > >>>>>>>>> around tests followed up > by a > > >> period > > >> >>>> of 1-2 > > >> >>>> > > weeks > > >> >>>> > > > > > during > > >> >>>> > > > > > > which > > >> >>>> > > > > > > > > >>> we > > >> >>>> > > > > > > > > >>>>>>> observe > > >> >>>> > > > > > > > > >>>>>>>>> the failures closely to > ensure > > >> that no > > >> >>>> > > critical > > >> >>>> > > > paths > > >> >>>> > > > > > are > > >> >>>> > > > > > > > > >>>> impacted? > > >> >>>> > > > > > > > > >>>>>> If > > >> >>>> > > > > > > > > >>>>>>> we > > >> >>>> > > > > > > > > >>>>>>>>> are in a good shape by > then, > > >> we could > > >> >>>> > > continue the > > >> >>>> > > > > > release > > >> >>>> > > > > > > > > >>> process > > >> >>>> > > > > > > > > >>>>>> and > > >> >>>> > > > > > > > > >>>>>>> at > > >> >>>> > > > > > > > > >>>>>>>>> the same time have the > > >> advantage of > > >> >>>> giving > > >> >>>> > > > > > contributors more > > >> >>>> > > > > > > > > >>> lead > > >> >>>> > > > > > > > > >>>>>> time > > >> >>>> > > > > > > > > >>>>>>> to > > >> >>>> > > > > > > > > >>>>>>>>> finish their work to > ensure it > > >> gets > > >> >>>> into > > >> >>>> > the > > >> >>>> > > > release > > >> >>>> > > > > > in the > > >> >>>> > > > > > > > > >>>> desired > > >> >>>> > > > > > > > > >>>>>>>>> quality. > > >> >>>> > > > > > > > > >>>>>>>>> > > >> >>>> > > > > > > > > >>>>>>>>> Again, thanks to everybody > for > > >> their > > >> >>>> > efforts > > >> >>>> > > during > > >> >>>> > > > > > the last > > >> >>>> > > > > > > > > >>> weeks > > >> >>>> > > > > > > > > >>>>>> to > > >> >>>> > > > > > > > > >>>>>>>>> improve the usability and > > >> stability of > > >> >>>> > > MXNet. This > > >> >>>> > > > is > > >> >>>> > > > > > great > > >> >>>> > > > > > > > > >>>>>> community > > >> >>>> > >
