Re: Apache MXNet v1.4.0 release status
Hey Steffen, thanks for allowing a little extra time to merge PRs. All my PRs are in. #13905 looks good to me, but we may want someone a little more familiar to MKLDNN / the ndarray.h file review as well. On Sun, Jan 20, 2019 at 11:53 AM Steffen Rochel wrote: > Still waiting for merge of > https://github.com/apache/incubator-mxnet/pull/13905 > > All other PR are merged and CI tests are passing. Please no more changes on > 1.4.x branch beside merge for PR 13905, so we can move forward with 1.4.0 > release. > > Best Regards, > Steffen > > On Fri, Jan 18, 2019 at 9:48 AM Steffen Rochel > wrote: > > > Dear MXNet community - > > thanks for merging previously agreed PR's into v1.4.x branch. > > > > Kellen and Zhennan - what is the ETA for > > https://github.com/apache/incubator-mxnet/pull/13905 ? Please try to > > merge today. > > > > Yuxi asked offline to merge > > https://github.com/apache/incubator-mxnet/pull/13922 to complete Horovod > > integration. PR will be merged today. > > > > After above PR are merge and CI passed successfully 1.4.0.rc1 will be > > created and voting started. > > > > Regards, > > Steffen > > > > On Wed, Jan 16, 2019 at 2:34 PM kellen sunderland < > > kellen.sunderl...@gmail.com> wrote: > > > >> Sounds good Steffen. I believe most of these PRs only fix functional > >> problems (they don't add features) and should be fairly low risk. > >> > >> Update from my side: > >> 13695: https://github.com/apache/incubator-mxnet/pull/13899 <- Already > >> merged, thanks Haibin! > >> > >> Ready for review / merge with all tests passed: > >> https://github.com/apache/incubator-mxnet/pull/13898 > >> https://github.com/apache/incubator-mxnet/pull/13900 > >> https://github.com/apache/incubator-mxnet/pull/13897 > >> > >> -Kellen > >> > >> On Tue, Jan 15, 2019 at 10:06 PM Steffen Rochel < > steffenroc...@gmail.com> > >> wrote: > >> > >> > Kellen - thanks, please go ahead. I'm ok as long we avoid risky PR and > >> can > >> > get to a stable and tested build by Friday. > >> > > >> > Best, > >> > Steffen > >> > > >> > On Tue, Jan 15, 2019 at 9:48 PM kellen sunderland < > >> > kellen.sunderl...@gmail.com> wrote: > >> > > >> > > Many thanks for the license fixes and allowing some other PRs to > come > >> > into > >> > > the release. > >> > > > >> > > For #13697 I've contacted the author Zhennan and let him know he can > >> cut > >> > a > >> > > branch to v1.4.x to update any APIs that are required. > >> > > > >> > > For the other PRs listed here's some new PRs for the v1.4.x branch. > >> > > 13188: https://github.com/apache/incubator-mxnet/pull/13898 > >> > > 13727: https://github.com/apache/incubator-mxnet/pull/13900 > >> > > 13695: https://github.com/apache/incubator-mxnet/pull/13899 <- > >> Already > >> > > merged, thanks Haibin! > >> > > > >> > > I'd also propose that we include this TensorRT PR which fixes > >> inference > >> > > bugs and updates to a more stable commit of onnx-trt: > >> > > https://github.com/apache/incubator-mxnet/pull/13897 > >> > > > >> > > -Kellen > >> > > > >> > > On Tue, Jan 15, 2019 at 5:57 PM Steffen Rochel < > >> steffenroc...@gmail.com> > >> > > wrote: > >> > > > >> > > > Hi Lin - please go ahead to integrate into 1.4.x. > >> > > > Steffen > >> > > > > >> > > > On Tue, Jan 15, 2019 at 4:17 PM Lin Yuan > >> wrote: > >> > > > > >> > > > > Hi Steffen, > >> > > > > > >> > > > > I would like to ask to include one more PR for 1.4.0.rc1: > >> > > > > https://github.com/apache/incubator-mxnet/pull/13845 > >> > > > > > >> > > > > This PR exports exception handling API of MXNet. It is needed by > >> > > Horovod > >> > > > > with MXNet integration to elegantly throw exception at Python > >> level > >> > > > rather > >> > > > > than a C++ abort. > >> > > > > > >> > > > > Thanks, > >> > > > > > >> > > > > Lin > >> > > > > > >> > > > > > >> > > > > On Tue, Jan 15, 2019 at 2:24 PM Steffen Rochel < > >> > > steffenroc...@gmail.com> > >> > > > > wrote: > >> > > > > > >> > > > > > Dear MXNet community - > >> > > > > > Zach & friends made good progress resolving the licensing > >> issues. > >> > One > >> > > > > more > >> > > > > > PR on 1.4.x branch is expected today. > >> > > > > > The code freeze for 1.4.0.rc1 is Thursday Jan 17th 6pm PST. > >> > > > > > I'm asking the requester to add following PR to 1.4.x branch: > >> > > > > > Tao: > >> > > > > > https://github.com/apache/incubator-mxnet/pull/13882 > >> > > > > > Kellen: > >> > > > > > https://github.com/apache/incubator-mxnet/pull/13697 > >> > > > > > https://github.com/apache/incubator-mxnet/pull/13188 > >> > > > > > https://github.com/apache/incubator-mxnet/pull/13727 > >> > > > > > https://github.com/apache/incubator-mxnet/pull/13695 > >> > > > > > Pedro: > >> > > > > > https://github.com/apache/incubator-mxnet/pull/13535 > >> > > > > > > >> > > > > > If there are additional PR to be considered for 1.4.0.rc1 > please > >> > send > >> > > > > > request to dev@. > >> > > > > > > >> > > > > > Regards, > >> > > > > > Steffen > >> >
Re: Apache MXNet v1.4.0 release status
Still waiting for merge of https://github.com/apache/incubator-mxnet/pull/13905 All other PR are merged and CI tests are passing. Please no more changes on 1.4.x branch beside merge for PR 13905, so we can move forward with 1.4.0 release. Best Regards, Steffen On Fri, Jan 18, 2019 at 9:48 AM Steffen Rochel wrote: > Dear MXNet community - > thanks for merging previously agreed PR's into v1.4.x branch. > > Kellen and Zhennan - what is the ETA for > https://github.com/apache/incubator-mxnet/pull/13905 ? Please try to > merge today. > > Yuxi asked offline to merge > https://github.com/apache/incubator-mxnet/pull/13922 to complete Horovod > integration. PR will be merged today. > > After above PR are merge and CI passed successfully 1.4.0.rc1 will be > created and voting started. > > Regards, > Steffen > > On Wed, Jan 16, 2019 at 2:34 PM kellen sunderland < > kellen.sunderl...@gmail.com> wrote: > >> Sounds good Steffen. I believe most of these PRs only fix functional >> problems (they don't add features) and should be fairly low risk. >> >> Update from my side: >> 13695: https://github.com/apache/incubator-mxnet/pull/13899 <- Already >> merged, thanks Haibin! >> >> Ready for review / merge with all tests passed: >> https://github.com/apache/incubator-mxnet/pull/13898 >> https://github.com/apache/incubator-mxnet/pull/13900 >> https://github.com/apache/incubator-mxnet/pull/13897 >> >> -Kellen >> >> On Tue, Jan 15, 2019 at 10:06 PM Steffen Rochel >> wrote: >> >> > Kellen - thanks, please go ahead. I'm ok as long we avoid risky PR and >> can >> > get to a stable and tested build by Friday. >> > >> > Best, >> > Steffen >> > >> > On Tue, Jan 15, 2019 at 9:48 PM kellen sunderland < >> > kellen.sunderl...@gmail.com> wrote: >> > >> > > Many thanks for the license fixes and allowing some other PRs to come >> > into >> > > the release. >> > > >> > > For #13697 I've contacted the author Zhennan and let him know he can >> cut >> > a >> > > branch to v1.4.x to update any APIs that are required. >> > > >> > > For the other PRs listed here's some new PRs for the v1.4.x branch. >> > > 13188: https://github.com/apache/incubator-mxnet/pull/13898 >> > > 13727: https://github.com/apache/incubator-mxnet/pull/13900 >> > > 13695: https://github.com/apache/incubator-mxnet/pull/13899 <- >> Already >> > > merged, thanks Haibin! >> > > >> > > I'd also propose that we include this TensorRT PR which fixes >> inference >> > > bugs and updates to a more stable commit of onnx-trt: >> > > https://github.com/apache/incubator-mxnet/pull/13897 >> > > >> > > -Kellen >> > > >> > > On Tue, Jan 15, 2019 at 5:57 PM Steffen Rochel < >> steffenroc...@gmail.com> >> > > wrote: >> > > >> > > > Hi Lin - please go ahead to integrate into 1.4.x. >> > > > Steffen >> > > > >> > > > On Tue, Jan 15, 2019 at 4:17 PM Lin Yuan >> wrote: >> > > > >> > > > > Hi Steffen, >> > > > > >> > > > > I would like to ask to include one more PR for 1.4.0.rc1: >> > > > > https://github.com/apache/incubator-mxnet/pull/13845 >> > > > > >> > > > > This PR exports exception handling API of MXNet. It is needed by >> > > Horovod >> > > > > with MXNet integration to elegantly throw exception at Python >> level >> > > > rather >> > > > > than a C++ abort. >> > > > > >> > > > > Thanks, >> > > > > >> > > > > Lin >> > > > > >> > > > > >> > > > > On Tue, Jan 15, 2019 at 2:24 PM Steffen Rochel < >> > > steffenroc...@gmail.com> >> > > > > wrote: >> > > > > >> > > > > > Dear MXNet community - >> > > > > > Zach & friends made good progress resolving the licensing >> issues. >> > One >> > > > > more >> > > > > > PR on 1.4.x branch is expected today. >> > > > > > The code freeze for 1.4.0.rc1 is Thursday Jan 17th 6pm PST. >> > > > > > I'm asking the requester to add following PR to 1.4.x branch: >> > > > > > Tao: >> > > > > > https://github.com/apache/incubator-mxnet/pull/13882 >> > > > > > Kellen: >> > > > > > https://github.com/apache/incubator-mxnet/pull/13697 >> > > > > > https://github.com/apache/incubator-mxnet/pull/13188 >> > > > > > https://github.com/apache/incubator-mxnet/pull/13727 >> > > > > > https://github.com/apache/incubator-mxnet/pull/13695 >> > > > > > Pedro: >> > > > > > https://github.com/apache/incubator-mxnet/pull/13535 >> > > > > > >> > > > > > If there are additional PR to be considered for 1.4.0.rc1 please >> > send >> > > > > > request to dev@. >> > > > > > >> > > > > > Regards, >> > > > > > Steffen >> > > > > > >> > > > > > On Tue, Jan 8, 2019 at 11:28 AM Qing Lan >> > > wrote: >> > > > > > >> > > > > > > Hi all, >> > > > > > > >> > > > > > > I added a section F in the document that explained the current >> > > > > > > static-linked dependencies we used for official release. As >> there >> > > > are a >> > > > > > few >> > > > > > > licenses are under BSD3 and GPL, we need to handle them in our >> > next >> > > > > > > release. Please take a look and leave any concerns you may >> have. >> > > > > > > >> > > > > > > Thanks, >> > > > > > > Qing >> > > > > > > >> >
Re: Random data in tests produces unstable test coverage
That's a great question! Since we record branch-coverage as well, we would now reliably notice these test-gaps and could make targeted tests. Additionally, we could define test-coverage thresholds for certain files to enforce this - e.g. 100% for all operators. While the stochastic approach certainly covers a wider area, failures are currently rather ignored and rerun instead of properly analyzed. Proper tooling like a consistent test-coverage would then enable us to properly investigate these inconsistencies if we encounter them. -Marco On Sun, Jan 20, 2019 at 5:55 PM Chris Olivier wrote: > wouldn’t abandoning stochastic testing mean that execution paths would no > longer be tested and therefore increase issues found in production? > > On Sun, Jan 20, 2019 at 5:13 AM Marco de Abreu > wrote: > > > Hello everyone, > > > > a few months ago, I have enabled test coverage recording for the MXNet > > repository [1]. The next step would be to report the coverage changes to > > pull requests so we can make them part of the review. For this, I've > > created a pull request here at [2]. > > > > Before this feature can be enabled though, we have to stabilize the test > > coverage. Due to the extensive usage of random inputs in our tests, the > > execution takes different paths and thus, the coverage results undergo a > > heavy variance. If we would enable the coverage report for PRs, they > would > > be marked as increased/decreased coverage without actually doing so. To > get > > a few examples, visit [3]. > > > > I'd like to provide some detailed examples or screenshots, but with big > > reports [4], CodeCov's webinterface tends to time out. I currently have > an > > open ticket with CodeCov that will hopefully improve the situation soon, > > but we have to live with the timeouts until then. > > > > My proposal to improve the situation would be to divide the work across > the > > community in two stages: > > 1. Replace random inputs in tests with deterministic ones if applicable. > > 2. If randomness cannot be prevented or this 'flakiness' persists, we > could > > run the same test multiple times with coverage, look at the coverage diff > > and then write targeted unittests for the inconsistent parts. > > > > If there is more interest towards test coverage, I'd be happy to write a > > guide that explains how to measure test coverage and detect flaky > coverage > > areas so everybody can help contribute towards a stable test coverage. > > Please let me know what you think. > > > > Best regards, > > Marco > > > > [1]: https://codecov.io/gh/apache/incubator-mxnet > > [2]: https://github.com/apache/incubator-mxnet/pull/12648 > > [3]: https://codecov.io/gh/apache/incubator-mxnet/pulls > > [4]: https://codecov.io/gh/apache/incubator-mxnet/pull/13849 > > >
Re: [Article] Object Detection with Clojure MXNet
Thanks - props for the images belongs to the original Scala article https://medium.com/apache-mxnet/object-detection-with-mxnet-scala-inference-api-9049230c77fd and Kedar - the contributor that ported the object detection feature to Clojure. They are pretty awesome though. On Sat, Jan 19, 2019 at 4:57 PM Marco de Abreu wrote: > Great article! I have to admit I always love your picture choices :D > > -Marco > > Am Sa., 19. Jan. 2019, 21:57 hat Carin Meier > geschrieben: > > > I just blogged about the new Object Detection feature that was just > ported > > from the Scala package into the Clojure package. > > > > > http://gigasquidsoftware.com/blog/2019/01/19/object-detection-with-clojure-mxnet/ > > > > I posted it in Slack but in an effort to direct more communication > towards > > this mailing list, I'm putting it out here too :) > > > > - Carin > > >
Random data in tests produces unstable test coverage
Hello everyone, a few months ago, I have enabled test coverage recording for the MXNet repository [1]. The next step would be to report the coverage changes to pull requests so we can make them part of the review. For this, I've created a pull request here at [2]. Before this feature can be enabled though, we have to stabilize the test coverage. Due to the extensive usage of random inputs in our tests, the execution takes different paths and thus, the coverage results undergo a heavy variance. If we would enable the coverage report for PRs, they would be marked as increased/decreased coverage without actually doing so. To get a few examples, visit [3]. I'd like to provide some detailed examples or screenshots, but with big reports [4], CodeCov's webinterface tends to time out. I currently have an open ticket with CodeCov that will hopefully improve the situation soon, but we have to live with the timeouts until then. My proposal to improve the situation would be to divide the work across the community in two stages: 1. Replace random inputs in tests with deterministic ones if applicable. 2. If randomness cannot be prevented or this 'flakiness' persists, we could run the same test multiple times with coverage, look at the coverage diff and then write targeted unittests for the inconsistent parts. If there is more interest towards test coverage, I'd be happy to write a guide that explains how to measure test coverage and detect flaky coverage areas so everybody can help contribute towards a stable test coverage. Please let me know what you think. Best regards, Marco [1]: https://codecov.io/gh/apache/incubator-mxnet [2]: https://github.com/apache/incubator-mxnet/pull/12648 [3]: https://codecov.io/gh/apache/incubator-mxnet/pulls [4]: https://codecov.io/gh/apache/incubator-mxnet/pull/13849
