Re: [VOTE] Remove conflicting OpenMP from CMake builds

2019-06-19 Thread Pedro Larroy
we can keep gathering facts to understand and reproduce your results so we can have a constructive discussion? I think this would be a compromise to move forward in a non confrontational way. Thank you. Pedro. On Tue, Jun 18, 2019 at 1:35 PM Pedro Larroy wrote: > > Good suggestion. I t

Re: [VOTE] Remove conflicting OpenMP from CMake builds

2019-06-18 Thread Pedro Larroy
would suggest a DISCUSS thread is more proper > before the voting one. > As the tension can generally be high in a voting thread and it is hard to > make a technical decision without previous discussions and pros/cons being > listed. > > Tianqi > > On Fri, Jun 14, 2019 at

Re: [VOTE] Remove conflicting OpenMP from CMake builds

2019-06-18 Thread Pedro Larroy
veto of Chris has technical justification. It > would be great if the committee could cast a vote on this to clarify the > situation. > > Best > Anton > > [1] https://www.apache.org/foundation/glossary.html > > > On Mon, 17 Jun 2019 at 22:01, Pedro Larroy > wrote: > > &g

Re: OMP

2019-06-18 Thread Pedro Larroy
First of all, thanks for following up on this topic and not swiping the problem under the rug. You might very well be right and have some numbers which corroborate your findings, this might be something to celebrate. Before continuing our technical discussion I would like to take a step back and

Re: [VOTE] Remove conflicting OpenMP from CMake builds

2019-06-17 Thread Pedro Larroy
rgence? As explained in the PR there seems to be no performance advantage on adding our own version of openmp. Thanks. On Mon, Jun 17, 2019 at 12:55 PM Pedro Larroy wrote: > > Thanks for your answer Chris. I'm not militant in regards to one > option or another nor belong of any club t

Re: [VOTE] Remove conflicting OpenMP from CMake builds

2019-06-17 Thread Pedro Larroy
7f0b073f4000) > libquadmath.so.0 => > /usr/local/lib/python3.6/dist-packages/mxnet/libquadmath.so.0 > (0x7f0af910) > > > On Mon, Jun 17, 2019 at 10:58 AM Pedro Larroy > wrote: > > > I had read the "Apache Voting Process" guide here: > &

Re: [VOTE] Remove conflicting OpenMP from CMake builds

2019-06-17 Thread Pedro Larroy
Could you, or someone more familiar which the Apache way enlighten on how to move this issue forward in a constructive way? Thanks a lot. Pedro. On Mon, Jun 17, 2019 at 10:46 AM Pedro Larroy wrote: > > Thanks. > > How do we go on advancing this PR then? all the questions have been &

Re: [VOTE] Remove conflicting OpenMP from CMake builds

2019-06-17 Thread Pedro Larroy
iginal PR has been vetoed by a committer. A > vote on dev@ won't help you circumvent a veto. > > -sz > > On 2019/06/14 23:59:33, Pedro Larroy wrote: > > Hi all > > > > This is a 5-day vote to act and wrap up an outstanding PR that removes > > linkage with mult

[VOTE] Remove conflicting OpenMP from CMake builds

2019-06-14 Thread Pedro Larroy
Hi all This is a 5-day vote to act and wrap up an outstanding PR that removes linkage with multiple OpenMP from 3rdparty and uses the system provided one which might resolve a number of difficult to debug issues and possible undefined behaviour.

Re: [VOTE] Release Apache MXNet (incubating) version 1.5.0.rc0

2019-06-13 Thread Pedro Larroy
o this? > > Thanks, > Junru > > On Thu, Jun 13, 2019 at 10:33 Pedro Larroy > wrote: > > > I have isolated some of the commits that are causing performance > > regressions in wavenet like models: > > > > Title: 83d2c2d0e:[MXNET-1324] Add NaiveRunGra

Re: [VOTE] Release Apache MXNet (incubating) version 1.5.0.rc0

2019-06-13 Thread Pedro Larroy
master, need to backport to 1.5.x > > Thanks for everyone's help! Please let us know if there is any other issue > with 1.5.0 > > [1] https://github.com/apache/incubator-mxnet/pull/15213 > [2] https://github.com/apache/incubator-mxnet/pull/15216 > > > > Best Reg

Re: default site version & temp bug fix for nav

2019-06-12 Thread Pedro Larroy
the navbar is broken. On Wed, Jun 12, 2019 at 6:26 PM Pedro Larroy wrote: > > I would suggest bisecting to find when the navbar was gone. I have a > python script that I use for bisecting that I can offer. > > On Wed, Jun 12, 2019 at 10:28 AM Aaron Markham > wrote: > >

Re: default site version & temp bug fix for nav

2019-06-12 Thread Pedro Larroy
I would suggest bisecting to find when the navbar was gone. I have a python script that I use for bisecting that I can offer. On Wed, Jun 12, 2019 at 10:28 AM Aaron Markham wrote: > > Hi everyone, there's an issue with the navigation for master. When > you browse any of the API docs, the left

Re: [VOTE] Release Apache MXNet (incubating) version 1.5.0.rc0

2019-06-11 Thread Pedro Larroy
t; > [1]https://github.com/apache/incubator-mxnet/pull/15213 > > > Thanks > > Best Regards > > Lai > > > On Tue, Jun 11, 2019 at 1:46 PM Zhi Zhang wrote: > > > > > > > On 2019/06/11 18:53:56, Pedro Larroy > > wrote: > > > The

Re: [VOTE] Release Apache MXNet (incubating) version 1.5.0.rc0

2019-06-11 Thread Pedro Larroy
nd static_shape is 10% faster than without. > > Overwall we are still 33% faster when comparing master to 1.5. > > Let me know if you think this is a release blocker or not. > > Pedro. > > On Mon, Jun 10, 2019 at 4:51 PM Pedro Larroy > wrote: > > > > -1 > >

Re: Does internal quality matters to users?

2019-06-11 Thread Pedro Larroy
of importance: debuggability, clarity of definition of data structures and navigability with an IDE which breaks with an untyped field keyed with string. Pedro. Pedro. On Tue, Jun 11, 2019 at 11:07 AM Pedro Larroy wrote: > > To put a recent specific example and focus the discussion (there are

Re: [VOTE] Release Apache MXNet (incubating) version 1.5.0.rc0

2019-06-11 Thread Pedro Larroy
The stack trace doesn't seem to come from MXNet, do you have more info? On Tue, Jun 11, 2019 at 11:46 AM Zhi Zhang wrote: > > > > On 2019/06/11 17:36:09, Pedro Larroy wrote: > > A bit more background into this: > > > > While tuning a model using LSTM and c

Re: Does internal quality matters to users?

2019-06-11 Thread Pedro Larroy
attribute to the graph, as we always need the vector of shapes and operate on it while doing shape inference. On Tue, Jun 11, 2019 at 10:18 AM Pedro Larroy wrote: > > Thanks for the good discussion. > > I actually wasn't referring particularly to our conversations in > gith

Re: [VOTE] Release Apache MXNet (incubating) version 1.5.0.rc0

2019-06-11 Thread Pedro Larroy
. Overwall we are still 33% faster when comparing master to 1.5. Let me know if you think this is a release blocker or not. Pedro. On Mon, Jun 10, 2019 at 4:51 PM Pedro Larroy wrote: > > -1 > > We found a performance regression vs 1.4 related to CachedOp which > affects Hybrid fo

Re: Does internal quality matters to users?

2019-06-11 Thread Pedro Larroy
off decisions. > Sometimes the decision may not make every developer mostly happy, but a good > technical compromise could move the project forward and help the community in > general. > > Tianqi > > On Fri, May 31, 2019 at 6:26 AM Isabel Drost-Fromm wrote: >> >&g

Re: Context-specific operator parameters

2019-06-10 Thread Pedro Larroy
I would sideload that kind of back-end specific configuration through a configuration file / configuration object such as what we discussed to save the results of tuning of cudnn convolutions. A configuration object could inject special parameters inside operators, but I wouldn't expose such

Re: [VOTE] Release Apache MXNet (incubating) version 1.5.0.rc0

2019-06-10 Thread Pedro Larroy
-1 We found a performance regression vs 1.4 related to CachedOp which affects Hybrid forward, which we are looking into. Pedro. On Mon, Jun 10, 2019 at 4:33 PM Lin Yuan wrote: > > -1 (Tentatively until resolved) > > I tried to build MXNet 1.5.0 from source and pip install horovod but got > the

Re: CUDA / CUDNN support revisited

2019-06-03 Thread Pedro Larroy
Your proposal of having support for N and N-1 makes a lot of sense to me. Are there use cases for supporting older CUDA versions? Thanks. On Mon, Jun 3, 2019 at 3:06 PM Dick Carter wrote: > > I'd like to revisit the discussion of: >

Does internal quality matters to users?

2019-05-31 Thread Pedro Larroy
Hi folks I would like to share this interesting article. I had a few conversations with members of the community about refactors and internal enhancements that have no apparent impact in user experience and are sometimes put into question. I think Martin does a very good job explaining why

Re: [RFC] Support for creation of Large Tensors in MXNet

2019-05-29 Thread Pedro Larroy
e agreed on merging the end to > end tests for now. The condition is that they have to be replaced by 15th > of July or the next minor release that includes large tensor support; > whichever comes first. > > Thanks everyone for their input on this topic! > > Best regards, &g

Re: [RFC] Support for creation of Large Tensors in MXNet

2019-05-29 Thread Pedro Larroy
Hi Marco I think this is a very good point that we both have made before, and is good that we bring up the topic again, as currently the unit test suite is heavy, costly and takes too long to get feedback for development and doesn't run on embedded. The problem here is that we don't know how to

Re: warnings as errors

2019-05-22 Thread Pedro Larroy
d be ignored? > - for the rest of the warning types, can we turn them on one by one? > > -sz > > On 2019/05/21 22:33:51, Pedro Larroy wrote: > > Hi dev@ > > > > I try to fix any warning that I see during compilation of MXNet in my > > platform and with the build tog

Re: [Discussion] Remove bundled llvm OpenMP

2019-05-22 Thread Pedro Larroy
have necessary rights to reopen this PR. > > > > пн, 20 мая 2019 г. в 08:00, Pedro Larroy : > > > > > Hi Anton, Stas. > > > > > > Can we reopen this PR and get it merged as per the data collected by > > Stas? > > > > > > https://g

Re: Report of MXNet NumPy Project Status

2019-05-22 Thread Pedro Larroy
Thanks, that's a nice summary. Great job and good to know the progress. I think we can do some exciting stuff in terms of parsing the Python AST and converting to a computational graph. Maybe we could brainstorm on that further on the linked ticket. On Wed, May 22, 2019 at 12:12 AM Jun Wu wrote:

Re: New PMC member: Dick Carter

2019-05-21 Thread Pedro Larroy
Finally! Welcome! On Tue, May 21, 2019 at 6:28 PM Steffen Rochel wrote: > > Congratulation Dick! > > On Tue, May 21, 2019 at 2:43 PM Carin Meier wrote: > > > Congrats and welcome! > > > > On Tue, May 21, 2019 at 4:37 PM Marco de Abreu > > wrote: > > > > > The Project Management Committee (PMC)

warnings as errors

2019-05-21 Thread Pedro Larroy
Hi dev@ I try to fix any warning that I see during compilation of MXNet in my platform and with the build toggles that I care about. These seemingly trivial and ungrateful efforts, take nonetheless energy on the contributor side. I think overall I submitted myself more than a dozen of PRs fixing

Re: [Discussion] Remove bundled llvm OpenMP

2019-05-20 Thread Pedro Larroy
Hi Anton, Stas. Can we reopen this PR and get it merged as per the data collected by Stas? https://github.com/apache/incubator-mxnet/pull/12160 https://cwiki.apache.org/confluence/display/MXNET/Benchmarking+MXNet+with+different+OpenMP+implementations There are multiple issues that will be

Re: [Proposal] New operator graph for MXNet

2019-05-17 Thread Pedro Larroy
ows > - The ability of interpolation with multi-language frontend, e.g. being > able to prototype graph optimizations in python/scala/clojure if needed. > > Adding these support needs significant engineering effort, and I do hope we > only have to do it once. While I don't want to force a

Re: [Proposal] New operator graph for MXNet

2019-05-15 Thread Pedro Larroy
> Adding these support needs significant engineering effort, and I do hope we > only have to do it once. While I don't want to force any conclusion here, > I do think Relay is one such candidate. > > Tianqi > > > On Tue, May 14, 2019 at 5:58 PM Pedro Larroy > wrote: &

Re: [Proposal] New operator graph for MXNet

2019-05-14 Thread Pedro Larroy
scenario that you are concerned about. Let me know what you think. Pedro. On Tue, May 14, 2019 at 5:50 PM Pedro Larroy wrote: > > Hi Tianqi > > Thanks for the quick response. > > Could you point to examples where graph.h is being exposed which would > not be possible with what I p

Re: [Proposal] New operator graph for MXNet

2019-05-14 Thread Pedro Larroy
at 5:06 PM Sheng Zha wrote: > > > Hi Pedro, > > > > Thanks for taking the inititaive. Skimming through the design doc, I > > didn't see comparison with existing solutions such as relay in tvm, which > > is already a dependency of mxnet already. Could you el

Re: [Proposal] New operator graph for MXNet

2019-05-14 Thread Pedro Larroy
t; -sz > > On 2019/05/14 23:49:30, Pedro Larroy wrote: > > Hi dev@ > > > > As a result of my deep dives on the graph machinery I have created a > > new proposal to improve the operator graph in MXNet. > > > > This would mean superseding the use of NN

[Proposal] New operator graph for MXNet

2019-05-14 Thread Pedro Larroy
Hi dev@ As a result of my deep dives on the graph machinery I have created a new proposal to improve the operator graph in MXNet. This would mean superseding the use of NNVM Graph in MXNet and having a new implementation that we can use to simplify a lot of code and do powerful graph

Re: assimilation of mshadow into the MXNet codebase

2019-05-14 Thread Pedro Larroy
Hi Sheng. Do you need some help with this? Do we plan to have this for 1.5? Pedro. On Wed, Apr 24, 2019 at 4:26 PM Pedro Larroy wrote: > > Thanks. Great to read. > > On Wed, Apr 24, 2019 at 2:19 PM Sheng Zha wrote: > > > > The community has agreed to donate mshadow t

Re: Python2 End of Life

2019-05-14 Thread Pedro Larroy
+1 Let python2 rest, let's simplify our infrastructure and need to support old Python versions. On Mon, May 13, 2019 at 1:58 PM Jake Lee wrote: > > +1 Recently I upgraded the Numpy version and found out that Pylint had > false alarm on it. The Pylint fix is only available on Python3. So I >

Re: [Announcement] New Committer - Zach Kimberg

2019-05-13 Thread Pedro Larroy
Congratulations On Thu, May 9, 2019 at 11:29 AM Chaitanya Bapat wrote: > > Congratulations Zachary! Way to go! > > On Thu, 9 May 2019 at 14:01, Carin Meier wrote: > > > Congrats! > > > > On Thu, May 9, 2019 at 1:41 PM Per da Silva wrote: > > > > > Nice one! Congratulations =) > > > > > > On

Re: [VOTE] Release Apache MXNet (incubating) version 1.4.1.rc0

2019-05-01 Thread Pedro Larroy
+1 (non-binding) Tried CPU build + C++ tests + 714 Python unit tests in 605s. ARMv7 build + small unit test in QEMU + ARMv8 builds. Thanks. Regards Pedro. On Wed, May 1, 2019 at 10:41 AM Qing Lan wrote: > > +1 (binding) > > build from source works for OSX and Ubuntu CPU > Scala build/test

Re: [Announcement] New Committer - Hao Jin

2019-05-01 Thread Pedro Larroy
Congrats! On Wed, May 1, 2019 at 11:08 AM Lin Yuan wrote: > > Congrats! > > On Tue, Apr 30, 2019 at 11:28 PM Alex Zai wrote: > > > Congrats Hao! > > > > On Tue, Apr 30, 2019 at 10:53 PM Steffen Rochel > > wrote: > > > > > congratulation Hao! > > > > > > On Tue, Apr 30, 2019 at 8:05 AM MiraiWK

Re: docs updates

2019-04-25 Thread Pedro Larroy
Hi Aaron. I'm no design expert, but wouldn't smoothing out the boxes a bit rounding the corners would make the design more pleasant? As an old fart learning from Knuth books, I'm all pro having documentation as close to the source as possible. Could you maybe put two concrete examples so the

Re: [Announcement] New Committer - Wang Jiajun

2019-04-25 Thread Pedro Larroy
Welcome! On Tue, Apr 16, 2019 at 6:34 PM kellen sunderland wrote: > > Welcome! Very impressed with the work fixing memory leaks so far. > > On Tue, Apr 16, 2019 at 9:14 AM Carin Meier wrote: > > > Congrats! > > > > On Tue, Apr 16, 2019 at 11:58 AM Anirudh Subramanian < > >

DNS failures in jenkins

2019-04-25 Thread Pedro Larroy
Hi I see some DNS resolution failures on jenkins, I think this is the cause of jenkins not reporting the build status sometimes. What dns server are we using in the master? should we add a couple of secondary resolvers to remediate?

Re: assimilation of mshadow into the MXNet codebase

2019-04-24 Thread Pedro Larroy
> > > > > > > mshadow is mostly used for tensor arithmetic. There have been > > > > > discussions > > > > > about including it within mxnet. I think it is a good idea. > > > > > > > > > > As a more long term solut

Re: Benchmarking MXNet with different compilers and different OpenMP implementations (results)

2019-04-12 Thread Pedro Larroy
Are there any updates on this? This is still affecting multiprocessing, some tests hang: rces. For information on submitting this issue, please see https://bugs.llvm.org/. [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=2124604270 to reproduce. Assertion failure at

duplicated nnvm code

2019-04-11 Thread Pedro Larroy
Hi I found that src/nnvm and 3rdparty/tvm/nnvm/src/pass/ has duplicated code that we are linking in: ./CMakeFiles/mxnet_static.dir/3rdparty/tvm/nnvm/src/pass/gradient.cc.o ./CMakeFiles/mxnet_static.dir/src/nnvm/gradient.cc.o This can potentially cause problems when linking. The symbol that

Re: [MXNET 2.0 Wishlist] [DISCUSS] Backend choices during runtime

2019-04-11 Thread Pedro Larroy
> > We have a systematic solution to go without ABI headache. I am struggling > > with some errants, and will share our proposal here as soon as I could. > > This will be very interesting topic to discuss. Let's work hard together > > and make it perfect :-) > > &g

Re: [MXNET 2.0 Wishlist] [DISCUSS] Backend choices during runtime

2019-04-11 Thread Pedro Larroy
Thanks Marco for raising this issue. I think we can certainly do some improvements in modularization and build. At the same time Tianqi's point of view is important to consider and on point. I see a high risk of overengineering in such endeavor. I also see increased complexity, difficulty

Re: assimilation of mshadow into the MXNet codebase

2019-04-08 Thread Pedro Larroy
> > > > Do you have a link to both of these proposals? > > > > > > > > > > > > On Fri, Apr 5, 2019 at 20:14 Anirudh Acharya > > > > > > > > > > > > wrote: > > > > > > >

assimilation of mshadow into the MXNet codebase

2019-04-05 Thread Pedro Larroy
Hi Some developers have noticed that working in mshadow is cumbersome as it's a 3rdparty subrepo. Since mshadow is a bunch of headers which don't have much of independent tests / library functionality, me and other developers believe that it would be good to assimilate this code in the

Re: Discussing plans for next MXNet releases

2019-04-02 Thread Pedro Larroy
Great initiative. I would like to add the issue that tracks APIs that we would like to break for 2.0 so we can take the chance to streamline and improve customer facing code: https://github.com/apache/incubator-mxnet/issues/9686 I would be happy to volunteer for 2.0 release with assistance from

Developer tools for MXNet

2019-03-28 Thread Pedro Larroy
Hi developers! We did a session on working with developer tools for debugging and extending the MXNet engine in CLion and posted the screencast in Vimeo in case you are interested. https://vimeo.com/326697272 I also added it to the wiki. Pedro.

Re: [DISCUSS] Rebrand Gluon to MXNet imperative or something MXNet.

2019-03-28 Thread Pedro Larroy
Drost-Fromm wrote: > > I'm a bit confused. Can you point me to the source files that make up Gluon? > Are they part of Apache MxNet? > > Isabel > > > Am 23. März 2019 00:01:56 MEZ schrieb Pedro Larroy > : > >Hi dev@ > > > >We heard feedback from

UB, type narrowing and integer overflows (in this case in mshadow)

2019-03-28 Thread Pedro Larroy
Hi While looking at the failures reporting in this issue: https://github.com/apache/incubator-mxnet/issues/14522 I have noticed that in mshadow when calling the BLAS Engine we are doing narrowing integer conversions from index_t (int64_t) to int, and then operations on dimensions that can

Re: [DISCUSS] Rebrand Gluon to MXNet imperative or something MXNet.

2019-03-22 Thread Pedro Larroy
lace MXNet?" > > > > > 2) We visited customers in a unicorn company who showed > > interests > > > in > > > > MXNet > > > > > but none of the engineers knew the relationship between > > > GluonNLP/GluonCV >

[DISCUSS] Rebrand Gluon to MXNet imperative or something MXNet.

2019-03-22 Thread Pedro Larroy
Hi dev@ We heard feedback from users that the Gluon name is confusing. Some of them don't even know it's MXNet and it's unclear the relationship with MXNet Would it make sense to rebrand Gluon to just MXNet or MXNet imperative? Diluting brands and names is never a good idea. There's also

Re: [Announcement] New Committer - Nicolas Modrzyk

2019-03-21 Thread Pedro Larroy
welcome On Fri, Feb 15, 2019 at 8:15 PM Marco de Abreu wrote: > > Welcome! > > Am Fr., 15. Feb. 2019, 20:23 hat sandeep krishnamurthy < > sandeep.krishn...@gmail.com> geschrieben: > > > Welcome Nicolas. Thank you for all your contributions to the community. > > > > On Fri, Feb 15, 2019 at 9:53

Re: [Announcement] New Committer - Patric Zhao

2019-03-21 Thread Pedro Larroy
Congratulations On Thu, Mar 21, 2019 at 12:04 PM Jake Lee wrote: > > Congrats, Patric! > > Jake > > On Thu, Mar 21, 2019 at 12:03 PM Lin Yuan wrote: > > > Congrats, Patric! > > > > On Thu, Mar 21, 2019 at 10:32 AM Yuxi Hu wrote: > > > > > Congrats, Patric! Well deserved! > > > > > > On Wed,

Re: Rust Client Lib

2019-02-19 Thread Pedro Larroy
; someone > > > > > stumbles across this in the future and wants to pick it up, don't let > > > > > me > > > > > stand in the way! > > > > > > > > > > - Zach > > > > > > > > > > > > > >

Re: [Announce] Runtime feature detection

2019-02-12 Thread Pedro Larroy
t; On Jan 25, 2019, at 9:27 AM, Pedro Larroy > > wrote: > > > > That's Great! There's a PR that we should merge first which > > internalizes the enum inside the library as per Sheng's suggestion. > > > > https://github.com/apache/incubator-mxnet/pull/

Re: [Announcement] New Committer -- Steffen Rochel

2019-02-06 Thread Pedro Larroy
Congrats Steffen. On Tue, Feb 5, 2019 at 7:48 PM Yuan Tang wrote: > > Welcome! > > On Tue, Feb 5, 2019 at 1:46 PM Gavin M. Bell > wrote: > > > Great news! > > > > > > On Tue, Feb 5, 2019 at 6:16 PM Lin Yuan wrote: > > > > > Welcome Steffen! > > > > > > Lin > > > > > > On Mon, Feb 4, 2019 at

Re: Rust Client Lib

2019-01-29 Thread Pedro Larroy
I have been thinking about this and I find really exciting to have Rust bindings and bring a powerful framework like MXNet to the Rust community and to native applications in a convenient Rust crate. I would love to see this happen. I think basically MXNet needs to be wrapped in a Rust crate via

Re: [Announce] Runtime feature detection

2019-01-23 Thread Pedro Larroy
I'm still refining the feature given some late feedback and that it will be public API. I guess with the help of Aaron we will get some nice documentation in, as it's not showing up in the master python API docs. I thought it would be taken automatically from the Python doc. Is this a correct

[Announce] Runtime feature detection

2019-01-22 Thread Pedro Larroy
Hi I'm pleased to announce that runtime feature detection has been merged in master, thanks to Aaron for the merge and the many reviewers who gave feedback on the PR. ( https://github.com/apache/incubator-mxnet/pull/13549 ) As the functionality matures and is exposed through other bindings,

Re: Taxonomy on our cwiki

2019-01-19 Thread Pedro Larroy
+1 On Sat, Jan 19, 2019 at 2:51 PM Zhao, Patric wrote: > > +1, Good idea. > > It's not very easy to find out the related contents since lots of folders in > the website. > > > > -Original Message- > > From: Sheng Zha [mailto:zhash...@apache.org] > > Sent: Saturday, January 19, 2019 3:28

Re: Question about notification on nightly test failures

2019-01-19 Thread Pedro Larroy
t; > Please let me know any feedback on this or concerns. > > Thanks, > Carin > > > On Tue, Jan 15, 2019 at 12:24 PM Pedro Larroy > wrote: > > > Why don't we enable a slack notifier? I think it would be useful to > > interact with notifications from slack direc

Re: Proposal for a recurrent architecture meeting and long term direction

2019-01-19 Thread Pedro Larroy
3:42PM +0100, Pedro Larroy wrote: > > If you wish to join the monthly architecture meeting today, please > > join the hangout below: > > > > https://hangouts.google.com/call/ZXXqJ0ZL5m_dcHOVIeTcAEEE > > Likely I've missed the mail - can you point me to the summary of the above > meeting? > > > Isabel

Re: CI reporting & diagnostics improvements

2019-01-19 Thread Pedro Larroy
Hi Aaron. Looking at the log this could well be due to docker container being rebuilt: http://jenkins.mxnet-ci.amazon-ml.com/job/restricted-website-build/986/consoleFull Trend is around 45 min. http://jenkins.mxnet-ci.amazon-ml.com/job/restricted-website-build/buildTimeTrend I think one of the

Re: Cherry pick bug fix from master branch to v1.4.x

2019-01-15 Thread Pedro Larroy
I would add as well https://github.com/apache/incubator-mxnet/pull/13535 On Tue, Jan 15, 2019 at 4:37 PM kellen sunderland wrote: > > We may want to consider having a new code freeze deadline for RC1. We > could allow users to open PRs against the 1.4.x branch up until this > deadline. > >

Re: Question about notification on nightly test failures

2019-01-15 Thread Pedro Larroy
Why don't we enable a slack notifier? I think it would be useful to interact with notifications from slack directly, including the label bot for example. Pedro. On Sun, Jan 13, 2019 at 1:55 AM Carin Meier wrote: > > Thanks for the explanation Marco :) > > - Carin > > On Sat, Jan 12, 2019 at

Re: Proposal for a recurrent architecture meeting and long term direction

2019-01-14 Thread Pedro Larroy
Hi If you wish to join the monthly architecture meeting today, please join the hangout below: https://hangouts.google.com/call/ZXXqJ0ZL5m_dcHOVIeTcAEEE Regards. Pedro. On Mon, Dec 17, 2018 at 1:18 AM Pedro Larroy wrote: > > Hi > > I think you make good points. We can address y

Re: Apache MXNet v1.4.0 release status

2018-12-17 Thread Pedro Larroy
Hi Steffen Added some notes in your PR for the release notes. In particular, I'm a bit concerned about the status of topology aware communication, since it has open issues and is not being tested in CI. (The tests also fail). I think we should anounce it when it's working properly and it's well

Re: Cambricon MLU support for MXNet.

2018-12-17 Thread Pedro Larroy
Hi Haochong Welcome to MXNet, It's exciting to have additional hardware platforms added and supported in the project. The CI system for MXNet is donated by AWS to the project. We have a small hardware lab with embedded physical hardware like ARM boards including NVidia Jetson which we are

Re: Proposal for a recurrent architecture meeting and long term direction

2018-12-16 Thread Pedro Larroy
y archivable. > > Maybe we can not try more asynchronous way? (RFC discussion in issues. > and/or discuss @dev). Just my two cents and I am not blocking the proposal > > Tianqi > > > On Fri, Dec 14, 2018 at 5:34 AM Pedro Larroy > wrote: > > > Hi MXNetters >

Proposal for a recurrent architecture meeting and long term direction

2018-12-14 Thread Pedro Larroy
Hi MXNetters To address the project growth and increased contributions I'm proposing a monthly meeting / hangout to have community discussions about MXNet architecture and mid / longer term technical directions that require coordination beyond single PRs. TOPICS: The goal of this series is to

Re: Scala standard library is included in the mxnet jar

2018-12-04 Thread Pedro Larroy
Chris, I asked in github before and in other private forums before and I wanted to bring attention to the topic nothing more, you are reading too much between the lines. There's nothing impolite about my question and I think dev@ is the right forum for such conversations. Pedro. On Tue, Dec 4,

Re: Scala standard library is included in the mxnet jar

2018-12-04 Thread Pedro Larroy
In my opinion that's not a good reason and the majority of other libraries in maven don't do that. Is the first time I see a Scala library do that. If every library would include the standard library, asides from the unnecessary bloating you have the problem that at runtime, when you mix

Scala standard library is included in the mxnet jar

2018-12-04 Thread Pedro Larroy
Hi I filled this issue because I observed that the Scala standard library is included in the mxnet jar: https://github.com/apache/incubator-mxnet/issues/13528 I don't think this is correct. Libraries should not include the standard library. Why are we doing this? Pedro.

Re: v1.4.0 status December 3rd

2018-12-04 Thread Pedro Larroy
This is ready to merge: https://github.com/apache/incubator-mxnet/pull/13487 Parent PR was just merged in master.On Tue, Dec 4, 2018 at 6:08 AM Steffen Rochel wrote: > > Dear MXNet community - > thank you very much for everybody effort to resolve the identified issues. > We are down to 4 open

Re: [Announce] Virtualized testing on ARM with Qemu and Docker

2018-12-04 Thread Pedro Larroy
Abreu > wrote: > > > Great work, Pedro! This will help to have a consistent quality on all ARM > > devices. > > > > -Marco > > > > Am Fr., 2. Nov. 2018, 20:02 hat Pedro Larroy > > > > geschrieben: > > > > > Hi MXNet community > >

Re: Adding AMD CPU to CI

2018-11-30 Thread Pedro Larroy
; > Damn, knew i should have double-checked! Oh well it's also carbon neutral. > > On Fri, Nov 30, 2018 at 10:27 AM Pedro Larroy > wrote: > >> Agee with Tianqi and Hao. Adding AMD brings no value and increases >> complexity and CI cost. The instructions sets are the sam

Re: Adding AMD CPU to CI

2018-11-30 Thread Pedro Larroy
Agee with Tianqi and Hao. Adding AMD brings no value and increases complexity and CI cost. The instructions sets are the same. For benchmarking it might make sense though. Pedro > On 30. Nov 2018, at 18:19, Tianqi Chen wrote: > > I still think it is overkill to add AMD CPU to the CI, given

Re: [Announce] Upcoming Apache MXNet (incubating) 1.4.0 release

2018-11-29 Thread Pedro Larroy
Release+Plan+and+Status#ApacheMXNet(incubating)1.4.0ReleasePlanandStatus-OpenPRstotrack> > > Do you have already ETA? > Steffen > > On Thu, Nov 29, 2018 at 6:13 AM Pedro Larroy > wrote: > > > Hi all. > > > > There are two important issues / fixes that sh

Re: [Announce] Upcoming Apache MXNet (incubating) 1.4.0 release

2018-11-29 Thread Pedro Larroy
omp code to be sure but my suspicion is that the > environment variable is only read on startup, and at any rate, better to be > set through the api at runtime > > On Thu, Nov 29, 2018 at 8:11 AM Pedro Larroy > wrote: > > > To be precise, what would be the consequences of not h

Re: [Announce] Upcoming Apache MXNet (incubating) 1.4.0 release

2018-11-29 Thread Pedro Larroy
or a pragma. Definitely we shouldn't be mutating the environment from a different thread from what I understand, which is the likely cause of the random crashes some users are experiencing. Pedro On Thu, Nov 29, 2018 at 5:00 PM Pedro Larroy wrote: > > Chris. The problem is with setenv, not with

Re: [Announce] Upcoming Apache MXNet (incubating) 1.4.0 release

2018-11-29 Thread Pedro Larroy
he omp threads to finish, but the omp threads > belong to the pre-forked process and thus never execute, causing that > forked process to freeze. This behavior has been witnessed before. > > > > > On Thu, Nov 29, 2018 at 6:13 AM Pedro Larroy > wrote: > > > Hi all. > &g

Re: [Announce] Upcoming Apache MXNet (incubating) 1.4.0 release

2018-11-29 Thread Pedro Larroy
Hi all. There are two important issues / fixes that should go in the next release in my radar: 1) https://github.com/apache/incubator-mxnet/pull/13409/files There is a bug in shape inference on CPU when not using MKL, also we are running activation on CPU via MKL when we compile CUDNN+MKLDNN.

Re: using conan to manage Apache Incubator MXNet project dependencies

2018-11-27 Thread Pedro Larroy
ler for which we don't have prebuilt binaries), > > or if you want to build binaries yourself for some another reason, > > then libraries always might be built from source (by passing e.g. "--build > > always", "--build missing" or "--build "

Re: using conan to manage Apache Incubator MXNet project dependencies

2018-11-27 Thread Pedro Larroy
Hi Konstantin Thanks for this contribution. With your proposed changes, when building MXNet we will be pulling binaries for the libraries managed by conan? Pedro. On Mon, Nov 26, 2018 at 11:43 AM Konstantin Ivlev wrote: > > Hello, Ivan, > > could you possibly clarify your question (may be

Re: [Discussion] MXNet CMake build - raise minimal required version

2018-11-22 Thread Pedro Larroy
Thanks Anton for putting this together and your efforts here. I think it's crucial that we maintain and bring the CMake system forward. I have spent a lot of time dealing with CMake issues on different platforms, we really increase developer productivity and platform support by having a

soft relu gradient, is it correct?

2018-11-20 Thread Pedro Larroy
I bumped into the definition of the softrelu gradient: https://github.com/apache/incubator-mxnet/blob/master/src/operator/mshadow_op.h#L170 Which is defined as 1- exp(-x) As we define the forward of the softrelu as the softplus function, shouldn't the gradient be the logistic function? Is my

Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!

2018-11-20 Thread Pedro Larroy
I think this is a big problem, which has blocked us before. I want to point out that you are doing a great thing by avoiding everyone getting blocked by refactoring the pipelines. My concern is that we are kicking the can down the road and not addressing the root cause of the problem with is

Re: Should PR-860 (Use modernized range loops where possible) be reverted?

2018-11-20 Thread Pedro Larroy
Hi all I think we have to make the clear separation between the thread votes on "uniformly adopting C++11 range loops in the MXNet project" and a PR which refactored code to be more legible and with improved variable names. Merging that PR doesn't imply that we have to uniformly adopt the

collecting anonymous statistics

2018-11-19 Thread Pedro Larroy
Hi folks As you know we have a combinatorial explosion of build flavours and different libraries / platforms / environments. Is collecting and gathering anonymous usage statistics as it's done in other open source projects something which would be acceptable for the community? I think it would

Re: Catch divide-by-zero floating number exception in backend

2018-11-12 Thread Pedro Larroy
Hi Could you be specific about the bugs? While we could use this for debug some particular errors as you describe I would think that in the general case you would want to rely on unit testing and conditional checks for very small numbers on the denominator if you can’t have a NaN. I think we

Developer setup in AWS EC2 & menu based tool

2018-11-11 Thread Pedro Larroy
Hi Folks! We got the feedback that reproducing test results on EC2 was complex and time consuming. I have cleaned up my personal scripts to provision a development instance: https://cwiki.apache.org/confluence/display/MXNET/MXNet+Developer+setup+on+AWS+EC2

Unit test breakdown by time

2018-11-07 Thread Pedro Larroy
Hi I made a quick breakdown of time spent on unit tests (in seconds) per test and per test class. I run CPU tests on an m1 instance. As you can see the slowest test classes are: test_operator, 630.92699 test_gluon, 261.106017 test_profiler, 159.427002

<    1   2   3   4   >