Re: [Proposal] New operator graph for MXNet

2019-05-17 Thread Pedro Larroy
ows > - The ability of interpolation with multi-language frontend, e.g. being > able to prototype graph optimizations in python/scala/clojure if needed. > > Adding these support needs significant engineering effort, and I do hope we > only have to do it once. While I don't want to force a

Re: [Proposal] New operator graph for MXNet

2019-05-15 Thread Pedro Larroy
> Adding these support needs significant engineering effort, and I do hope we > only have to do it once. While I don't want to force any conclusion here, > I do think Relay is one such candidate. > > Tianqi > > > On Tue, May 14, 2019 at 5:58 PM Pedro Larroy > wrote: &

Re: [Proposal] New operator graph for MXNet

2019-05-14 Thread Pedro Larroy
scenario that you are concerned about. Let me know what you think. Pedro. On Tue, May 14, 2019 at 5:50 PM Pedro Larroy wrote: > > Hi Tianqi > > Thanks for the quick response. > > Could you point to examples where graph.h is being exposed which would > not be possible with what I p

Re: [Proposal] New operator graph for MXNet

2019-05-14 Thread Pedro Larroy
at 5:06 PM Sheng Zha wrote: > > > Hi Pedro, > > > > Thanks for taking the inititaive. Skimming through the design doc, I > > didn't see comparison with existing solutions such as relay in tvm, which > > is already a dependency of mxnet already. Could you el

Re: [Proposal] New operator graph for MXNet

2019-05-14 Thread Pedro Larroy
t; -sz > > On 2019/05/14 23:49:30, Pedro Larroy wrote: > > Hi dev@ > > > > As a result of my deep dives on the graph machinery I have created a > > new proposal to improve the operator graph in MXNet. > > > > This would mean superseding the use of NN

[Proposal] New operator graph for MXNet

2019-05-14 Thread Pedro Larroy
Hi dev@ As a result of my deep dives on the graph machinery I have created a new proposal to improve the operator graph in MXNet. This would mean superseding the use of NNVM Graph in MXNet and having a new implementation that we can use to simplify a lot of code and do powerful graph

Re: assimilation of mshadow into the MXNet codebase

2019-05-14 Thread Pedro Larroy
Hi Sheng. Do you need some help with this? Do we plan to have this for 1.5? Pedro. On Wed, Apr 24, 2019 at 4:26 PM Pedro Larroy wrote: > > Thanks. Great to read. > > On Wed, Apr 24, 2019 at 2:19 PM Sheng Zha wrote: > > > > The community has agreed to donate mshadow t

Re: Python2 End of Life

2019-05-14 Thread Pedro Larroy
+1 Let python2 rest, let's simplify our infrastructure and need to support old Python versions. On Mon, May 13, 2019 at 1:58 PM Jake Lee wrote: > > +1 Recently I upgraded the Numpy version and found out that Pylint had > false alarm on it. The Pylint fix is only available on Python3. So I >

Re: [Announcement] New Committer - Zach Kimberg

2019-05-13 Thread Pedro Larroy
Congratulations On Thu, May 9, 2019 at 11:29 AM Chaitanya Bapat wrote: > > Congratulations Zachary! Way to go! > > On Thu, 9 May 2019 at 14:01, Carin Meier wrote: > > > Congrats! > > > > On Thu, May 9, 2019 at 1:41 PM Per da Silva wrote: > > > > > Nice one! Congratulations =) > > > > > > On

Re: [VOTE] Release Apache MXNet (incubating) version 1.4.1.rc0

2019-05-01 Thread Pedro Larroy
+1 (non-binding) Tried CPU build + C++ tests + 714 Python unit tests in 605s. ARMv7 build + small unit test in QEMU + ARMv8 builds. Thanks. Regards Pedro. On Wed, May 1, 2019 at 10:41 AM Qing Lan wrote: > > +1 (binding) > > build from source works for OSX and Ubuntu CPU > Scala build/test

Re: [Announcement] New Committer - Hao Jin

2019-05-01 Thread Pedro Larroy
Congrats! On Wed, May 1, 2019 at 11:08 AM Lin Yuan wrote: > > Congrats! > > On Tue, Apr 30, 2019 at 11:28 PM Alex Zai wrote: > > > Congrats Hao! > > > > On Tue, Apr 30, 2019 at 10:53 PM Steffen Rochel > > wrote: > > > > > congratulation Hao! > > > > > > On Tue, Apr 30, 2019 at 8:05 AM MiraiWK

Re: Clojure MXNet Monthly Update

2019-04-29 Thread Pedro Larroy
nice! I would suggest that we use the medium account for MXNet and have a "this month in MXNet" as they do in other open source projects and just have a clojure section? I think it gives nice visibility to the project and attracts contributors. Maybe just copy your updates to the "main one"?

Re: docs updates

2019-04-25 Thread Pedro Larroy
Hi Aaron. I'm no design expert, but wouldn't smoothing out the boxes a bit rounding the corners would make the design more pleasant? As an old fart learning from Knuth books, I'm all pro having documentation as close to the source as possible. Could you maybe put two concrete examples so the

Re: [Announcement] New Committer - Wang Jiajun

2019-04-25 Thread Pedro Larroy
Welcome! On Tue, Apr 16, 2019 at 6:34 PM kellen sunderland wrote: > > Welcome! Very impressed with the work fixing memory leaks so far. > > On Tue, Apr 16, 2019 at 9:14 AM Carin Meier wrote: > > > Congrats! > > > > On Tue, Apr 16, 2019 at 11:58 AM Anirudh Subramanian < > >

DNS failures in jenkins

2019-04-25 Thread Pedro Larroy
Hi I see some DNS resolution failures on jenkins, I think this is the cause of jenkins not reporting the build status sometimes. What dns server are we using in the master? should we add a couple of secondary resolvers to remediate?

Re: assimilation of mshadow into the MXNet codebase

2019-04-24 Thread Pedro Larroy
> > > > > > > mshadow is mostly used for tensor arithmetic. There have been > > > > > discussions > > > > > about including it within mxnet. I think it is a good idea. > > > > > > > > > > As a more long term solut

Re: Benchmarking MXNet with different compilers and different OpenMP implementations (results)

2019-04-12 Thread Pedro Larroy
Are there any updates on this? This is still affecting multiprocessing, some tests hang: rces. For information on submitting this issue, please see https://bugs.llvm.org/. [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=2124604270 to reproduce. Assertion failure at

Re: duplicated nnvm code

2019-04-12 Thread Pedro Larroy
sense to reause across projects. @Tianqi? On Thu, Apr 11, 2019 at 10:16 PM Junru Shao wrote: > > We should remove 3rdparty/tvm/nnvm/gradient.cc.o imo > > On Thu, Apr 11, 2019 at 6:44 PM Pedro Larroy > wrote: > > > Hi > > > > I found that src/nnvm and 3rdpart

duplicated nnvm code

2019-04-11 Thread Pedro Larroy
Hi I found that src/nnvm and 3rdparty/tvm/nnvm/src/pass/ has duplicated code that we are linking in: ./CMakeFiles/mxnet_static.dir/3rdparty/tvm/nnvm/src/pass/gradient.cc.o ./CMakeFiles/mxnet_static.dir/src/nnvm/gradient.cc.o This can potentially cause problems when linking. The symbol that

Re: [MXNET 2.0 Wishlist] [DISCUSS] Backend choices during runtime

2019-04-11 Thread Pedro Larroy
> > We have a systematic solution to go without ABI headache. I am struggling > > with some errants, and will share our proposal here as soon as I could. > > This will be very interesting topic to discuss. Let's work hard together > > and make it perfect :-) > > &g

Re: [MXNET 2.0 Wishlist] [DISCUSS] Backend choices during runtime

2019-04-11 Thread Pedro Larroy
Thanks Marco for raising this issue. I think we can certainly do some improvements in modularization and build. At the same time Tianqi's point of view is important to consider and on point. I see a high risk of overengineering in such endeavor. I also see increased complexity, difficulty

Re: assimilation of mshadow into the MXNet codebase

2019-04-08 Thread Pedro Larroy
> > > > Do you have a link to both of these proposals? > > > > > > > > > > > > On Fri, Apr 5, 2019 at 20:14 Anirudh Acharya > > > > > > > > > > > > wrote: > > > > > > >

assimilation of mshadow into the MXNet codebase

2019-04-05 Thread Pedro Larroy
Hi Some developers have noticed that working in mshadow is cumbersome as it's a 3rdparty subrepo. Since mshadow is a bunch of headers which don't have much of independent tests / library functionality, me and other developers believe that it would be good to assimilate this code in the

Re: Discussing plans for next MXNet releases

2019-04-02 Thread Pedro Larroy
Great initiative. I would like to add the issue that tracks APIs that we would like to break for 2.0 so we can take the chance to streamline and improve customer facing code: https://github.com/apache/incubator-mxnet/issues/9686 I would be happy to volunteer for 2.0 release with assistance from

Developer tools for MXNet

2019-03-28 Thread Pedro Larroy
Hi developers! We did a session on working with developer tools for debugging and extending the MXNet engine in CLion and posted the screencast in Vimeo in case you are interested. https://vimeo.com/326697272 I also added it to the wiki. Pedro.

Re: [DISCUSS] Rebrand Gluon to MXNet imperative or something MXNet.

2019-03-28 Thread Pedro Larroy
Drost-Fromm wrote: > > I'm a bit confused. Can you point me to the source files that make up Gluon? > Are they part of Apache MxNet? > > Isabel > > > Am 23. März 2019 00:01:56 MEZ schrieb Pedro Larroy > : > >Hi dev@ > > > >We heard feedback from

UB, type narrowing and integer overflows (in this case in mshadow)

2019-03-28 Thread Pedro Larroy
Hi While looking at the failures reporting in this issue: https://github.com/apache/incubator-mxnet/issues/14522 I have noticed that in mshadow when calling the BLAS Engine we are doing narrowing integer conversions from index_t (int64_t) to int, and then operations on dimensions that can

Re: [DISCUSS] Rebrand Gluon to MXNet imperative or something MXNet.

2019-03-22 Thread Pedro Larroy
lace MXNet?" > > > > > 2) We visited customers in a unicorn company who showed > > interests > > > in > > > > MXNet > > > > > but none of the engineers knew the relationship between > > > GluonNLP/GluonCV >

[DISCUSS] Rebrand Gluon to MXNet imperative or something MXNet.

2019-03-22 Thread Pedro Larroy
Hi dev@ We heard feedback from users that the Gluon name is confusing. Some of them don't even know it's MXNet and it's unclear the relationship with MXNet Would it make sense to rebrand Gluon to just MXNet or MXNet imperative? Diluting brands and names is never a good idea. There's also

Re: [Announcement] New Committer - Nicolas Modrzyk

2019-03-21 Thread Pedro Larroy
welcome On Fri, Feb 15, 2019 at 8:15 PM Marco de Abreu wrote: > > Welcome! > > Am Fr., 15. Feb. 2019, 20:23 hat sandeep krishnamurthy < > sandeep.krishn...@gmail.com> geschrieben: > > > Welcome Nicolas. Thank you for all your contributions to the community. > > > > On Fri, Feb 15, 2019 at 9:53

Re: [Announcement] New Committer - Patric Zhao

2019-03-21 Thread Pedro Larroy
Congratulations On Thu, Mar 21, 2019 at 12:04 PM Jake Lee wrote: > > Congrats, Patric! > > Jake > > On Thu, Mar 21, 2019 at 12:03 PM Lin Yuan wrote: > > > Congrats, Patric! > > > > On Thu, Mar 21, 2019 at 10:32 AM Yuxi Hu wrote: > > > > > Congrats, Patric! Well deserved! > > > > > > On Wed,

Re: Rust Client Lib

2019-02-19 Thread Pedro Larroy
; someone > > > > > stumbles across this in the future and wants to pick it up, don't let > > > > > me > > > > > stand in the way! > > > > > > > > > > - Zach > > > > > > > > > > > > > >

Re: [Announce] Runtime feature detection

2019-02-12 Thread Pedro Larroy
t; On Jan 25, 2019, at 9:27 AM, Pedro Larroy > > wrote: > > > > That's Great! There's a PR that we should merge first which > > internalizes the enum inside the library as per Sheng's suggestion. > > > > https://github.com/apache/incubator-mxnet/pull/

Re: [Announcement] New Committer -- Steffen Rochel

2019-02-06 Thread Pedro Larroy
Congrats Steffen. On Tue, Feb 5, 2019 at 7:48 PM Yuan Tang wrote: > > Welcome! > > On Tue, Feb 5, 2019 at 1:46 PM Gavin M. Bell > wrote: > > > Great news! > > > > > > On Tue, Feb 5, 2019 at 6:16 PM Lin Yuan wrote: > > > > > Welcome Steffen! > > > > > > Lin > > > > > > On Mon, Feb 4, 2019 at

Re: Rust Client Lib

2019-01-29 Thread Pedro Larroy
I have been thinking about this and I find really exciting to have Rust bindings and bring a powerful framework like MXNet to the Rust community and to native applications in a convenient Rust crate. I would love to see this happen. I think basically MXNet needs to be wrapped in a Rust crate via

Re: [Announce] Runtime feature detection

2019-01-25 Thread Pedro Larroy
. Thanks a lot! Pedro. On Fri, Jan 25, 2019 at 2:22 PM Iblis Lin wrote: > > Hi, > > I added the Julia binding for it. > PR is here: > https://github.com/apache/incubator-mxnet/pull/13992 > > Iblis Lin > 林峻頤 > > On 1/23/19 12:39 AM, Pedro Larroy wrote: > > Hi >

Re: [Announce] Runtime feature detection

2019-01-23 Thread Pedro Larroy
I'm still refining the feature given some late feedback and that it will be public API. I guess with the help of Aaron we will get some nice documentation in, as it's not showing up in the master python API docs. I thought it would be taken automatically from the Python doc. Is this a correct

[Announce] Runtime feature detection

2019-01-22 Thread Pedro Larroy
Hi I'm pleased to announce that runtime feature detection has been merged in master, thanks to Aaron for the merge and the many reviewers who gave feedback on the PR. ( https://github.com/apache/incubator-mxnet/pull/13549 ) As the functionality matures and is exposed through other bindings,

Re: Taxonomy on our cwiki

2019-01-19 Thread Pedro Larroy
+1 On Sat, Jan 19, 2019 at 2:51 PM Zhao, Patric wrote: > > +1, Good idea. > > It's not very easy to find out the related contents since lots of folders in > the website. > > > > -Original Message- > > From: Sheng Zha [mailto:zhash...@apache.org] > > Sent: Saturday, January 19, 2019 3:28

Re: Question about notification on nightly test failures

2019-01-19 Thread Pedro Larroy
t; > Please let me know any feedback on this or concerns. > > Thanks, > Carin > > > On Tue, Jan 15, 2019 at 12:24 PM Pedro Larroy > wrote: > > > Why don't we enable a slack notifier? I think it would be useful to > > interact with notifications from slack direc

Re: Proposal for a recurrent architecture meeting and long term direction

2019-01-19 Thread Pedro Larroy
3:42PM +0100, Pedro Larroy wrote: > > If you wish to join the monthly architecture meeting today, please > > join the hangout below: > > > > https://hangouts.google.com/call/ZXXqJ0ZL5m_dcHOVIeTcAEEE > > Likely I've missed the mail - can you point me to the summary of the above > meeting? > > > Isabel

Re: CI reporting & diagnostics improvements

2019-01-19 Thread Pedro Larroy
Hi Aaron. Looking at the log this could well be due to docker container being rebuilt: http://jenkins.mxnet-ci.amazon-ml.com/job/restricted-website-build/986/consoleFull Trend is around 45 min. http://jenkins.mxnet-ci.amazon-ml.com/job/restricted-website-build/buildTimeTrend I think one of the

Re: Cherry pick bug fix from master branch to v1.4.x

2019-01-15 Thread Pedro Larroy
I would add as well https://github.com/apache/incubator-mxnet/pull/13535 On Tue, Jan 15, 2019 at 4:37 PM kellen sunderland wrote: > > We may want to consider having a new code freeze deadline for RC1. We > could allow users to open PRs against the 1.4.x branch up until this > deadline. > >

Re: Question about notification on nightly test failures

2019-01-15 Thread Pedro Larroy
Why don't we enable a slack notifier? I think it would be useful to interact with notifications from slack directly, including the label bot for example. Pedro. On Sun, Jan 13, 2019 at 1:55 AM Carin Meier wrote: > > Thanks for the explanation Marco :) > > - Carin > > On Sat, Jan 12, 2019 at

Re: Proposal for a recurrent architecture meeting and long term direction

2019-01-14 Thread Pedro Larroy
Hi If you wish to join the monthly architecture meeting today, please join the hangout below: https://hangouts.google.com/call/ZXXqJ0ZL5m_dcHOVIeTcAEEE Regards. Pedro. On Mon, Dec 17, 2018 at 1:18 AM Pedro Larroy wrote: > > Hi > > I think you make good points. We can address y

Re: Order of includes in cpplint

2019-01-08 Thread Pedro Larroy
> As MXNet follows this style, I guess we shouldn't break it unless we have > some problems. Do you have such a case that need the change? > > Thanks, > Zhennan > > -Original Message- > From: Pedro Larroy [mailto:pedro.larroy.li...@gmail.com] > Sent: Wednesday,

Order of includes in cpplint

2019-01-08 Thread Pedro Larroy
Hi MXNet community cpplint seems to complain when the order of includes is not [own, system, other] But the general best practice in C++ is [own, project, 3rd party, system] for the reasons explained in this stackoverflow answer: ( https://stackoverflow.com/questions/614302/c-header-order ) A

Re: Apache MXNet v1.4.0 release status

2018-12-17 Thread Pedro Larroy
Hi Steffen Added some notes in your PR for the release notes. In particular, I'm a bit concerned about the status of topology aware communication, since it has open issues and is not being tested in CI. (The tests also fail). I think we should anounce it when it's working properly and it's well

Re: Cambricon MLU support for MXNet.

2018-12-17 Thread Pedro Larroy
Hi Haochong Welcome to MXNet, It's exciting to have additional hardware platforms added and supported in the project. The CI system for MXNet is donated by AWS to the project. We have a small hardware lab with embedded physical hardware like ARM boards including NVidia Jetson which we are

Re: Proposal for a recurrent architecture meeting and long term direction

2018-12-16 Thread Pedro Larroy
y archivable. > > Maybe we can not try more asynchronous way? (RFC discussion in issues. > and/or discuss @dev). Just my two cents and I am not blocking the proposal > > Tianqi > > > On Fri, Dec 14, 2018 at 5:34 AM Pedro Larroy > wrote: > > > Hi MXNetters >

Proposal for a recurrent architecture meeting and long term direction

2018-12-14 Thread Pedro Larroy
Hi MXNetters To address the project growth and increased contributions I'm proposing a monthly meeting / hangout to have community discussions about MXNet architecture and mid / longer term technical directions that require coordination beyond single PRs. TOPICS: The goal of this series is to

Re: Scala standard library is included in the mxnet jar

2018-12-04 Thread Pedro Larroy
Chris, I asked in github before and in other private forums before and I wanted to bring attention to the topic nothing more, you are reading too much between the lines. There's nothing impolite about my question and I think dev@ is the right forum for such conversations. Pedro. On Tue, Dec 4,

Re: Scala standard library is included in the mxnet jar

2018-12-04 Thread Pedro Larroy
In my opinion that's not a good reason and the majority of other libraries in maven don't do that. Is the first time I see a Scala library do that. If every library would include the standard library, asides from the unnecessary bloating you have the problem that at runtime, when you mix

Scala standard library is included in the mxnet jar

2018-12-04 Thread Pedro Larroy
Hi I filled this issue because I observed that the Scala standard library is included in the mxnet jar: https://github.com/apache/incubator-mxnet/issues/13528 I don't think this is correct. Libraries should not include the standard library. Why are we doing this? Pedro.

Re: v1.4.0 status December 3rd

2018-12-04 Thread Pedro Larroy
This is ready to merge: https://github.com/apache/incubator-mxnet/pull/13487 Parent PR was just merged in master.On Tue, Dec 4, 2018 at 6:08 AM Steffen Rochel wrote: > > Dear MXNet community - > thank you very much for everybody effort to resolve the identified issues. > We are down to 4 open

Re: [Announce] Virtualized testing on ARM with Qemu and Docker

2018-12-04 Thread Pedro Larroy
Abreu > wrote: > > > Great work, Pedro! This will help to have a consistent quality on all ARM > > devices. > > > > -Marco > > > > Am Fr., 2. Nov. 2018, 20:02 hat Pedro Larroy > > > > geschrieben: > > > > > Hi MXNet community > >

Re: Adding AMD CPU to CI

2018-11-30 Thread Pedro Larroy
; > Damn, knew i should have double-checked! Oh well it's also carbon neutral. > > On Fri, Nov 30, 2018 at 10:27 AM Pedro Larroy > wrote: > >> Agee with Tianqi and Hao. Adding AMD brings no value and increases >> complexity and CI cost. The instructions sets are the sam

Re: Adding AMD CPU to CI

2018-11-30 Thread Pedro Larroy
Agee with Tianqi and Hao. Adding AMD brings no value and increases complexity and CI cost. The instructions sets are the same. For benchmarking it might make sense though. Pedro > On 30. Nov 2018, at 18:19, Tianqi Chen wrote: > > I still think it is overkill to add AMD CPU to the CI, given

Re: [Announce] Upcoming Apache MXNet (incubating) 1.4.0 release

2018-11-29 Thread Pedro Larroy
Release+Plan+and+Status#ApacheMXNet(incubating)1.4.0ReleasePlanandStatus-OpenPRstotrack> > > Do you have already ETA? > Steffen > > On Thu, Nov 29, 2018 at 6:13 AM Pedro Larroy > wrote: > > > Hi all. > > > > There are two important issues / fixes that sh

Re: [Announce] Upcoming Apache MXNet (incubating) 1.4.0 release

2018-11-29 Thread Pedro Larroy
omp code to be sure but my suspicion is that the > environment variable is only read on startup, and at any rate, better to be > set through the api at runtime > > On Thu, Nov 29, 2018 at 8:11 AM Pedro Larroy > wrote: > > > To be precise, what would be the consequences of not h

Re: [Announce] Upcoming Apache MXNet (incubating) 1.4.0 release

2018-11-29 Thread Pedro Larroy
or a pragma. Definitely we shouldn't be mutating the environment from a different thread from what I understand, which is the likely cause of the random crashes some users are experiencing. Pedro On Thu, Nov 29, 2018 at 5:00 PM Pedro Larroy wrote: > > Chris. The problem is with setenv, not with

Re: [Announce] Upcoming Apache MXNet (incubating) 1.4.0 release

2018-11-29 Thread Pedro Larroy
he omp threads to finish, but the omp threads > belong to the pre-forked process and thus never execute, causing that > forked process to freeze. This behavior has been witnessed before. > > > > > On Thu, Nov 29, 2018 at 6:13 AM Pedro Larroy > wrote: > > > Hi all. > &g

Re: [Announce] Upcoming Apache MXNet (incubating) 1.4.0 release

2018-11-29 Thread Pedro Larroy
Hi all. There are two important issues / fixes that should go in the next release in my radar: 1) https://github.com/apache/incubator-mxnet/pull/13409/files There is a bug in shape inference on CPU when not using MKL, also we are running activation on CPU via MKL when we compile CUDNN+MKLDNN.

Re: using conan to manage Apache Incubator MXNet project dependencies

2018-11-27 Thread Pedro Larroy
ler for which we don't have prebuilt binaries), > > or if you want to build binaries yourself for some another reason, > > then libraries always might be built from source (by passing e.g. "--build > > always", "--build missing" or "--build "

Re: using conan to manage Apache Incubator MXNet project dependencies

2018-11-27 Thread Pedro Larroy
Hi Konstantin Thanks for this contribution. With your proposed changes, when building MXNet we will be pulling binaries for the libraries managed by conan? Pedro. On Mon, Nov 26, 2018 at 11:43 AM Konstantin Ivlev wrote: > > Hello, Ivan, > > could you possibly clarify your question (may be

Re: [Discussion] MXNet CMake build - raise minimal required version

2018-11-22 Thread Pedro Larroy
Thanks Anton for putting this together and your efforts here. I think it's crucial that we maintain and bring the CMake system forward. I have spent a lot of time dealing with CMake issues on different platforms, we really increase developer productivity and platform support by having a

soft relu gradient, is it correct?

2018-11-20 Thread Pedro Larroy
I bumped into the definition of the softrelu gradient: https://github.com/apache/incubator-mxnet/blob/master/src/operator/mshadow_op.h#L170 Which is defined as 1- exp(-x) As we define the forward of the softrelu as the softplus function, shouldn't the gradient be the logistic function? Is my

Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!

2018-11-20 Thread Pedro Larroy
I think this is a big problem, which has blocked us before. I want to point out that you are doing a great thing by avoiding everyone getting blocked by refactoring the pipelines. My concern is that we are kicking the can down the road and not addressing the root cause of the problem with is

Re: Should PR-860 (Use modernized range loops where possible) be reverted?

2018-11-20 Thread Pedro Larroy
Hi all I think we have to make the clear separation between the thread votes on "uniformly adopting C++11 range loops in the MXNet project" and a PR which refactored code to be more legible and with improved variable names. Merging that PR doesn't imply that we have to uniformly adopt the

collecting anonymous statistics

2018-11-19 Thread Pedro Larroy
Hi folks As you know we have a combinatorial explosion of build flavours and different libraries / platforms / environments. Is collecting and gathering anonymous usage statistics as it's done in other open source projects something which would be acceptable for the community? I think it would

Re: Catch divide-by-zero floating number exception in backend

2018-11-12 Thread Pedro Larroy
Hi Could you be specific about the bugs? While we could use this for debug some particular errors as you describe I would think that in the general case you would want to rely on unit testing and conditional checks for very small numbers on the denominator if you can’t have a NaN. I think we

Developer setup in AWS EC2 & menu based tool

2018-11-11 Thread Pedro Larroy
Hi Folks! We got the feedback that reproducing test results on EC2 was complex and time consuming. I have cleaned up my personal scripts to provision a development instance: https://cwiki.apache.org/confluence/display/MXNET/MXNet+Developer+setup+on+AWS+EC2

Unit test breakdown by time

2018-11-07 Thread Pedro Larroy
Hi I made a quick breakdown of time spent on unit tests (in seconds) per test and per test class. I run CPU tests on an m1 instance. As you can see the slowest test classes are: test_operator, 630.92699 test_gluon, 261.106017 test_profiler, 159.427002

Re: [DISCUSS] Speedup non-code PR in CI

2018-11-06 Thread Pedro Larroy
It has been raised but there are practical complications about introducing an additional layer of logic for skipping CI in some scenarios. How many of these PRs do we have which will justify investing human effort on optimizing an automated process? How much effort shall it be dedicated to this

[Announce] Virtualized testing on ARM with Qemu and Docker

2018-11-02 Thread Pedro Larroy
Hi MXNet community AI on MCUs can enable cheaper, lower power, better privacy and lower latency applications. There’s an estimate of more than 20 billion connected devices to be deployed in 2020 and a part of them will do some amount of AI / ML tasks. Testing in embedded devices is very

Re: Coverity scan

2018-11-02 Thread Pedro Larroy
Thanks a lot, I think is very beneficial that we invest in these kind of tooling for code quality. As a developer I wonder, do we have actionable items for looking at / fixing these issues or right now is done in an informational / good will basis? Is there a way to colorize this output? Pedro.

possible bug in gpu_topology.h ComputeDepth ?

2018-11-01 Thread Pedro Larroy
Hi I'm investigating this issue: https://github.com/apache/incubator-mxnet/issues/12994 To me this code seems suspicious, as it doesn't do what is stated in the comment. https://github.com/apache/incubator-mxnet/blob/master/src/kvstore/gpu_topology.h#L577 I don't think the depth of the binary

Re: [VOTE] - Adopt "Become a Committer and PPMC Member" Document

2018-11-01 Thread Pedro Larroy
+1 non-binding. Thanks for driving this, looking forward to see the positive impact. On Mon, Oct 29, 2018 at 11:47 PM Carin Meier wrote: > This vote is to adopt the document > >

[Discuss] Feature detection at runtime / test skipping depending on features

2018-11-01 Thread Pedro Larroy
Hi There are some tests that fail when some features are not compiled in, such as Opencv. In some cases we skip the test according to some precondition such as: @unittest.skipIf(not graphviz_exists(), I would propose that we have a Python module that exports a set of methods to check what

Re: [DISCUSS] - Revisions to Committer Criteria

2018-10-25 Thread Pedro Larroy
This is the first hangout that I was able to attend, I liked the format and found them valuable. Thanks for organizing and publishing the notes. Looking forward to the next one. Pedro On Thu, Oct 25, 2018 at 6:44 AM Steffen Rochel wrote: > Carin - please see > >

Re: Up-streaming MXNet HIP Port

2018-10-24 Thread Pedro Larroy
Hi Srihari Thanks for the document. We could add it to the relevant section in the wiki so it's easier to keep track whenever you feel comfortable with it. https://cwiki.apache.org/confluence/display/MXNET/Design+Proposals The calls to synchronize and wait looks very similar. Shall we use

Re: [Discussion] PMC and Committer Courtesy: Only Propose Candidate in a Different Organization

2018-10-23 Thread Pedro Larroy
for nominees affiliated with Amazon AWS. > > Please comment on the proposal (see thread "[DISCUSS] - Revisions to > Committer Criteria). > > Steffen > > On Tue, Oct 23, 2018 at 7:09 AM Pedro Larroy > > wrote: > > > Hi > > > > Tianqi, there's a sayin

Re: [Discussion] PMC and Committer Courtesy: Only Propose Candidate in a Different Organization

2018-10-23 Thread Pedro Larroy
Hi Tianqi, there's a saying that the road to hell is paved with good intentions. I think most of us here are enthusiastic about increasing diversity of contributions to this project and are working actively towards this. Having any kind of positive or negative discrimination seems to me like it

Re: Include MKLDNN into default mxnet pip package

2018-10-19 Thread Pedro Larroy
I did pip install mxnet-mkl==1.3.1b20181018 on an AMD Ryzen 1950X and unit tests are passing. Is this build using AVX512? in /proc/cpuinfo I see only "avx" flag. There's no "avx2" like on recent intel cpus. Pedro. On Fri, Oct 19, 2018 at 5:12 PM Hagay Lupesko wrote: > Awesome collaborative

Re: [LAZY VOTE]: rename dockerfiles s/.build.//

2018-10-19 Thread Pedro Larroy
t; > making the change I see more potential risk than obvious benefits. I > > also > > > feel that this change will make the difference between the runtime > docker > > > files and the CI docker files less clear to users, not more clear. In > > > ge

Re: Include MKLDNN into default mxnet pip package

2018-10-18 Thread Pedro Larroy
Very nice! Pedro > On 17. Oct 2018, at 23:12, Alfredo Luque > wrote: > > This is huge. Thanks for working on this. Is there a similar plan with eg; > tensor-rt support being ported into the main cuda-9.x packages? > > On October 17, 2018 at 2:10:20 PM, Alex Zai (aza...@gmail.com) wrote: >

Re: Storing PGP Key for Publishing packages

2018-10-17 Thread Pedro Larroy
Do nightly artifacts need to be signed? For releases what you wrote and what Apache recommends makes total sense. Thus artifacts from cd can’t be signed manually. Pedro > On 17. Oct 2018, at 22:29, Naveen Swamy wrote: > > I am collaborating with Zach Kimberg and Qing to work on automatic ( >

Re: [LAZY VOTE]: rename dockerfiles s/.build.//

2018-10-17 Thread Pedro Larroy
lightweight and suited to their task > as possible. > > On Wed, Oct 17, 2018, 1:58 AM Hagay Lupesko wrote: > > > The PR provides a good explanation of this change and all code updates. > > LGTM. > > > > On Tue, Oct 16, 2018 at 8:41 AM Pedro Larroy < > pedr

Re: Call for participation: evaluate Java API

2018-10-16 Thread Pedro Larroy
To play around we use the java api branch? is there a link to some example code? Thanks. On Fri, Oct 12, 2018 at 9:16 PM Davydenko, Denis < dzianis.davydze...@gmail.com> wrote: > Not so long ago there was a design shared for MXNet Java API: [1] > > In a couple of days we are going to have

Re: Reproducing test failures on CI

2018-10-16 Thread Pedro Larroy
XNET/Reproducing+test+results But seems not many people read it, also it doesn't solve provisioning the instance and installing the initial dependencies. Pedro. On Mon, Oct 15, 2018 at 8:58 PM Naveen Swamy wrote: > Timur, > Here is a meetup Scheduled for 23rd October in London, where Pe

[LAZY VOTE]: rename dockerfiles s/.build.//

2018-10-16 Thread Pedro Larroy
Hi I would like to rename the dockerfiles since they are used as a runtime environment and not only as build as they were initially intended. More info about the change in this PR: https://github.com/apache/incubator-mxnet/pull/12423/files Pedro.

Re: Maturity Model and Graduation

2018-10-06 Thread Pedro Larroy
our guidance if the assessment should be > discussed further within PPMC, on private@ and next steps. > > Steffen > > On Fri, Sep 28, 2018 at 1:53 PM Pedro Larroy > > wrote: > > > So Isabel, are you saying that if we publish a clearer TODO list or > > contribution

Re: Which merge option to use on the Import Julia binding PR?

2018-10-05 Thread Pedro Larroy
So - Does the regular merge seem like a good > option? > > > > > > > >> > >> > > If so, what is the best way to make that happen? > > > > > > > >> > >> > > > > > > > > > >> > >>

Re: [DISCUSS] Use modernized C++11 range loops uniformly throughout the project

2018-10-03 Thread Pedro Larroy
+1 @Chris: do you have data on the performance difference? As far as I know there's a "rewrite rule" like the one between lambdas and C++ functors, so performance should be very well defined, but maybe there's something that you are point out that we are missing. Having a consistent and concise

Re: CUDNN algorithm selection failure

2018-10-03 Thread Pedro Larroy
ugh. The PR that introduced that test > is https://github.com/apache/incubator-mxnet/pull/10921 - it was merged > two > days ago. > > Best regards, > Marco > > On Tue, Oct 2, 2018 at 8:43 AM Pedro Larroy > wrote: > > > Thanks for checking Lin. If it happens again we wil

Re: CUDNN algorithm selection failure

2018-10-02 Thread Pedro Larroy
I also suspect it was due to resource usage limit on ci Instance. > > On Mon, Oct 1, 2018 at 10:40 PM Pedro Larroy > > wrote: > > > It doesn't look like flakiness to me at first sight. I think it might be > > related to resource usage / allocation / leak

Re: CUDNN algorithm selection failure

2018-10-01 Thread Pedro Larroy
Are you > suggesting there is some flakiness in the master branch too? > > Thanks, > > Lin > > On Mon, Oct 1, 2018 at 4:55 PM Pedro Larroy > wrote: > > > Hi > > > > I saw this failure on CI: > > > > > http://jenkins.mxnet-ci.amazon-ml.com/blue/organi

Re: Time out for Travis CI

2018-10-01 Thread Pedro Larroy
I think there's two approaches that we can take to mitigate the build & test time problem, in one hand use a paid travis CI plan, in other improve the unit tests in suites and only run a core set of tests, as we should do on devices, but on this case we reduce coverage.

CUDNN algorithm selection failure

2018-10-01 Thread Pedro Larroy
Hi I saw this failure on CI: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/master/1697/pipeline Have you seen other cases where we fail to select the best CUDNN algorithm? In which circumstances this could happen, and do you think is a good idea to have

Re: Maturity Model and Graduation

2018-09-28 Thread Pedro Larroy
So Isabel, are you saying that if we publish a clearer TODO list or contributions needed material we might get more contribution there? One thing that I like from other projects is to make a list of low-hanging fruit issues or easy contributions that newcomers can pick to get familiar with the

  1   2   3   >