Re: ICLAs?

2018-07-12 Thread Marco de Abreu
Hi Sergio, we are only collecting ICLAs from people if they become a committer. Otherwise, we don't request any. -Marco On Thu, Jul 12, 2018 at 8:07 PM Sergio Fernández wrote: > Hi, > > one detail I've notices: is the Podling collecting all ICLA when people > contribute to the project? > > I c

Re: Disabling flaky tests

2018-07-12 Thread Marco de Abreu
Quick update: We are now at 27 disabled tests https://github.com/apache/incubator-mxnet/issues?q=is%3Aopen+is%3Aissue+label%3A%22Disabled+test%22 -Marco On Thu, Jun 28, 2018 at 2:31 AM Marco de Abreu wrote: > Hello, > > we currently have a false-failure rate of about 59% in our CI. T

Re: C++ api issue labeling

2018-07-12 Thread Marco de Abreu
ween these two labels. > > > > It would make sense to merge the "Feature" label into "Feature Request". > > > > > > Thanks > > Anirudh > > > > > > On Wed, Jun 27, 2018 at 3:50 PM Hagay Lupesko wrote: > > > &

Re: Squash/Merge PRs

2018-07-12 Thread Marco de Abreu
Hi Naveen, I'm in favour of the squashing, considering the number of commits in some PRs and especially because of some people making commit messages a la "fix" "fix" "fix" all the time. Additionally, it gets hard (not impossible, just more inconvenient) to determine the atomic states of master -

Re: Abandoned MXNet readthedocs site

2018-07-05 Thread Marco de Abreu
We could add that doc to our CI nightly pipeline or auto generate it by extracting certain parts from our website, thus reducing the maintenance overhead. In general, I'd think that it's not feasible to add a redirect. But I think it would be desirable to get the readthedocs page under our control

Re: Release process for R

2018-07-04 Thread Marco de Abreu
nvironment configuration (e.g. for the package > `rJava` one needs to take care of $PATH and the permission of the > installation). > > > Best regards, > > Tong He > > 2018-07-04 1:20 GMT-07:00 Marco de Abreu < > marco.g.ab...@googlemail.com.invalid>: > > >

Re: Release process for R

2018-07-04 Thread Marco de Abreu
t; Thanks > Anirudh > > > On Tue, Jul 3, 2018 at 2:48 AM Marco de Abreu > wrote: > > > Hello, > > > > do we have a release process for our R frontend? I noticed the issue at > [1] > > and it seems like we're only publishing to an S3 bucket which i

Re: Allow SSL Verification to be off in mx.gluon.utils.download?

2018-07-04 Thread Marco de Abreu
Thanks for raising this issue Sheng. My proposal would be to always print a warning message when this function is called with the ssl check disabled. This functionality would be tested by a unit test which mocks the network access. Additionally, I'd like to propose that we set a policy for oursel

Release process for R

2018-07-03 Thread Marco de Abreu
Hello, do we have a release process for our R frontend? I noticed the issue at [1] and it seems like we're only publishing to an S3 bucket which is not under Apache. Is there another channel for our users to retrieve that package or is this our only supported official way? Best regards, Marco [1

Re: Master doesn't build, RAT license check failure

2018-07-01 Thread Marco de Abreu
Okay, the issue has been resolved and master is stable again. Please just add a new commit to your pull requests so they take effect. Thanks to Pedro, Anton and Carin! -Marco Marco de Abreu schrieb am So., 1. Juli 2018, 11:57: > I'm on it. > > Pedro Larroy schrieb am So., 1. J

Re: Merging Clojure PR

2018-07-01 Thread Marco de Abreu
018 at 6:55 AM, Marco de Abreu < > marco.g.ab...@googlemail.com.invalid> wrote: > > > PR is available at https://github.com/apache/incubator-mxnet/pull/11512. > > So > > far, it's looking good. None the less, please review :) > > > > -Marco > > >

Re: Merging Clojure PR

2018-07-01 Thread Marco de Abreu
gt; > On Sun, Jul 1, 2018 at 5:56 AM, Marco de Abreu < > marco.g.ab...@googlemail.com.invalid> wrote: > > > Congratulations! Unfortunately was the last CI run of that PR stale and > did > > not include the latest updates to our pipeline. We just added the rat > > lic

Re: Master doesn't build, RAT license check failure

2018-07-01 Thread Marco de Abreu
I'm on it. Pedro Larroy schrieb am So., 1. Juli 2018, 09:03: > Hi > > Master is not building due to RAT license check on the clojure package. Is > anyone having a look at this? > > Pedro. >

Re: Merging Clojure PR

2018-07-01 Thread Marco de Abreu
Congratulations! Unfortunately was the last CI run of that PR stale and did not include the latest updates to our pipeline. We just added the rat license check which apparently did not run on your PR send is now failing on master as well as blocking all pull request builds. I will try to add the li

Re: Beam's recent community development work

2018-06-30 Thread Marco de Abreu
Very interesting! I especially like the different levels of engagement and beam committers actively engaging potential committers to mentor them towards reaching the next level. This fact together with the feedback loop sounds like a very promising and encouraging system. Thanks a lot Yasser and K

RAT license check as part of PRs

2018-06-29 Thread Marco de Abreu
Hello, so far, we've been running RAT license checks as part of our nightly pipeline. To avoid back and forth before a release, we'd like to make it part of the sanity stage of our PR validation. In future, you will receive a notification if you added a file which requires an Apache license. The

Out-of-bound read performed by the SequenceLastKernel

2018-06-28 Thread Marco de Abreu
Hello, we encountered a test failure on Windows which caused memory corruption. This resulted in spilled over test failures, causing further CUDA kernels to fail to launch [1]. Thanks to Dick Carter who debugged the issue and tracked it down to a bad implementation of SequenceLastKernel, leading t

Re: Vote to stop using JIRA

2018-06-28 Thread Marco de Abreu
ng links, I > > see MXNet already has configured project management boards in Jira and it > > seems people have assigned related tasks there, I mean I see it's already > > working and we don't need an extra To Do, right? > > > > Regards. >

Disabling flaky tests

2018-06-27 Thread Marco de Abreu
Hello, we currently have a false-failure rate of about 59% in our CI. This leads to a lot of PRs failing due to flaky tests. We're currently in the process to fix problematic tests, but we're still not at a point which we can consider as stable. A few colleagues proposed to disable tests temporar

Re: Broken dependency in Caffe converter causes all CI runs to fail

2018-06-27 Thread Marco de Abreu
qskuze8K2d+ck2p/t9fsuXLMYE7Se6f7T/wCjkP2e/wDhH1+q5cnXYnhBXHfdZ/ashx39n932XLksuwro8bsPP7plS3HmuXIBOtf6j/JVD+o3yXq5AB1Hc+ZUbb+p6H6rxcszDNvvfnReVP5XLljDC12CKXLkxj//2Q== > > > 2018-06-27 14:45 GMT+01:00 Chen HY : > > > Is that the cat you need? > > > > 2018-06-27 13:03 GMT+01:0

Re: Broken dependency in Caffe converter causes all CI runs to fail

2018-06-27 Thread Marco de Abreu
ke to make sure that we re-enable this test before the next release. -Marco [1]: https://github.com/apache/incubator-mxnet/pull/11419 On Wed, Jun 27, 2018 at 1:53 PM Marco de Abreu wrote: > Hello, > > I have noticed an increased number of CI failures. The reason seems to be > htt

Re: Vote to stop using JIRA

2018-06-27 Thread Marco de Abreu
; > On 6/11/2018 8:53 PM, Marco de Abreu wrote: > > How about you set up a personal instance of JIRA (let us know if the > public > > ones differ too much from the one we're running under Apache) and play > > around a bit? We can then review the setup and migrate th

Broken dependency in Caffe converter causes all CI runs to fail

2018-06-27 Thread Marco de Abreu
Hello, I have noticed an increased number of CI failures. The reason seems to be https://github.com/apache/incubator-mxnet/issues/11407. A third party dependency (a cat image) has been deleted from the webserver and thus causes all our CI runs to fail. Is anybody familiar with that test and has a

Re: C++ api issue labeling

2018-06-25 Thread Marco de Abreu
+1 to renaming to Backend On Mon, Jun 25, 2018 at 10:13 AM Hagay Lupesko wrote: > Thanks Lin for your feedback. > Bumping again to get more feedback before concluding. > > On Fri, Jun 22, 2018 at 8:53 AM Lin Yuan wrote: > > > I agree with Hagay. Using "Backend" as label makes it much easier to

Re: Catching segmentation faults in nosetests

2018-06-21 Thread Marco de Abreu
value explained here ( > > https://stackoverflow.com/questions/14599670/what-error-code-does-a-process-that-segfaults-return > ) > set. Then you can mark the full suite as crashed. > > Pedro. > > On Thu, Jun 21, 2018 at 1:35 PM Marco de Abreu > wrote: > > > Hello, > >

Re: [VOTE] Release MXNet version 1.2.1.RC0 (Patch Release)

2018-06-21 Thread Marco de Abreu
We had 868 successful runs and no failures for that test so far, Pedro. On Thu, Jun 21, 2018 at 11:17 PM Pedro Larroy wrote: > Observed just one failure in OSX in test_imdecode: > > > == > FAIL: test_imdecode (test_image.TestIma

Catching segmentation faults in nosetests

2018-06-21 Thread Marco de Abreu
Hello, is anybody aware of a way to catch segmentation faults as part of the nosetests execution and log them as ERROR? Right now, the nosetests process gets terminated without a stack trace or further test execution. This has additional significance because our result-recording that has been intr

Re: [LAZY VOTE] Test coverage of PRs

2018-06-21 Thread Marco de Abreu
ersonally like it a > lot even with some limitations with it. It doesn't allow us to merge a > PR with a not tested code. It reports us where are those not covered new > codes. > > Regards. > > On 6/21/2018 7:13 PM, Marco de Abreu wrote: > > To avoid any confu

Re: C++ api issue labeling

2018-06-21 Thread Marco de Abreu
Hello Hagay, as far as I understand, the CPP Package label is for the C++ Frontend which is generated at [1] while the C++ label is intended for the backend. But I agree that we don't really have to label backend bugs as C++, so we could just deprecate the CPP Package label and move everything ove

Re: [LAZY VOTE] Test coverage of PRs

2018-06-21 Thread Marco de Abreu
On Thu, Jun 21, 2018 at 6:43 AM Marco de Abreu wrote: > Hi Kellen, > > Thanks for your feedback. > > The same argument about a hosted service could be applied to GitHub - we > could just use the git repository hosted under Apache Infra then and > completely stick to JIRA. This

Re: [LAZY VOTE] Test coverage of PRs

2018-06-21 Thread Marco de Abreu
t; (i.e. Scala) might also be nice eventually. > > -Kellen > > On Thu, Jun 21, 2018 at 4:59 AM Chris Olivier > wrote: > > > we should have a betting pool on what the current coverage is. > > > > On Wed, Jun 20, 2018 at 7:54 PM Naveen Swamy

Re: The operator check for Scala Package

2018-06-20 Thread Marco de Abreu
> > > Firstly, this would only be triggered if there is an operator > > changes > > > > > (Only general operators). > > > > > Secondly, it is simple to go through. Just need to change 1 > line > > of > > > > code > > >

Re: [LAZY VOTE] Test coverage of PRs

2018-06-20 Thread Marco de Abreu
on the pull request merging. > > Tianqi > > On Wed, Jun 20, 2018 at 7:14 PM, Marco de Abreu < > marco.g.ab...@googlemail.com.invalid> wrote: > > > Hello, > > > > I'd like to introduce test coverage metrics of PRs using > > https://codecov.io/.

[LAZY VOTE] Test coverage of PRs

2018-06-20 Thread Marco de Abreu
Hello, I'd like to introduce test coverage metrics of PRs using https://codecov.io/. This tool will aggregate coverage reports across multiple runs, platforms and technologies and gives contributors as well as reviewers a new tool that allows to improve the quality of a pull request. Since we nee

Re: The operator check for Scala Package

2018-06-19 Thread Marco de Abreu
working together on this. The idea is that we fail the build > if there is a operator change on the backend and have not synced to the > Scala API. We want to catch this before breaking the user's code which will > be a pretty bad experience. > > > > On Tue, Jun 1

Re: The operator check for Scala Package

2018-06-19 Thread Marco de Abreu
Hi Qing, thank you for working on improving the compatibility of our APIs! Your linked proposal does not describe the mentioned FILEHASH. Could you elaborate a bit? Would this be a hash of the entire file, some hash created based on the signature of the underlying C++ methods or maybe a different

Re: Nightly tests README accurate?

2018-06-18 Thread Marco de Abreu
GPUs. After reading your email, I'm > thinking I'll run most of the notebooks on P3.2xl and only the ones that > require multiple GPUs on P3.8xl. Thanks for the reply. > > > On Mon, Jun 18, 2018 at 2:24 PM Marco de Abreu > wrote: > > > In future (next few days

Re: Nightly tests README accurate?

2018-06-18 Thread Marco de Abreu
In future (next few days) the nightlies will be running on our regular CI at http://jenkins.mxnet-ci.amazon-ml.com/ On there, we support C5.18xlarge (Ubuntu, CentOS within Docker and Windows), G3.8xlarge (Ubuntu, CentOS within Docker and Windows), P3.2xlarge (Ubuntu, CentOS within docker) and (upc

Re: users@mxnet

2018-06-16 Thread Marco de Abreu
> we > > > > > > stick > > > > > > > with that given the user community prefers that > > > > > > > > > > > > > > Tianqi > > > > > > > > > > > > > > On Fri, Jun 15, 2018

Re: users@mxnet

2018-06-15 Thread Marco de Abreu
I think nobody was opposed to it in the past, right? I'd propose that all emails automatically get copied to dev@ to ensure high visibility initially. What do you think? Sebastian schrieb am Fr., 15. Juni 2018, 20:51: > I have already proposed this many times in the past and would strongly > en

Re: Reverting pull request

2018-06-15 Thread Marco de Abreu
c Xie wrote: > Hi Marco de Abreu, > > CI has been totally broken recently. It randomly fails for no good reason > more often than it passes. For example the ccache/efs failure has been > really annoying. > > Looks like there has been many changes to Jenkins and Docker lately. Do &g

Re: Reverting pull request

2018-06-15 Thread Marco de Abreu
tly. It breaks a not > recommended usage of the API from an unmaintained tutorial, I don't think > adding more reviewers will help it. > > Besides, I'm less sure if we can find enough reviewers to provide useful > feedbacks for major changes. > > On Fri, Jun 15, 2018

Re: Reverting pull request

2018-06-15 Thread Marco de Abreu
that brings improvements in the long term > > Tianqi > > On Fri, Jun 15, 2018 at 2:15 PM, Mu Li wrote: > > > Why reverting instead of fixing the bugs? Static memory aims to > reduce > > memory allocation, it's a key feature to bridge the perf

Re: MXNet issues labeling

2018-06-15 Thread Marco de Abreu
Mentors, do you know if it is possible to get a bot account with these restricted permissions? I know that previous requests have been declined by Apache Infra. Shall we try it again or how shall we continue moving forward? -Marco On Fri, Jun 15, 2018 at 2:09 PM Qing Lan wrote: > Hi All, > I wo

Re: Reverting pull request

2018-06-15 Thread Marco de Abreu
it's a key feature to bridge the perf gap between gluon > and symbol. > > On Fri, Jun 15, 2018 at 2:06 PM, Marco de Abreu < > marco.g.ab...@googlemail.com.invalid> wrote: > > > Hello, > > > > I'm reverting https://github.com/apache/incubator-mxnet/pul

Reverting pull request

2018-06-15 Thread Marco de Abreu
Hello, I'm reverting https://github.com/apache/incubator-mxnet/pull/10817 as of https://github.com/apache/incubator-mxnet/pull/11311 due to regressions described in https://github.com/apache/incubator-mxnet/issues/11171 and https://github.com/apache/incubator-mxnet/pull/10817. The pull request ha

Re: Update on 1.2.1 release

2018-06-15 Thread Marco de Abreu
; also seen it happen on Ubuntu, there are probably some links in the > issue > >>> (on my phone right now). > >>> > >>> Anirudh schrieb am Mi., 13. Juni 2018, 22:16: > >>> > >>> > By segfaulting test do you mean : test_gru_bidirect

Re: About Becoming a Committer

2018-06-15 Thread Marco de Abreu
itters not having to modify the action (PR, > proposed-answer, proposed-wiki-change etc). > > Hen > > On Fri, Jun 15, 2018 at 12:26 AM, Marco de Abreu < > marco.g.ab...@googlemail.com.invalid> wrote: > > > Hen, > > > > As you stated, it's of signifi

Re: About Becoming a Committer

2018-06-15 Thread Marco de Abreu
Hen, As you stated, it's of significance of how much a PR has to be changed as a result of a review. I think this is what this project defines as quality. If people submit a bunch of PRs and we reviewers spend a lot of time on every single one of them to give the contributors advice about how to

Re: Ccache errors

2018-06-14 Thread Marco de Abreu
in main default=default_ccache_dir(), File "ci/build.py", line 127, in default_ccache_dir return ccache_dirpython NameError: name 'ccache_dirpython' is not defined script returned exit code 1 Best regards, Marco [1]: https://github.com/apache/incubator-mxnet/pull/11269 On Thu, Jun 1

Ccache errors

2018-06-14 Thread Marco de Abreu
Hello, I'd like to explain the recent errors we had in the past 12 hours as part of CI. This was caused by my preparation to enable shared ccache volumes. Due to a mistake from my side, the option got applied to all runs instead of just my PR, causing all runs to fail. I have reverted all changes

Re: Update on 1.2.1 release

2018-06-13 Thread Marco de Abreu
don't see the > segfault in the logs. Can you point me to the test. > Also, this seems to be specific to the master and not in 1.2: > https://github.com/apache/incubator-mxnet/pull/10311 > > Anirudh > > On Wed, Jun 13, 2018 at 10:00 PM, Marco de Abreu < > marco.g.ab.

Re: Update on 1.2.1 release

2018-06-13 Thread Marco de Abreu
I can confirm that this segfaulting test has a big impact. On Wed, Jun 13, 2018 at 9:39 PM Aaron Markham wrote: > I'd keep an eye on this one... Flaky test: test_gru_bidirectional #11219 > > https://github.com/apache/incubator-mxnet/issues/11219 > > Just reran several PR's CI runs that all had

Re: Pip packages for Raspberry Pi & other ARM architectures

2018-06-13 Thread Marco de Abreu
I think Sheng manages the publishing to PyPi and we should be fine with just adding it as another flavour. -Marco On Wed, Jun 13, 2018 at 6:36 AM Pedro Larroy wrote: > Hi > > We would like to start publishing pre-built beta pip wheels of MXNet for > Raspberry pi and ARM. > > What's the procedur

Re: GitHub Label Bot Design

2018-06-12 Thread Marco de Abreu
Hello Cathy, that's a great proposal. Thank you! A few comments from my side: - Good idea with the alias. We should have a special email-list for automated reports to prevent spamming dev@. - "Create weekly email to internal team members:" -> email-list - "Part II - Label Bot - Amazon cloudwatch e

Re: Clojure Package

2018-06-11 Thread Marco de Abreu
it > up. > > > On Tue, Jun 12, 2018 at 12:07 AM, Marco de Abreu < > marco.g.ab...@googlemail.com> wrote: > > > The problem I see here is that the migration will have different type of > > challenges which should be handled isolated. Trying to solve them all at

Re: Clojure Package

2018-06-11 Thread Marco de Abreu
res iteratively well > tested and well documented. MXNet already has many different language > bindings which has quite a bit of tech-debt, I don't want just to add more > tech-debt to the code base with new language bindings as well. > > On Mon, Jun 11, 2018 at 11:17 PM, M

Re: Clojure Package

2018-06-11 Thread Marco de Abreu
I think we should try to separate this migration into multiple stages: 1. Move into contrib 2. Migrate release pipeline 3. Migrate tests 4. Improvements 5. Mark as stable and announce Julia officially I know how much effort Carin and the other maintainers are putting into that project and thus I'd

Re: Details regarding upcoming PR for runtime TensorRT integration

2018-06-11 Thread Marco de Abreu
I don't know how to grant permission on Confluence. If somebody else knows how to do so, please grant Marek the edit permissions. -Marco On Mon, Jun 11, 2018 at 11:05 AM Marek Kolodziej wrote: > Hi Rajan, > > I wanted to share on Confluence, but it didn't allow me to create a new > document. If

Re: Details regarding upcoming PR for runtime TensorRT integration

2018-06-11 Thread Marco de Abreu
Hello Marek, this sounds great! Definitely looking forward to it. It seems like our mailing list destroyed your formatting. You might want to consider putting it into a Google Docs document or uploading it to confluence. Best regards, Marco On Mon, Jun 11, 2018 at 10:50 AM Marek Kolodziej wrot

Re: Vote to stop using JIRA

2018-06-11 Thread Marco de Abreu
Thanks for this detailed description! I think GitHub and JIRA are a good thing to co-exist. GitHub for issues and JIRA for action items (as a result of the issues). I like GitHub for it's good community integration - just mention somebody or link it in another issue/PR and everything will automati

Re: Feature branches for ARM and Android

2018-06-10 Thread Marco de Abreu
t work it off of a fork? > > > On Thu, Jun 7, 2018 at 5:32 PM, Marco de Abreu < > marco.g.ab...@googlemail.com > > wrote: > > > Hello, > > > > we are currently revisiting our setup for ARM and Android. Since we're > not > > in a stable state but need

Re: Accidental commits to 1.2.0 branch using Maven apache-release profile

2018-06-09 Thread Marco de Abreu
hear that. I contacted several customers, most of them are > waiting for 1.2 to be ready on the internal system. Did they fork by their > own? If they changed the codes, will them submit their changes? > > On Sat, Jun 9, 2018 at 2:51 PM, Marco de Abreu < > marco.g.ab...@googlemail.com

Re: Vote to stop using JIRA

2018-06-09 Thread Marco de Abreu
committers. -Marco On Sat, Jun 9, 2018 at 5:32 PM Yasser Zamani wrote: > > > On 6/9/2018 5:01 PM, Marco de Abreu wrote: > > Would you mind creating a proposal document at > > [1], describing what you would have in mind and how project planning, > > -management, developme

Re: Accidental commits to 1.2.0 branch using Maven apache-release profile

2018-06-09 Thread Marco de Abreu
> 1.2 some day between May 21 and June 7 and want to PR their changes later. > But it less likely happens. > > Best, > Mu > > > On Jun 9, 2018, at 2:05 PM, Marco de Abreu > wrote: > > > > Would there be any benefit besides cosmetics? I'd propose to just

Re: Accidental commits to 1.2.0 branch using Maven apache-release profile

2018-06-09 Thread Marco de Abreu
change is pushed, and rebase > against the code after squash, it will be fine. The problem will only > happen if someone checked out 1.2.0 after these commits get in, push their > own changes, then try to rebase > > Tianqi > > On Sat, Jun 9, 2018 at 1:25 PM, Marco de Abreu <

Re: Accidental commits to 1.2.0 branch using Maven apache-release profile

2018-06-09 Thread Marco de Abreu
Jun 9, 2018 at 1:18 PM, Marco de Abreu < > marco.g.ab...@googlemail.com > > wrote: > > > I think we should never ever force push to a publish repository since we > > don't know what depends on it. I'd say we take this as a lesson learned > and > &g

Re: Accidental commits to 1.2.0 branch using Maven apache-release profile

2018-06-09 Thread Marco de Abreu
It's good to have a stable > > release only patched with meaningful commits. > > > > Best, > > Mu > > > > > On Jun 7, 2018, at 2:55 PM, Marco de Abreu < > marco.g.ab...@googlemail.com> > > wrote: > > > > > > Ah yeah that's e

Re: Vote to stop using JIRA

2018-06-09 Thread Marco de Abreu
s you can think of would look like then? Please let us know if you need permissions to edit the wiki. Best regards, Marco [1]: https://cwiki.apache.org/confluence/display/MXNET/Design+Proposals On Sat, Jun 9, 2018 at 12:56 PM Yasser Zamani wrote: > > > On 6/9/2018 12:43 PM, Marco de Abre

Re: Vote to stop using JIRA

2018-06-09 Thread Marco de Abreu
e github issue, another for only > contributer to access, which of course will have some more harder to answer > or codding questions and will be published at JIRA. > For example, the simple one can be documents, bug fix or new ideas, the the > harder one can be new idea implementation.

Re: Vote to stop using JIRA

2018-06-09 Thread Marco de Abreu
Hi, thanks for your feedback! I think the big problem is that we have nobody who maintains JIRA properly. If we had a dedicated person who owns that system, I'm sure we could excell at it's usage and it would provide better value. About the plugins, I have to agree that we can just request assist

Re: Vote to stop using JIRA

2018-06-08 Thread Marco de Abreu
+1 While the intention of JIRA was good, we have proven as a community that we are not able to use it properly and it's only being a burden. The initial idea was that it allows us to manage projects publicly, make plans and allow everybody to engage into these projects by just picking small tasks.

Re: Regarding 1.2.1 patch release

2018-06-08 Thread Marco de Abreu
Thanks a lot Aaron for addressing this in such a timely manner! It would definitely be great to see the raspberry build fixes in that branch since it's currently broken. -Marco Aaron Markham schrieb am Fr., 8. Juni 2018, 18:10: > Hi Anirudh, > Marco brought my attention to CI failures on the 1

Re: PR validation and runtime of CI

2018-06-07 Thread Marco de Abreu
ot build all components to reduce cost and the time it > > > takes to run CI. A Scala build need not build everything and run tests > > > related to Python, etc., > > > > > > Thanks, Naveen > > > > > > On Thu, Jun 7, 2018 at 9:57 AM, Marco de Abreu < > &g

Re: Accidental commits to 1.2.0 branch using Maven apache-release profile

2018-06-07 Thread Marco de Abreu
7, 2018 at 2:41 PM, Marco de Abreu < > marco.g.ab...@googlemail.com > > wrote: > > > Hello Naveen, > > > > thank you for addressing this. Is it possible to let the plugin point to > a > > private fork so you can review it's actions manually before they a

Re: Accidental commits to 1.2.0 branch using Maven apache-release profile

2018-06-07 Thread Marco de Abreu
Hello Naveen, thank you for addressing this. Is it possible to let the plugin point to a private fork so you can review it's actions manually before they are written to the main repository? That way, we could just open a PR and double check everything is as expected. Thanks a lot for taking care

Re: Github link on MXNet Homepage

2018-06-07 Thread Marco de Abreu
Yeah, why not :) -Marco On Thu, Jun 7, 2018 at 7:03 PM singh.raja...@gmail.com < singh.raja...@gmail.com> wrote: > Hi All, > > On MXNet home page, the link to github is currently inside > Community->GitHub. I think we should have link to GitHub readily available > on homepage ( one click), simil

Re: PR validation and runtime of CI

2018-06-07 Thread Marco de Abreu
> that is killing the productivity, we can redesign the test pipeline to run > integration and unit test in parallel and that would give us straight away > a 30 minutes reduced time in the CI run. Then we'd be always at <2h for a > build, which seems reasonable if it never fails

Feature branches for ARM and Android

2018-06-07 Thread Marco de Abreu
Hello, we are currently revisiting our setup for ARM and Android. Since we're not in a stable state but need some way to collaborate while having support from CI, we proposed the usage of feature branches. Thus, I have create the two branches devel-arm and devel-android into which Anton and Pedro

Re: PR validation and runtime of CI

2018-06-07 Thread Marco de Abreu
Yeah, I think we are at the point at which we have to disable tests.. If a test fails in nightly, the commit would not be reverted since it's hard to pin a failure to a specific PR. We will have reporting for failures on nightly (they have proven to be stable, so we can enable it right from the be

Fwd: ICLA?

2018-06-06 Thread Marco de Abreu
Hello secretary, could you please confirm that all committers have signed their ICLAs and that the wiki (see history of this thread) is obsolete. Best regards, Marco de abreu On behalf of Apache MXNet (incubating) -- Forwarded message - From: Steffen Rochel Date: Mi., 6. Juni

Re: ICLA?

2018-06-06 Thread Marco de Abreu
Thanks a lot Isabel! Considering ICLAs are automatically managed by the secretary of Apache before anybody can become a committer and no CLA is required for non-committers, I'd say that we can delete that wiki page. Does anybody disagree? -Marco Isabel Drost-Fromm schrieb am Mi., 6. Juni 2018,

Re: ICLA?

2018-06-05 Thread Marco de Abreu
Henry, was this document only related to the migration towards Apache or do we have to keep it up to date? I've heard that sometimes every contributor has to sign it before a PR can be accepted. I have seen other communities that use a bot to keep track of signed ICLA documents, but I don't know

Re: A proposal for unified integration with external acceleration libraries

2018-06-05 Thread Marco de Abreu
I definitely like this idea. We have also been discussing about the exact same idea within our team and think that it allows MXNet to scale better with the increasing number of backends. Imagine we get AMD-CPU, AMD-GPU, ARM, Snapdragon and other vendors/chips. We are definitely running into problem

Re: backward compatibility of models saved with 1.2.0

2018-05-28 Thread Marco de Abreu
Hello Sergio, you are right. We are following semantic versioning and thus, every model produced within the same major version (e.g.1.x) has to be backwards compatible. Could you please provide a small example so we can reproduce this? We definitely do not want our users to retrain their model if

Introduction of restricted slaves and jobs

2018-05-25 Thread Marco de Abreu
Hello MXNet community, I'd like to announce the launch of restricted slaves and jobs for our CI system. The purpose of this feature is to allow separating slaves that execute arbitrary code from verified code like on our version-branches and the master. This step is necessary in order to increase

Jenkins master received security updates

2018-05-24 Thread Marco de Abreu
Hello, I'd to inform you that our Jenkins Master and the underlying plugins have all just been updated to the latest versions to include the latest security updates. We don't expect any problems caused by this, but if you experience any issues due to the update, please let me know so we can inves

Re: Scala Packages in Maven

2018-05-21 Thread Marco de Abreu
pache.org/confluence/display/MXNET/Release+Process# > ReleaseProcess-Step1.12.CreateScalaMavenPackages(WIP) > > 2) https://cwiki.apache.org/confluence/display/MXNET/MXNet-Scala > > On Mon, May 21, 2018 at 5:37 PM, Marco de Abreu < > marco.g.ab...@googlemail.com> wrote: > &g

Re: MXNet issues labeling

2018-05-21 Thread Marco de Abreu
Haha, Aaron could work in marketing - AI solves everything! :P But yeah, a bit more details would be good. As an alternative approach, I'd like to propose a bot that does actions on your behalf. In this case, we could have a bit that listens for comments and (based on a whitelist), it would apply

Re: Scala Packages in Maven

2018-05-21 Thread Marco de Abreu
Agree, we should make a proper assessment of what all these packages are before we try managing them. Do you know who published them before? Moving forward, it'd be great if you could document the entire process (and all issues you encounter) in confluence. This would allow us to re-visit decision

Re: Config for Scala Maven Package

2018-05-21 Thread Marco de Abreu
Hi Naveen, thank you for driving the releases for the Scala packages! For CPU, I'd propose add MKL/MKLDNN builds. For GPU, do we have to specify the minor version of CUDA up front or are they interchangeable? CUDA 9.2, for example, received some quite significant performance updates and it would

Re: Using parts of code from library under PyTorch organization

2018-05-18 Thread Marco de Abreu
By the way. In general, I'd prefer to have a solution in C++ as part of our Backend, as this would allow us to share this information with all frontend APIs. Tobias is currently working on a PR which does exactly the same, but for GPUs. Marco de Abreu schrieb am Sa., 19. Mai 2018, 02:45:

Re: Using parts of code from library under PyTorch organization

2018-05-18 Thread Marco de Abreu
I would have to check the license, but I assume it's under Apache. In that case, it should be enough to copy the file and keep the original license header. Additionally, you have to document any changes you made and where you got it from. You can do this in the header, below the license. In any ca

Re: [LAZY VOTE][RESULT] Upgrade CI to CUDA 9.1 with CuDNN 7.0

2018-05-17 Thread Marco de Abreu
found during > 1.2 rc voting > - https://github.com/apache/incubator-mxnet/issues/10981 were reported by > an user with CUDA 7 > > Having these covered in CI will help catch the issues early. I don't recall > if we decided to drop CUDA 7 support for MXNet. > >

Re: Auto scaling for MXNet CI

2018-05-16 Thread Marco de Abreu
at 3:47 AM, Thomas DELTEIL wrote: > Great news :) thanks Marco! > > On Tue, May 15, 2018, 18:29 Steffen Rochel > wrote: > > > Thanks Marco, good step forward. > > What is the expected, typical and worst case TAT time for PR checks? > > > > Steffen > &

Auto scaling for MXNet CI

2018-05-15 Thread Marco de Abreu
Hello, I'd like to announce the deployment of auto scaling for our CI system (see [1] for reference, setup documentation at [2]) for today at 11:00PM PST 05/15/18. I expect no downtime since these changes are happening outside of Jenkins. This system will increase the flexibility of our system to

Re: Problems with test_sparse_operator.test_sparse_mathematical_core

2018-05-11 Thread Marco de Abreu
on mxnet master. One thing on my wish >> list is a database which stores all the occurrences of test failures and >> their commit ids, which would be very helpful for initial diagnosing what >> code changes potentially introduced bugs. Otherwise clicking all past >> tests

Re: Problems with test_sparse_operator.test_sparse_mathematical_core

2018-05-09 Thread Marco de Abreu
l diagnosing what > code changes potentially introduced bugs. Otherwise clicking all past tests > and reading those logs requires a lot of manual work. > > Best, > Haibin > > On Wed, May 9, 2018 at 5:32 AM, Marco de Abreu < > marco.g.ab...@googlemail.com > > wrote: > >

Problems with test_sparse_operator.test_sparse_mathematical_core

2018-05-09 Thread Marco de Abreu
Hello, I'm currently working on auto scaling and encountering a consistent test failure on CPU. At the moment, I'm not really sure what's causing this, considering the setup should be identical. http://jenkins.mxnet-ci-dev.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/ci-master/

Re: [VOTE] Release Apache MXNet(incubating) version 1.2.0.RC2

2018-05-08 Thread Marco de Abreu
@ accordingly? > > Currently you are on record as -1. Just trying to help Anirud to get > proper > > vote count. > > > > Thanks > > Steffen (MXNet contributor hat on) > > > > On Tue, May 8, 2018 at 6:37 AM Marco de Abreu < > > marco.g.ab...

<    1   2   3   4   5   6   >