Re: MXNet developer setup on Mac with VSCode for develop, test and debug

[Markham, Aaron]

This is tangential, but Lin, I noticed during the RC1 tests you said you tried 
it out on Windows and it worked for you. I'd like to get VS2017 or VS Code 
working, take Sandeep's setup content and possibly your Windows experience, and 
improve the MXNet Windows setup guide. I've tried it and failed. Multiple 
times. I also tried the MKLDNN instructions and failed. I tried the setup tools 
batch file and was hit with a lot of dependency errors. Some of the problem 
isn't in the MXNet docs, but in the dependencies' documentation, but I'm left 
to go figure that out on my own. Anyway, any help you can provide here would be 
great. Also, if any of you reading this has a sort of checklist or guide for 
Windows, I'd love to see it.

BTW, I'm using Windows 10 with an NVIDIA GeForce GTX 980, and was trying to use 
VS2017 Community Edition and MKL. I went to MKL after OpenBLAS wasn't 
installing/building.

On 7/18/18, 10:59 AM, "Lin Yuan"  wrote:

Thanks for the well-written document! As a new MXNet developer, I have
found it very helpful.

Lin

On Wed, Jul 18, 2018 at 10:50 AM sandeep krishnamurthy 
wrote:

> Hello Community,
>
>
>
> As a MXNet contributor, I had issues and took me some time on getting
> hands-on with MXNet codebase, being able to code, test, DEBUG python/CPP
> combination. I have documented the steps for MXNet development setup using
> VSCode on Mac. Document starts from installing all required
> tools/packages/IDEs/extensions and then provides steps for debugging mix 
of
> Python/CPP code, which is most likely the case for any MXNet developer, 
all
> in single IDE window. By end of this document, anyone should be able to
> walk through the MXNet code, debug and be able to make first code change.
>
>
>
> Please feel free to add comments, make changes as necessary.
>
>
> 
https://cwiki.apache.org/confluence/display/MXNET/MXNet+Developer+Setup+on+Mac
>
> Best,
> Sandeep
>

Re: Adding section on how to develop with MXNet to the website

[Markham, Aaron]

This is the section where development guides are being maintained:
https://cwiki.apache.org/confluence/display/MXNET/Development

That way you can edit more freely and have comments versus the mxnet.io site 
which is more for users and static content.


On 7/2/18, 2:03 PM, "Hagay Lupesko"  wrote:

Now I understand Pedro. Agree with Naveen this would be helpful.

On Sun, Jul 1, 2018 at 12:34 AM Naveen Swamy  wrote:

> No please add, this would be super helpful. I have struggled with this and
> thanks for the help you offered offline. If you can please make a small
> screencast
>
> > On Jul 1, 2018, at 12:07 AM, Pedro Larroy 
> wrote:
> >
> > Hi Hagay
> >
> > Meaning how to set the developer environment like CLion to debug MXNet,
> how
> > to build and run tests etc. Or is this already documented?
> >
> > Pedro.
> >
> >> On Tue, Jun 26, 2018 at 6:14 AM Hagay Lupesko 
> wrote:
> >>
> >> Pedro,
> >>
> >> Anything that helps bring in more contributors is good IMO.
> >> But can you please clarify what you mean by "develop MXNet itself"?
> >>
> >> Hagay
> >>
> >> On Mon, Jun 25, 2018, 19:33 Markham, Aaron  >
> >> wrote:
> >>
> >>> More or less... Instructions are in the readme in the docs folder.
> Focus
> >>> on the developer sections. Dependencies and other info is provided.
> Link
> >>> to your info from the contribute page that's under community.
> >>>
> >>> Ping me if you need help.
> >>>
> >>> Sent from VMware Boxer
> >>>
> >>> On Jun 25, 2018 19:17, Pedro Larroy 
> >> wrote:
> >>> Hi
> >>>
> >>> I want to add a section on how to develop MXNet itself to attract
> >>> contributors. Would this be acceptable for the website?
> >>>
> >>> Is there any recommended workflow for this?   Any tools?
> >>>
> >>> is it going into docs and `make html` or something else?
> >>>
> >>> Thanks.
> >>>
> >>> Pedro
> >>>
> >>
>

Re: Adding section on how to develop with MXNet to the website

[Markham, Aaron]

More or less... Instructions are in the readme in the docs folder. Focus on the 
developer sections. Dependencies and other info is provided.  Link to your info 
from the contribute page that's under community.

Ping me if you need help.

Sent from VMware Boxer

On Jun 25, 2018 19:17, Pedro Larroy  wrote:
Hi

I want to add a section on how to develop MXNet itself to attract
contributors. Would this be acceptable for the website?

Is there any recommended workflow for this?   Any tools?

is it going into docs and `make html` or something else?

Thanks.

Pedro

Re: Abandoned MXNet readthedocs site

[Markham, Aaron]

Hi, just bumping this issue again. Can we get a trademark C&D sent to the 
readthedocs folks, so they'll go ahead and remove the derelict website? It's 
still the #4 result in Google for "mxnet install".

What's Apache's process?

Thanks,
Aaron

On 5/16/18, 8:20 AM, "Hen"  wrote:

It seems we (Apache MXNet) have an old documentation site posted at:

https://newdocs.readthedocs.io/en/latest/

and only the absent @Awyan has the permissions to the site. See
https://github.com/apache/incubator-mxnet/issues/10409 for more details.

Raising this with readthedocs.org, we've been asked to submit a DMCA
request to take down the account:

https://github.com/rtfd/readthedocs.org/issues/4042

Given our docs are Apache licensed I suspect it's really a trademark
takedown (confused users etc). Does anyone here have any guidance with this
situation? Has anyone had this issue with readthedocs before? Should we be
sending them a Trademark takedown request?

Thanks,

Hen

Re: installation page UX

[Markham, Aaron]

Ultimately, I want to retire the version selector that swaps out the whole 
site. Version selection can be at the API docs and at the install page where 
people would expect to have that option.

Side note, because this come up when mentioning removal of the versions 
dropdown for the site: tutorials can be tagged with their minimum version & 
Python or other language support. Keeping around v0.11.0 tutorials just creates 
really high bounce rates for the site.

I was hoping to tackle one problem at a time. We need clear install 
instructions for the 90% of the visitors. We shouldn't complicate the install 
page with every corner case. I'd cut the install page down to the bare minimum.

Basically +1 for PyTorch UI.
But... other considerations come into play. Is it ok to push the non-Python 
language binding to their own pages?

And +1 for the direct linking capability - regardless of the UI.


On 5/22/18, 12:05 PM, "sandeep krishnamurthy"  
wrote:

Thanks Aaron for this quick preview.

Version is selected at the top level documentation? Should we show how to
install v1.0 when the user is viewing v1.2 docs?

From UX functionality perspective - MXNet installation page is similar to
PyTorch? Collection of choices to make.

Behind the scenes, one single monolithic markdown is a severe problem as
rightly pointed out.
Aaron, can you please include more details on how this issue is planned to
be handled?
Another issue we need to list down is - We cannot hyperlink to specific
choices in the install guide. For example, Hyperlink that takes directly to
Ubuntu, GPU, Source build. This might be helpful in search engine, blogs,
and documentation.

Best,
Sandeep

On Tue, May 22, 2018 at 11:53 AM, Naveen Swamy  wrote:

> MXNet UI definitely needs more love.
>
> +1 - pytorch style
> +0.5 - caffe2
>
>
> On Tue, May 22, 2018 at 11:48 AM, Markham, Aaron 
> wrote:
>
> > Hi everyone,
> > In addition to the options on the wiki (pros & cons), there's this
> preview.
> > It uses a dropdown next to the install options to make it clearer what
> > versions you can install… then updates the pip commands…
> >
> > http://54.210.6.225/install/index.html#
> >
> > Thoughts?
> >
> >
> > On 5/16/18, 9:31 AM, "Aaron Markham"  wrote:
> >
> > Hi,
> > I've written some notes on the wiki about issues with the
> installation
> > page
> > along with some suggestions. I'd appreciate your feedback here or on
> > the
> > wiki.
> >
> > https://cwiki.apache.org/confluence/display/MXNET/
> Installation+page+UX
> >
> > Cheers,
> > Aaron
> >
> >
> >
>



-- 
Sandeep Krishnamurthy

Re: installation page UX

[Markham, Aaron]

Hi everyone,
In addition to the options on the wiki (pros & cons), there's this preview.
It uses a dropdown next to the install options to make it clearer what versions 
you can install… then updates the pip commands…

http://54.210.6.225/install/index.html#

Thoughts?


On 5/16/18, 9:31 AM, "Aaron Markham"  wrote:

Hi,
I've written some notes on the wiki about issues with the installation page
along with some suggestions. I'd appreciate your feedback here or on the
wiki.

https://cwiki.apache.org/confluence/display/MXNET/Installation+page+UX

Cheers,
Aaron

QA & build process for tutorials

[Markham, Aaron]

Hi everyone,
Do you have any opinion on the structure of the builds for tutorials & 
notebooks? I’ve found close to 200 scripts and notebooks that need attention.

The current state of notebooks is…

MXNet:
md --> html + ipynb

The Straight Dope:
ipynb --> html

Many of the notebooks are broken*, and testing doesn’t seem to happen as part 
of the build. However, we can implement notebook testing as part of the 
build or 
even as a regular nightly process.

*Broken: urllib bug (Python 3), six module needs upgrading (Python 2), 
AttributeError: 'NoneType' object has no attribute 'readlines', and so on…

Considerations:

  *   Python 2 || Python 3 support. Just 3 or both?
  *   Default context: testing the nbconvert tool to check for errors reveals 
that we should have ctx.cpu() as our default context, so that testing can be 
automated on non-GPU machines, docker/CI setups, etc.
  *   Review of ipynb’s on github is a PITA.
  *   Source of truth: md or ipynb?
  *   Inclusion: how to make it easy and clear how to contribute and edit 
tutorials?
  *   Scripts/readmes: can we index these and have them on the html version of 
the site by translating the readme.md to index.html during the build (this 
already sort of happens and is probably just a configuration step needed).

Suggestion:

  *   Nightly build: look for the latest md or ipynb file and translate it, so 
a newer md makes an ipynb, or a newer ipynb makes an md.
  *   QA: if the latest ipynb fails testing, roll back, create a ticket (or 
whatever notification makes sense).
  *   Conformity: have mxnet and TSD use a similar process.
  *   Documentation of scripts: each folder should have a readme, if not, it’s 
flagged (this should help clean up the issues with undocumented scripts)


Cheers,
Aaron

QA & build process for tutorials

[Markham, Aaron]

Hi everyone,
Do you have any opinion on the structure of the builds for tutorials & 
notebooks? I’ve found close to 200 scripts and notebooks that need attention.

The current state of notebooks is…

MXNet:
md --> html + ipynb

The Straight Dope:
ipynb --> html

Many of the notebooks are broken*, and testing doesn’t seem to happen as part 
of the build. However, we can implement notebook testing as part of the 
build or 
even as a regular nightly process.

*Broken: urllib bug (Python 3), six module needs upgrading (Python 2), 
AttributeError: 'NoneType' object has no attribute 'readlines', and so on…

Considerations:

  *   Python 2 || Python 3 support. Just 3 or both?
  *   Default context: testing the nbconvert tool to check for errors reveals 
that we should have ctx.cpu() as our default context, so that testing can be 
automated on non-GPU machines, docker/CI setups, etc.
  *   Review of ipynb’s on github is a PITA.
  *   Source of truth: md or ipynb?
  *   Inclusion: how to make it easy and clear how to contribute and edit 
tutorials?
  *   Scripts/readmes: can we index these and have them on the html version of 
the site by translating the readme.md to index.html during the build (this 
already sort of happens and is probably just a configuration step needed).

Suggestion:

  *   Nightly build: look for the latest md or ipynb file and translate it, so 
a newer md makes an ipynb, or a newer ipynb makes an md.
  *   QA: if the latest ipynb fails testing, roll back, create a ticket (or 
whatever notification makes sense).
  *   Conformity: have mxnet and TSD use a similar process.
  *   Documentation of scripts: each folder should have a readme, if not, it’s 
flagged (this should help clean up the issues with undocumented scripts)


Cheers,
Aaron