Re: MXNet 1.6.0 release

[Skalicky, Sam]

Thanks Tao, 

I’m working closely with Jun to ensure the Numpy effort is included in the 
release. 

Sam

> On Oct 11, 2019, at 10:35 AM, Tao Lv  wrote:
> 
> Hi Przemek,
> 
> Thank you for volunteering!
> 
> I remember Jun (reminisce@github) has already been working on 1.6.0. You
> might need check with him to avoid duplicate effort.
> 
> Thanks,
> -tao
> 
> 
>> On Fri, Oct 11, 2019 at 8:00 AM Przemysław Trędak 
>> wrote:
>> 
>> Hi MXNet Community,
>> 
>> As the 1.5.1 patch release is done (many thanks Tao!), it is time to
>> prepare for the next minor release of MXNet - 1.6.0.
>> 
>> I (ptrendx@github / ptredak@mxnet Slack) would like to manage the release
>> of 1.6.0. As it will be the first time for me to manage a release, Sam
>> (samskalicky) and Lin (apeforest) agreed to help guiding me through the
>> process.
>> 
>> Thanks to Sheng there is a GitHub issue[1] listing major features that
>> should go into the 1.6.0, please add any features that you want included
>> there.
>> 
>> That said, as we target November for the release, to accommodate for
>> extensive testing and bugfixing, the code freeze date is set to October
>> 24th 23:59PST. Please reach out to me as soon as possible if you feel that
>> you will need an extension of that deadline for your feature.
>> 
>> Sheng created a page on cwiki[2] about the release, I will populate it
>> with the information and tracked issues and PRs.
>> 
>> Thank you and let's make the great 1.6.0 release together!
>> Przemek
>> 
>> [1] https://github.com/apache/incubator-mxnet/issues/15589
>> [2]
>> https://cwiki.apache.org/confluence/display/MXNET/1.6.0+Release+Plan+and+Status
>> 

[ANNOUNCE] Release Apache MXNet (incubating) version 1.5.1

[Tao Lv]

Hi all,

The Apache MXNet (incubating) community is happy to announce Apache MXNet
(incubating) version 1.5.1!

Apache MXNet (incubating) is a deep learning framework designed for both
efficiency and flexibility. It allows you to mix symbolic and imperative
programming to maximize efficiency and productivity.

1.5.1 is a maintenance release incorporating important bug fixes and
important performance improvements.

A full list of the changes in this release can be found in the release
notes:
https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Notes

A link to the download can be found here:
http://mxnet.incubator.apache.org/get_started/download

If you prefer to build from source and experiment with various compile-time
configuration options, use this link to get the instructions:
http://mxnet.incubator.apache.org/get_started/ubuntu_setup.html

http://mxnet.incubator.apache.org/get_started/centos_setup.html

 Or you can download and play with MXNet easily using one of the options
below:

1. The Pip packages can be found here:
https://pypi.python.org/pypi/mxnet

2. The Docker Images can be found here:
https://hub.docker.com/r/mxnet/python/

Links in Maven to the published Scala packages:

https://repository.apache.org/content/repositories/releases/org/apache/mxnet/
https://repository.apache.org/#nexus-search;quick~org.apache.mxnet

and to the experimental Clojure packages:
https://repository.apache.org/content/repositories/releases/org/apache/mxnet/contrib/clojure/

The Release Tag:
https://github.com/apache/incubator-mxnet/tree/1.5.
1

MXNet Resources
- Our discussion forum (https://discuss.mxnet.io)
- MXNet user mailing list (
https://lists.apache.org/list.html?u...@mxnet.apache.org)
- MXNet dev mailing list (
https://lists.apache.org/list.html?d...@mxnet.apache.org)
- StackOverflow mxnet tag (https://stackoverflow.com/questions/tagged/mxnet)
- MXNet website (https://mxnet.incubator.apache.org)
- Github issues (https://github.com/apache/incubator-mxnet/issues)
- Wiki (https://cwiki.apache.org/confluence/display/MXNET)

Attend one of the regular user groups meetings:
https://cwiki.apache.org/confluence/x/7BY0BQ

For more information on Apache MXNet (incubating), please see:
https://mxnet.incubator.apache.org/


Best regards,
Apache MXNet (incubating) Team

___

DISCLAIMER:

Apache MXNet (incubating) is an effort undergoing incubation at The Apache
Software Foundation (ASF), sponsored by the name of Apache Incubator PMC.
Incubation is required of all newly accepted projects until a further
review indicates that the infrastructure, communications, and decision
making process have stabilized in a manner consistent with other successful
ASF projects. While incubation status is not necessarily a reflection of
the completeness or stability of the code, it does indicate that the
project has yet to be fully endorsed by the ASF.

https://cwiki.apache.org/confluence/x/BINjB

Re: MXNet 1.6.0 release

[Tao Lv]

Hi Przemek,

Thank you for volunteering!

I remember Jun (reminisce@github) has already been working on 1.6.0. You
might need check with him to avoid duplicate effort.

Thanks,
-tao


On Fri, Oct 11, 2019 at 8:00 AM Przemysław Trędak 
wrote:

> Hi MXNet Community,
>
> As the 1.5.1 patch release is done (many thanks Tao!), it is time to
> prepare for the next minor release of MXNet - 1.6.0.
>
> I (ptrendx@github / ptredak@mxnet Slack) would like to manage the release
> of 1.6.0. As it will be the first time for me to manage a release, Sam
> (samskalicky) and Lin (apeforest) agreed to help guiding me through the
> process.
>
> Thanks to Sheng there is a GitHub issue[1] listing major features that
> should go into the 1.6.0, please add any features that you want included
> there.
>
> That said, as we target November for the release, to accommodate for
> extensive testing and bugfixing, the code freeze date is set to October
> 24th 23:59PST. Please reach out to me as soon as possible if you feel that
> you will need an extension of that deadline for your feature.
>
> Sheng created a page on cwiki[2] about the release, I will populate it
> with the information and tracked issues and PRs.
>
> Thank you and let's make the great 1.6.0 release together!
> Przemek
>
> [1] https://github.com/apache/incubator-mxnet/issues/15589
> [2]
> https://cwiki.apache.org/confluence/display/MXNET/1.6.0+Release+Plan+and+Status
>

MXNet 1.6.0 release

[Przemysław Trędak]

Hi MXNet Community,

As the 1.5.1 patch release is done (many thanks Tao!), it is time to prepare 
for the next minor release of MXNet - 1.6.0.

I (ptrendx@github / ptredak@mxnet Slack) would like to manage the release of 
1.6.0. As it will be the first time for me to manage a release, Sam 
(samskalicky) and Lin (apeforest) agreed to help guiding me through the process.

Thanks to Sheng there is a GitHub issue[1] listing major features that should 
go into the 1.6.0, please add any features that you want included there.

That said, as we target November for the release, to accommodate for extensive 
testing and bugfixing, the code freeze date is set to October 24th 23:59PST. 
Please reach out to me as soon as possible if you feel that you will need an 
extension of that deadline for your feature.

Sheng created a page on cwiki[2] about the release, I will populate it with the 
information and tracked issues and PRs.

Thank you and let's make the great 1.6.0 release together!
Przemek

[1] https://github.com/apache/incubator-mxnet/issues/15589
[2] 
https://cwiki.apache.org/confluence/display/MXNET/1.6.0+Release+Plan+and+Status

Re: [apache/incubator-mxnet] [RFC] MXNet Multithreaded Inference Interface (#16431)

[Anirudh Subramanian]

Thanks @marcoabreu ! 

> Will the new C-API functions be threadsafe in general? Speak, I can invoke 
> them at any point in time from any thread without the need of a lock, 
> sticky-thread or a thread hierarchy? (I'm thinking of the thread-safety being 
> done on the backend level)

The issue I found with C API thread safety especially with the cached op use 
case was the ThreadLocalStore. If we fix this issue then C APIs related to 
CreateCachedOp and InvokeCachedOp should be threadsafe.

>  Will this also support the GPU use-case? Speak, the parameters are only 
> copied into GPU memory once in the same fashion as you're describing for the 
> CPU?

This should still support the single GPU use-case for 1.6. Multi GPU inference 
use case requires more verification at the cached op level .

> Do you think there's a path forward to make all inference-related C-APIs 
> threadsafe instead of splitting off another execution branch?

I don't think we have such a strict split between inference and training APIs 
at the C API level. For example for gluon cached op we call InvokeCachedOp for 
both training and Inference.

But if I rephrase your question to:
Will I be able to do multi threaded inference from every frontend API which I 
can use to do inference today ?  
Right now, I am targeting only gluon since most users have been directed 
towards gluon. The other ways are using module, symbolic and using C Predict 
API. To support these two frontend APIs requires the graph executor to be 
thread safe.  This would definitely be a great add for MXNet since it would 
ensure that they can do multi-threaded inference from any of these APIs in 
MXNet, but not something I have planned for currently.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-mxnet/issues/16431#issuecomment-540834971

[apache/incubator-mxnet] [RFC] MXNet Multithreaded Inference Interface (#16431)

[Anirudh Subramanian]

Thanks to @nswamy for his inputs and design discussions related to this project 
and @frankfliu for explaining the requirements and the use case from customer 
perspective.

# Problem Statement

One of the big un-catered for use cases in MXNet is loading a model and being 
able to run parallel inference on the model from multiple threads while sharing 
the parameters. There are multiple user requests for the same 
[[1]](https://github.com/apache/incubator-mxnet/issues/3946). There also has 
been a lot of confusion around the current state of MXNet with respect to 
thread safety.

This doc attempts to address three things : 

1. Tries to clarify the current state of MXNet with respect to thread safety.
2. Tries to give an idea of the benefits to expect from adding this feature.
3. Attempts to solve the problem of parallel inference by providing a 
multi-threaded inference API ( C APIs and frontend APIs in CPP and Python), 

# Current State of MXNet Thread Safety

## MXNet Dependency Engine Thread Safety

Examining MXNet dependency engine code, it looks like it was designed  to be 
thread safe. Tried to push Convolution op from multiple threads into MXNet 
Engine, to see if there are any issues with thread safety. Used CPP Package for 
the same. The script is provided here : 
https://github.com/anirudh2290/mxnet/tree/multithreaded_inference_poc/cpp-package/example/multithreading_engine_push_mxnet_op.cpp

```
./build/cpp-package/example/multithreading_engine_push_mxnet_op 2
```

The script pushes Convolution op to the engine from multiple threads. You can 
verify the correctness of the op with this script : 
https://github.com/anirudh2290/mxnet/tree/multithreaded_inference_poc/test_cached_op_ts_check.py

```
python3 test_cached_op_ts_check.py
```

## MXNet Graph Executor Thread Safety

Removed NaiveEngine only restriction for C Predict API and tried to run multi 
threaded inference with C Predict API using ThreadedEngine by commenting the 
check : 
https://github.com/anirudh2290/mxnet/tree/multithreaded_inference_poc/src/c_api/c_predict_api.cc

When running this example the program core dumps with memory leaks in Graph 
Executor Bind. This shows that graph executor is not thread safe. 

## Cached Op (Gluon Backend) Thread Safety

Try to create cached op in the main thread and spawn multiple threads to invoke 
the same cached op inside each of the threads. Here is the script which does 
the same : 
https://github.com/anirudh2290/mxnet/tree/multithreaded_inference_poc/cpp-package/example/multithreading_engine_push_cached_op.cpp

```
# Usage
./build/cpp-package/example/multithreading_engine_push_cached_op  
 

# Example
./build/cpp-package/example/multithreading_engine_push_cached_op 20 cpu 0 // 
uses cached op available in master
```

Multiple failures seen when I run this: one is in the dmlc ThreadLocalStore 
[[2]](https://github.com/dmlc/dmlc-core/issues/571),  other is in MXPlanMemory, 
retrieving forward_ref_count attribute. These errors are because of race 
condition w.r.t reading and writing of shared states in CachedOp.

# Proposed Solution

### Additions (Prioritized for 1.6)

Proposing to add a minimal thread safe cached op for inference which will be 
the following :
1. Similar to cached op, except it supports only inference use cases. 
2. Doesn’t support inlining, dynamic shapes, bulking, static alloc. 
3. Use static thread_local variables for GraphInfo which maintains the 
fwd_graph state, buff which maintains all ndarray states and for op_states. [ 
There is scope for additional optimization here w.r.t separation of buffers for 
inputs and params]
4. The above addition means that we can instantiate only one thread safe cached 
op per process. The frontend API for SymbolBlockThreadSafe needs to be a 
singleton because of this limitation.

### C API Changes (Prioritized for 1.6)

Adding a new thread_safe flag for MXCreateCachedOpEx. When set to true this 
should create a thread_safe cached op instead of a cached op.

```
  /*!
   * \brief create cached operator
   */
  MXNET_DLL int MXCreateCachedOpEx(SymbolHandle handle,
   int num_flags,
   const char** keys,
   const char** vals,
   CachedOpHandle *out,
   bool thread_safe = false);
```

Add similar thread_safe flag flags to Invoke and Free to invoke thread safe 
cached op versions instead of the default versions. 

```
  /*!
   * \brief invoke a cached op
   * \param handle the handle to the cached op
   * \param num_inputs number of input NDArrays
   * \param inputs input NDArrays
   * \param num_outputs number of output NDArrays
   * \param outputs output NDArrays
   * \param out_stypes output ndarrays' stypes
   * \param thread_safe whether to invoke thread safe version of cached op.
   * \return 0 when success, -1 when failure happens
   */

  
  MXNET_DLL int 

Re: [REVIEW][ANNOUNCE] Release Apache MXNet (incubating) version 1.5.1

[kellen sunderland]

Lgtm Tao.

On Thu, Oct 10, 2019, 7:23 AM Tao Lv  wrote:

> Okay, looks like there is no objections. I will send the announcement to
> announce@ and general@ soon.
>
> Thanks,
> -tao
>
> On Wed, Oct 9, 2019 at 10:35 AM Tao Lv  wrote:
>
> > Dear community,
> >
> > This is to review the announcement for 1.5.1 release according to the
> > section 3.4 in release process.
> >
> > ===
> >
> > The Apache MXNet (incubating) community is happy to announce Apache MXNet
> > (incubating) version 1.5.1!
> >
> > Apache MXNet (incubating) is a deep learning framework designed for both
> > efficiency and flexibility. It allows you to mix symbolic and imperative
> > programming to maximize efficiency and productivity.
> >
> > 1.5.1 is a maintenance release incorporating important bug fixes and
> > important performance improvements.
> >
> > A full list of the changes in this release can be found in the release
> > notes:
> > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Notes
> >
> > A link to the download can be found here:
> > http://mxnet.incubator.apache.org/get_started/download
> >
> > If you prefer to build from source and experiment with various
> > compile-time configuration options, use this link to get the
> instructions:
> > http://mxnet.incubator.apache.org/get_started/ubuntu_setup.html
> >
> > http://mxnet.incubator.apache.org/get_started/centos_setup.html
> >
> >  Or you can download and play with MXNet easily using one of the options
> > below:
> >
> > 1. The Pip packages can be found here:
> > https://pypi.python.org/pypi/mxnet
> >
> > 2. The Docker Images can be found here:
> > https://hub.docker.com/r/mxnet/python/
> >
> > Links in Maven to the published Scala packages:
> >
> >
> >
> https://repository.apache.org/content/repositories/releases/org/apache/mxnet/
> > https://repository.apache.org/#nexus-search;quick~org.apache.mxnet
> >
> > and to the experimental Clojure packages:
> >
> >
> https://repository.apache.org/content/repositories/releases/org/apache/mxnet/contrib/clojure/
> >
> > The Release Tag:
> > https://github.com/apache/incubator-mxnet/tree/1.5.
> > 1
> >
> > MXNet Resources
> > - Our discussion forum (https://discuss.mxnet.io)
> > - MXNet user mailing list (
> > https://lists.apache.org/list.html?u...@mxnet.apache.org)
> > - MXNet dev mailing list (
> > https://lists.apache.org/list.html?d...@mxnet.apache.org)
> > - StackOverflow mxnet tag (
> > https://stackoverflow.com/questions/tagged/mxnet)
> > - MXNet website (https://mxnet.incubator.apache.org)
> > - Github issues (https://github.com/apache/incubator-mxnet/issues)
> > - Wiki (https://cwiki.apache.org/confluence/display/MXNET)
> >
> > Attend one of the regular user groups meetings:
> > https://cwiki.apache.org/confluence/x/7BY0BQ
> >
> > For more information on Apache MXNet (incubating), please see:
> > https://mxnet.incubator.apache.org/
> >
> >
> > Best regards,
> > Apache MXNet (incubating) Team
> >
> > ___
> >
> > DISCLAIMER:
> >
> > Apache MXNet (incubating) is an effort undergoing incubation at The
> Apache
> > Software Foundation (ASF), sponsored by the name of Apache Incubator PMC.
> > Incubation is required of all newly accepted projects until a further
> > review indicates that the infrastructure, communications, and decision
> > making process have stabilized in a manner consistent with other
> successful
> > ASF projects. While incubation status is not necessarily a reflection of
> > the completeness or stability of the code, it does indicate that the
> > project has yet to be fully endorsed by the ASF.
> >
> > https://cwiki.apache.org/confluence/x/BINjB
> >
>

Re: [REVIEW][ANNOUNCE] Release Apache MXNet (incubating) version 1.5.1

[Tao Lv]

Okay, looks like there is no objections. I will send the announcement to
announce@ and general@ soon.

Thanks,
-tao

On Wed, Oct 9, 2019 at 10:35 AM Tao Lv  wrote:

> Dear community,
>
> This is to review the announcement for 1.5.1 release according to the
> section 3.4 in release process.
>
> ===
>
> The Apache MXNet (incubating) community is happy to announce Apache MXNet
> (incubating) version 1.5.1!
>
> Apache MXNet (incubating) is a deep learning framework designed for both
> efficiency and flexibility. It allows you to mix symbolic and imperative
> programming to maximize efficiency and productivity.
>
> 1.5.1 is a maintenance release incorporating important bug fixes and
> important performance improvements.
>
> A full list of the changes in this release can be found in the release
> notes:
> https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Notes
>
> A link to the download can be found here:
> http://mxnet.incubator.apache.org/get_started/download
>
> If you prefer to build from source and experiment with various
> compile-time configuration options, use this link to get the instructions:
> http://mxnet.incubator.apache.org/get_started/ubuntu_setup.html
>
> http://mxnet.incubator.apache.org/get_started/centos_setup.html
>
>  Or you can download and play with MXNet easily using one of the options
> below:
>
> 1. The Pip packages can be found here:
> https://pypi.python.org/pypi/mxnet
>
> 2. The Docker Images can be found here:
> https://hub.docker.com/r/mxnet/python/
>
> Links in Maven to the published Scala packages:
>
>
> https://repository.apache.org/content/repositories/releases/org/apache/mxnet/
> https://repository.apache.org/#nexus-search;quick~org.apache.mxnet
>
> and to the experimental Clojure packages:
>
> https://repository.apache.org/content/repositories/releases/org/apache/mxnet/contrib/clojure/
>
> The Release Tag:
> https://github.com/apache/incubator-mxnet/tree/1.5.
> 1
>
> MXNet Resources
> - Our discussion forum (https://discuss.mxnet.io)
> - MXNet user mailing list (
> https://lists.apache.org/list.html?u...@mxnet.apache.org)
> - MXNet dev mailing list (
> https://lists.apache.org/list.html?d...@mxnet.apache.org)
> - StackOverflow mxnet tag (
> https://stackoverflow.com/questions/tagged/mxnet)
> - MXNet website (https://mxnet.incubator.apache.org)
> - Github issues (https://github.com/apache/incubator-mxnet/issues)
> - Wiki (https://cwiki.apache.org/confluence/display/MXNET)
>
> Attend one of the regular user groups meetings:
> https://cwiki.apache.org/confluence/x/7BY0BQ
>
> For more information on Apache MXNet (incubating), please see:
> https://mxnet.incubator.apache.org/
>
>
> Best regards,
> Apache MXNet (incubating) Team
>
> ___
>
> DISCLAIMER:
>
> Apache MXNet (incubating) is an effort undergoing incubation at The Apache
> Software Foundation (ASF), sponsored by the name of Apache Incubator PMC.
> Incubation is required of all newly accepted projects until a further
> review indicates that the infrastructure, communications, and decision
> making process have stabilized in a manner consistent with other successful
> ASF projects. While incubation status is not necessarily a reflection of
> the completeness or stability of the code, it does indicate that the
> project has yet to be fully endorsed by the ASF.
>
> https://cwiki.apache.org/confluence/x/BINjB
>