RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Hi MXNet Community,

I am canceling this vote as there's an issue which broke the Gluon CV Yolo and 
AutoGluon functionality.
Thanks Ziyi and Xingjian to root cause and fix the issue[1]. 

And the new code base will involve another two fixes (for Gluon activation[2] 
and monitor fix[3] respectively). 
I will update the artifacts and start a new vote for rc1 in the following days.

Thanks for everyone's help! Please let me know if there's any other issue with 
1.7.0.

[1] https://github.com/apache/incubator-mxnet/pull/18692
[2] https://github.com/apache/incubator-mxnet/pull/18700
[3] https://github.com/apache/incubator-mxnet/pull/18703

Thanks,
-Ciyong

-Original Message-
From: Chen, Ciyong  
Sent: Tuesday, July 14, 2020 10:13 AM
To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Thanks all for the effort to double check the performance status and the 
valuable comments, then let's not taking it as a blocker and moving forward 
with the 1.7.0 release process.

Thanks,
-Ciyong

-Original Message-
From: Skalicky, Sam  
Sent: Tuesday, July 14, 2020 4:41 AM
To: dev@mxnet.incubator.apache.org; lau...@apache.org; d...@mxnet.apache.org
Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

That’s a good point, 1.6 did have a performance regression since it dropped 
MKLML to simplify build an fix licensing. 2.0 will have performance degradation 
too in favor of new features. Clearly the community is focusing on features 
rather than performance, at least we're consistent :-)

I would prefer we move forward with the 1.7.0 release and consider performance 
fixes for 1.7.1 (like we did for 1.3.1/1.4.1)

Sam

On 7/13/20, 1:36 PM, "Leonard Lausen"  wrote:

CAUTION: This email originated from outside of the organization. Do not 
click links or open attachments unless you can confirm the sender and know the 
content is safe.



One of the selling points of MXNet is (or used to be) speed and having 
multiple
releases in series with speed regressions may not be acceptable to users 
that
adopted MXNet based on the speed advantage. Should we vote on a 1.7 Beta 
release
and only vote on 1.7 final release once the regressions have been fixed?

On Mon, 2020-07-13 at 19:33 +, Patrick Mu wrote:
> It happens only on CPU, and I did more runs and found that the runtime
> fluctuates very badly, but the average regression is ~10%.
>
>
> Through the previous benchmarks I also found some worse regression 
comparing
> 1.6 to 1.5 like inception inference on CPU and those regression was not
> caught.
>
> My 2-cent is it might not be a blocker for the release, and we can have 
room
> for improvement for upcoming 2.0 and 1.7.1 if necessary
>
> Ziyi
>
> On 2020/07/13 08:40:32, "Chen, Ciyong"  wrote:
> > Thanks Ziyi,
> >
> > May I know which platform did you notice the performance regression, 
CPU or
> > GPU? ~20% regression would be a large gap.
> >
> > Thanks,
> > -Ciyong
> >
> > -Original Message-
> > From: Patrick Mu 
    > > Sent: Monday, July 13, 2020 4:13 PM
> > To: d...@mxnet.apache.org
> > Subject: Re: RE: [VOTE] Release Apache MXNet (incubating) version 
1.7.0.rc0
> >
> > Hi Ciyong,
> >
> > I have reverted the commit, and I am able to train Yolov3 with no 
problem.
> >
> > However I also noticed there is a ~20% regression in 1.7 comparing with 
1.6
> > in inference Yolov3 with Module API, so we are going to discuss 
tomorrow if
> > that would be an issue for 1.7.
> >
> > Thanks,
> > Ziyi
> >
> > On 2020/07/13 02:19:28, "Chen, Ciyong"  wrote:
> > > Hi Ziyi, Xingjian,
> > >
> > > Thanks for reporting the issues from GluonCV/AutoGluon perspective.
> > > I just did a quick try by reverting the
> > > https://github.com/apache/incubator-mxnet/pull/18358, then the 
behavior is
> > > same as 1.6.0 with the cases in the gist (
> > > https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> > >
> > > Considering there's many end-users using Gluon based API/models, and
> > > introducing a new patch to fix this issue could be risky, so I agree 
that
> > > reverting this PR (#18358) might be the best option for the 1.7.0 
release.
> > > But I'm considering is there any other test cases to cover this 
feature,
> > > which could be helpful to track this kind of code changes in future, 
or
> > > can you help to verify if this r

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Thanks all for the effort to double check the performance status and the 
valuable comments, then let's not taking it as a blocker and moving forward 
with the 1.7.0 release process.

Thanks,
-Ciyong

-Original Message-
From: Skalicky, Sam  
Sent: Tuesday, July 14, 2020 4:41 AM
To: dev@mxnet.incubator.apache.org; lau...@apache.org; d...@mxnet.apache.org
Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

That’s a good point, 1.6 did have a performance regression since it dropped 
MKLML to simplify build an fix licensing. 2.0 will have performance degradation 
too in favor of new features. Clearly the community is focusing on features 
rather than performance, at least we're consistent :-)

I would prefer we move forward with the 1.7.0 release and consider performance 
fixes for 1.7.1 (like we did for 1.3.1/1.4.1)

Sam

On 7/13/20, 1:36 PM, "Leonard Lausen"  wrote:

CAUTION: This email originated from outside of the organization. Do not 
click links or open attachments unless you can confirm the sender and know the 
content is safe.



One of the selling points of MXNet is (or used to be) speed and having 
multiple
releases in series with speed regressions may not be acceptable to users 
that
adopted MXNet based on the speed advantage. Should we vote on a 1.7 Beta 
release
and only vote on 1.7 final release once the regressions have been fixed?

On Mon, 2020-07-13 at 19:33 +, Patrick Mu wrote:
> It happens only on CPU, and I did more runs and found that the runtime
> fluctuates very badly, but the average regression is ~10%.
>
>
> Through the previous benchmarks I also found some worse regression 
comparing
> 1.6 to 1.5 like inception inference on CPU and those regression was not
> caught.
>
> My 2-cent is it might not be a blocker for the release, and we can have 
room
> for improvement for upcoming 2.0 and 1.7.1 if necessary
>
> Ziyi
>
> On 2020/07/13 08:40:32, "Chen, Ciyong"  wrote:
> > Thanks Ziyi,
> >
> > May I know which platform did you notice the performance regression, 
CPU or
> > GPU? ~20% regression would be a large gap.
> >
> > Thanks,
> > -Ciyong
> >
> > -Original Message-
> > From: Patrick Mu 
    > > Sent: Monday, July 13, 2020 4:13 PM
> > To: d...@mxnet.apache.org
> > Subject: Re: RE: [VOTE] Release Apache MXNet (incubating) version 
1.7.0.rc0
> >
> > Hi Ciyong,
> >
> > I have reverted the commit, and I am able to train Yolov3 with no 
problem.
> >
> > However I also noticed there is a ~20% regression in 1.7 comparing with 
1.6
> > in inference Yolov3 with Module API, so we are going to discuss 
tomorrow if
> > that would be an issue for 1.7.
> >
> > Thanks,
> > Ziyi
> >
> > On 2020/07/13 02:19:28, "Chen, Ciyong"  wrote:
> > > Hi Ziyi, Xingjian,
> > >
> > > Thanks for reporting the issues from GluonCV/AutoGluon perspective.
> > > I just did a quick try by reverting the
> > > https://github.com/apache/incubator-mxnet/pull/18358, then the 
behavior is
> > > same as 1.6.0 with the cases in the gist (
> > > https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> > >
> > > Considering there's many end-users using Gluon based API/models, and
> > > introducing a new patch to fix this issue could be risky, so I agree 
that
> > > reverting this PR (#18358) might be the best option for the 1.7.0 
release.
> > > But I'm considering is there any other test cases to cover this 
feature,
> > > which could be helpful to track this kind of code changes in future, 
or
> > > can you help to verify if this revert do resolve the broken issue at 
your
> > > side?
> > >
> > > > Thus, the real issue is: Should we supporting pickling a Gluon 
Block? If
> > > > not, should we support combining multiprocessing.pool with the Gluon
> > > > Block?
    > > > Seems it's more like a new feature for MXNet Gluon Block, probably we 
can
> > > make it available in the next patch/minor release?
> > >
> > > Thanks,
> > > -Ciyong
> > >
> > > -Original Message-
> > > From: Xingjian SHI 
> > > Sent: Saturday, July 11, 2020 4:27 AM
> > > To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
> > > Subject: Re: [VOTE] Release Apache MXNet (incubating) version 
1.7.0.rc0
 

Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Skalicky, Sam]

That’s a good point, 1.6 did have a performance regression since it dropped 
MKLML to simplify build an fix licensing. 2.0 will have performance degradation 
too in favor of new features. Clearly the community is focusing on features 
rather than performance, at least we're consistent :-)

I would prefer we move forward with the 1.7.0 release and consider performance 
fixes for 1.7.1 (like we did for 1.3.1/1.4.1)

Sam

On 7/13/20, 1:36 PM, "Leonard Lausen"  wrote:

CAUTION: This email originated from outside of the organization. Do not 
click links or open attachments unless you can confirm the sender and know the 
content is safe.



One of the selling points of MXNet is (or used to be) speed and having 
multiple
releases in series with speed regressions may not be acceptable to users 
that
adopted MXNet based on the speed advantage. Should we vote on a 1.7 Beta 
release
and only vote on 1.7 final release once the regressions have been fixed?

On Mon, 2020-07-13 at 19:33 +, Patrick Mu wrote:
> It happens only on CPU, and I did more runs and found that the runtime
> fluctuates very badly, but the average regression is ~10%.
>
>
> Through the previous benchmarks I also found some worse regression 
comparing
> 1.6 to 1.5 like inception inference on CPU and those regression was not
> caught.
>
> My 2-cent is it might not be a blocker for the release, and we can have 
room
> for improvement for upcoming 2.0 and 1.7.1 if necessary
>
> Ziyi
>
> On 2020/07/13 08:40:32, "Chen, Ciyong"  wrote:
> > Thanks Ziyi,
> >
> > May I know which platform did you notice the performance regression, 
CPU or
> > GPU? ~20% regression would be a large gap.
> >
> > Thanks,
> > -Ciyong
> >
> > -Original Message-
> > From: Patrick Mu 
    > > Sent: Monday, July 13, 2020 4:13 PM
> > To: d...@mxnet.apache.org
> > Subject: Re: RE: [VOTE] Release Apache MXNet (incubating) version 
1.7.0.rc0
> >
> > Hi Ciyong,
> >
> > I have reverted the commit, and I am able to train Yolov3 with no 
problem.
> >
> > However I also noticed there is a ~20% regression in 1.7 comparing with 
1.6
> > in inference Yolov3 with Module API, so we are going to discuss 
tomorrow if
> > that would be an issue for 1.7.
> >
> > Thanks,
> > Ziyi
> >
> > On 2020/07/13 02:19:28, "Chen, Ciyong"  wrote:
> > > Hi Ziyi, Xingjian,
> > >
> > > Thanks for reporting the issues from GluonCV/AutoGluon perspective.
> > > I just did a quick try by reverting the
> > > https://github.com/apache/incubator-mxnet/pull/18358, then the 
behavior is
> > > same as 1.6.0 with the cases in the gist (
> > > https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> > >
> > > Considering there's many end-users using Gluon based API/models, and
> > > introducing a new patch to fix this issue could be risky, so I agree 
that
> > > reverting this PR (#18358) might be the best option for the 1.7.0 
release.
> > > But I'm considering is there any other test cases to cover this 
feature,
> > > which could be helpful to track this kind of code changes in future, 
or
> > > can you help to verify if this revert do resolve the broken issue at 
your
> > > side?
> > >
> > > > Thus, the real issue is: Should we supporting pickling a Gluon 
Block? If
> > > > not, should we support combining multiprocessing.pool with the Gluon
> > > > Block?
    > > > Seems it's more like a new feature for MXNet Gluon Block, probably we 
can
> > > make it available in the next patch/minor release?
> > >
> > > Thanks,
> > > -Ciyong
> > >
> > > -Original Message-
> > > From: Xingjian SHI 
> > > Sent: Saturday, July 11, 2020 4:27 AM
> > > To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
> > > Subject: Re: [VOTE] Release Apache MXNet (incubating) version 
1.7.0.rc0
> > >
> > > Thanks Ziyi,
> > >
> > > I've discovered the same issue when I'm trying to use AutoGluon with
> > > 1.7.0rc0 and would like to share my finding:
> > >
> > > Basically, I don't think Gluon Block is designed to be pickleble. But
> > > pickling do work for some cases in the old version:
> > >
> > >

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Leonard Lausen]

One of the selling points of MXNet is (or used to be) speed and having multiple
releases in series with speed regressions may not be acceptable to users that
adopted MXNet based on the speed advantage. Should we vote on a 1.7 Beta release
and only vote on 1.7 final release once the regressions have been fixed?

On Mon, 2020-07-13 at 19:33 +, Patrick Mu wrote:
> It happens only on CPU, and I did more runs and found that the runtime
> fluctuates very badly, but the average regression is ~10%.
> 
> 
> Through the previous benchmarks I also found some worse regression comparing
> 1.6 to 1.5 like inception inference on CPU and those regression was not
> caught.
> 
> My 2-cent is it might not be a blocker for the release, and we can have room
> for improvement for upcoming 2.0 and 1.7.1 if necessary
> 
> Ziyi
> 
> On 2020/07/13 08:40:32, "Chen, Ciyong"  wrote:
> > Thanks Ziyi,
> > 
> > May I know which platform did you notice the performance regression, CPU or
> > GPU? ~20% regression would be a large gap.
> > 
> > Thanks,
> > -Ciyong
> > 
> > -Original Message-
> > From: Patrick Mu 
> > Sent: Monday, July 13, 2020 4:13 PM
> > To: d...@mxnet.apache.org
> > Subject: Re: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> > 
> > Hi Ciyong,
> > 
> > I have reverted the commit, and I am able to train Yolov3 with no problem.
> > 
> > However I also noticed there is a ~20% regression in 1.7 comparing with 1.6
> > in inference Yolov3 with Module API, so we are going to discuss tomorrow if
> > that would be an issue for 1.7.
> > 
> > Thanks,
> > Ziyi
> > 
> > On 2020/07/13 02:19:28, "Chen, Ciyong"  wrote:
> > > Hi Ziyi, Xingjian,
> > > 
> > > Thanks for reporting the issues from GluonCV/AutoGluon perspective.
> > > I just did a quick try by reverting the 
> > > https://github.com/apache/incubator-mxnet/pull/18358, then the behavior is
> > > same as 1.6.0 with the cases in the gist (
> > > https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> > > 
> > > Considering there's many end-users using Gluon based API/models, and
> > > introducing a new patch to fix this issue could be risky, so I agree that
> > > reverting this PR (#18358) might be the best option for the 1.7.0 release.
> > > But I'm considering is there any other test cases to cover this feature,
> > > which could be helpful to track this kind of code changes in future, or
> > > can you help to verify if this revert do resolve the broken issue at your
> > > side?
> > > 
> > > > Thus, the real issue is: Should we supporting pickling a Gluon Block? If
> > > > not, should we support combining multiprocessing.pool with the Gluon
> > > > Block?
> > > Seems it's more like a new feature for MXNet Gluon Block, probably we can
> > > make it available in the next patch/minor release?
> > > 
> > > Thanks,
> > > -Ciyong
> > > 
> > > -Original Message-
> > > From: Xingjian SHI 
> > > Sent: Saturday, July 11, 2020 4:27 AM
> > > To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
> > > Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> > > 
> > > Thanks Ziyi,
> > > 
> > > I've discovered the same issue when I'm trying to use AutoGluon with
> > > 1.7.0rc0 and would like to share my finding:
> > > 
> > > Basically, I don't think Gluon Block is designed to be pickleble. But
> > > pickling do work for some cases in the old version:
> > > 
> > > I've included two cases in the gist (
> > > https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> > > 
> > > - Case1: we construct a gluon block, hybridize it and feed one NDArray to
> > > help initialize the block. After that, it will no longer be pickleble.
> > > - Case2: we just construct a gluon block and it will be pickleble in
> > > 1.6.0, but won't be pickleble in 1.7.0.
> > > 
> > > Thus, the real issue is: Should we supporting pickling a Gluon Block? If
> > > not, should we support combining multiprocessing.pool with the Gluon
> > > Block? For reference, PyTorch supports pickling the nn.Module as shown in:
> > > https://gist.github.com/sxjscience/90b812a66d445e759c55eedc3ef93668 and
> > > also in the doc (
> > > https://pytorch.org/tutorials/beginner/saving_loading_models.html).
> > > 
> > > Best,
> > > Xingjian
&

Re: RE: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Patrick Mu]

It happens only on CPU, and I did more runs and found that the runtime 
fluctuates very badly, but the average regression is ~10%. 

Through the previous benchmarks I also found some worse regression comparing 
1.6 to 1.5 like inception inference on CPU and those regression was not caught. 

My 2-cent is it might not be a blocker for the release, and we can have room 
for improvement for upcoming 2.0 and 1.7.1 if necessary

Ziyi

On 2020/07/13 08:40:32, "Chen, Ciyong"  wrote: 
> Thanks Ziyi,
> 
> May I know which platform did you notice the performance regression, CPU or 
> GPU? ~20% regression would be a large gap.
> 
> Thanks,
> -Ciyong
> 
> -Original Message-
> From: Patrick Mu  
> Sent: Monday, July 13, 2020 4:13 PM
> To: d...@mxnet.apache.org
> Subject: Re: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> Hi Ciyong,
> 
> I have reverted the commit, and I am able to train Yolov3 with no problem.
> 
> However I also noticed there is a ~20% regression in 1.7 comparing with 1.6 
> in inference Yolov3 with Module API, so we are going to discuss tomorrow if 
> that would be an issue for 1.7.
> 
> Thanks,
> Ziyi
> 
> On 2020/07/13 02:19:28, "Chen, Ciyong"  wrote: 
> > Hi Ziyi, Xingjian,
> > 
> > Thanks for reporting the issues from GluonCV/AutoGluon perspective.
> > I just did a quick try by reverting the 
> > https://github.com/apache/incubator-mxnet/pull/18358, then the behavior is 
> > same as 1.6.0 with the cases in the gist 
> > (https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> > 
> > Considering there's many end-users using Gluon based API/models, and 
> > introducing a new patch to fix this issue could be risky, so I agree that 
> > reverting this PR (#18358) might be the best option for the 1.7.0 release.
> > But I'm considering is there any other test cases to cover this feature, 
> > which could be helpful to track this kind of code changes in future, or can 
> > you help to verify if this revert do resolve the broken issue at your side?
> > 
> > > Thus, the real issue is: Should we supporting pickling a Gluon Block? If 
> > > not, should we support combining multiprocessing.pool with the Gluon 
> > > Block?
> > Seems it's more like a new feature for MXNet Gluon Block, probably we can 
> > make it available in the next patch/minor release?
> > 
> > Thanks,
> > -Ciyong
> > 
> > -Original Message-
> > From: Xingjian SHI  
> > Sent: Saturday, July 11, 2020 4:27 AM
> > To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
> > Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> > 
> > Thanks Ziyi,
> > 
> > I've discovered the same issue when I'm trying to use AutoGluon with 
> > 1.7.0rc0 and would like to share my finding:
> > 
> > Basically, I don't think Gluon Block is designed to be pickleble. But 
> > pickling do work for some cases in the old version:
> > 
> > I've included two cases in the gist 
> > (https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> > 
> > - Case1: we construct a gluon block, hybridize it and feed one NDArray to 
> > help initialize the block. After that, it will no longer be pickleble. 
> > - Case2: we just construct a gluon block and it will be pickleble in 1.6.0, 
> > but won't be pickleble in 1.7.0.
> > 
> > Thus, the real issue is: Should we supporting pickling a Gluon Block? If 
> > not, should we support combining multiprocessing.pool with the Gluon Block? 
> > For reference, PyTorch supports pickling the nn.Module as shown in: 
> > https://gist.github.com/sxjscience/90b812a66d445e759c55eedc3ef93668 and 
> > also in the doc 
> > (https://pytorch.org/tutorials/beginner/saving_loading_models.html). 
> > 
> > Best,
> > Xingjian
> > 
> > 
> > On 7/10/20, 11:31 AM, "Patrick Mu"  wrote:
> > 
> > Hi Ciyong, 
> > 
> > I just discovered an issue with the 1.7, which causes the Yolo training 
> > with latest Gluon CV Yolo to fail.
> > 
> > The PR that causes the failure is 
> > https://github.com/apache/incubator-mxnet/pull/18358, which modifies  basic 
> > blocks of Gluon to fix a memory leak issue.
> > 
> > Talked with Leonard, the author of the PR, and he said he found the 
> > root cause, but patching that PR would modifies those Gluon basic blocks 
> > further, which might be risky towards existing models and various customer 
> > models.
> > 
> > So my 2-cents is reverting this PR in 1.7, and try patc

RE: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Thanks Ziyi,

May I know which platform did you notice the performance regression, CPU or 
GPU? ~20% regression would be a large gap.

Thanks,
-Ciyong

-Original Message-
From: Patrick Mu  
Sent: Monday, July 13, 2020 4:13 PM
To: d...@mxnet.apache.org
Subject: Re: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Hi Ciyong,

I have reverted the commit, and I am able to train Yolov3 with no problem.

However I also noticed there is a ~20% regression in 1.7 comparing with 1.6 in 
inference Yolov3 with Module API, so we are going to discuss tomorrow if that 
would be an issue for 1.7.

Thanks,
Ziyi

On 2020/07/13 02:19:28, "Chen, Ciyong"  wrote: 
> Hi Ziyi, Xingjian,
> 
> Thanks for reporting the issues from GluonCV/AutoGluon perspective.
> I just did a quick try by reverting the 
> https://github.com/apache/incubator-mxnet/pull/18358, then the behavior is 
> same as 1.6.0 with the cases in the gist 
> (https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> 
> Considering there's many end-users using Gluon based API/models, and 
> introducing a new patch to fix this issue could be risky, so I agree that 
> reverting this PR (#18358) might be the best option for the 1.7.0 release.
> But I'm considering is there any other test cases to cover this feature, 
> which could be helpful to track this kind of code changes in future, or can 
> you help to verify if this revert do resolve the broken issue at your side?
> 
> > Thus, the real issue is: Should we supporting pickling a Gluon Block? If 
> > not, should we support combining multiprocessing.pool with the Gluon Block?
> Seems it's more like a new feature for MXNet Gluon Block, probably we can 
> make it available in the next patch/minor release?
> 
> Thanks,
> -Ciyong
> 
> -Original Message-
> From: Xingjian SHI  
> Sent: Saturday, July 11, 2020 4:27 AM
> To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
> Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> Thanks Ziyi,
> 
> I've discovered the same issue when I'm trying to use AutoGluon with 1.7.0rc0 
> and would like to share my finding:
> 
> Basically, I don't think Gluon Block is designed to be pickleble. But 
> pickling do work for some cases in the old version:
> 
> I've included two cases in the gist 
> (https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> 
> - Case1: we construct a gluon block, hybridize it and feed one NDArray to 
> help initialize the block. After that, it will no longer be pickleble. 
> - Case2: we just construct a gluon block and it will be pickleble in 1.6.0, 
> but won't be pickleble in 1.7.0.
> 
> Thus, the real issue is: Should we supporting pickling a Gluon Block? If not, 
> should we support combining multiprocessing.pool with the Gluon Block? For 
> reference, PyTorch supports pickling the nn.Module as shown in: 
> https://gist.github.com/sxjscience/90b812a66d445e759c55eedc3ef93668 and also 
> in the doc 
> (https://pytorch.org/tutorials/beginner/saving_loading_models.html). 
> 
> Best,
> Xingjian
> 
> 
> On 7/10/20, 11:31 AM, "Patrick Mu"  wrote:
> 
> Hi Ciyong, 
> 
> I just discovered an issue with the 1.7, which causes the Yolo training 
> with latest Gluon CV Yolo to fail.
> 
> The PR that causes the failure is 
> https://github.com/apache/incubator-mxnet/pull/18358, which modifies  basic 
> blocks of Gluon to fix a memory leak issue.
> 
> Talked with Leonard, the author of the PR, and he said he found the root 
> cause, but patching that PR would modifies those Gluon basic blocks further, 
> which might be risky towards existing models and various customer models.
> 
> So my 2-cents is reverting this PR in 1.7, and try patching the PR in 1.x 
> and 2.0, meaning that the 1.7 won't have memory usage optimized by that 
> feature.
> 
> I'd like to hear what you think about this issue.
> 
> Thanks,
> Ziyi
> 
> 
> On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote: 
> > Hi Community,
> > 
> > I would like to call for action to test/validate/vote for the release 
> candidate (1.7.0.rc0)
> > As there's not any voting result during the scheduled time window, I 
> would like to extend the time windows to July 13, 23:59:59 PST.
> > Please prepare your time and provide feedback if you've tried with the 
> pre-release code bases, thanks!
>     > 
> > Best regards,
> > Ciyong
> > 
> > -Original Message-
> > From: Chen, Ciyong  
> > Sent: Monday, July 6, 2020 10:48 PM
> > To: d...@mxnet.apache.org
> > Cc: Bob Paulin ;

Re: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Patrick Mu]

Hi Ciyong,

I have reverted the commit, and I am able to train Yolov3 with no problem.

However I also noticed there is a ~20% regression in 1.7 comparing with 1.6 in 
inference Yolov3 with Module API, so we are going to discuss tomorrow if that 
would be an issue for 1.7.

Thanks,
Ziyi

On 2020/07/13 02:19:28, "Chen, Ciyong"  wrote: 
> Hi Ziyi, Xingjian,
> 
> Thanks for reporting the issues from GluonCV/AutoGluon perspective.
> I just did a quick try by reverting the 
> https://github.com/apache/incubator-mxnet/pull/18358, then the behavior is 
> same as 1.6.0 with the cases in the gist 
> (https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> 
> Considering there's many end-users using Gluon based API/models, and 
> introducing a new patch to fix this issue could be risky, so I agree that 
> reverting this PR (#18358) might be the best option for the 1.7.0 release.
> But I'm considering is there any other test cases to cover this feature, 
> which could be helpful to track this kind of code changes in future, or can 
> you help to verify if this revert do resolve the broken issue at your side?
> 
> > Thus, the real issue is: Should we supporting pickling a Gluon Block? If 
> > not, should we support combining multiprocessing.pool with the Gluon Block?
> Seems it's more like a new feature for MXNet Gluon Block, probably we can 
> make it available in the next patch/minor release?
> 
> Thanks,
> -Ciyong
> 
> -Original Message-
> From: Xingjian SHI  
> Sent: Saturday, July 11, 2020 4:27 AM
> To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
> Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> Thanks Ziyi,
> 
> I've discovered the same issue when I'm trying to use AutoGluon with 1.7.0rc0 
> and would like to share my finding:
> 
> Basically, I don't think Gluon Block is designed to be pickleble. But 
> pickling do work for some cases in the old version:
> 
> I've included two cases in the gist 
> (https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> 
> - Case1: we construct a gluon block, hybridize it and feed one NDArray to 
> help initialize the block. After that, it will no longer be pickleble. 
> - Case2: we just construct a gluon block and it will be pickleble in 1.6.0, 
> but won't be pickleble in 1.7.0.
> 
> Thus, the real issue is: Should we supporting pickling a Gluon Block? If not, 
> should we support combining multiprocessing.pool with the Gluon Block? For 
> reference, PyTorch supports pickling the nn.Module as shown in: 
> https://gist.github.com/sxjscience/90b812a66d445e759c55eedc3ef93668 and also 
> in the doc 
> (https://pytorch.org/tutorials/beginner/saving_loading_models.html). 
> 
> Best,
> Xingjian
> 
> 
> On 7/10/20, 11:31 AM, "Patrick Mu"  wrote:
> 
> Hi Ciyong, 
> 
> I just discovered an issue with the 1.7, which causes the Yolo training 
> with latest Gluon CV Yolo to fail.
> 
> The PR that causes the failure is 
> https://github.com/apache/incubator-mxnet/pull/18358, which modifies  basic 
> blocks of Gluon to fix a memory leak issue.
> 
> Talked with Leonard, the author of the PR, and he said he found the root 
> cause, but patching that PR would modifies those Gluon basic blocks further, 
> which might be risky towards existing models and various customer models.
> 
> So my 2-cents is reverting this PR in 1.7, and try patching the PR in 1.x 
> and 2.0, meaning that the 1.7 won't have memory usage optimized by that 
> feature.
> 
> I'd like to hear what you think about this issue.
> 
> Thanks,
> Ziyi
> 
> 
> On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote: 
> > Hi Community,
> > 
> > I would like to call for action to test/validate/vote for the release 
> candidate (1.7.0.rc0)
> > As there's not any voting result during the scheduled time window, I 
> would like to extend the time windows to July 13, 23:59:59 PST.
> > Please prepare your time and provide feedback if you've tried with the 
> pre-release code bases, thanks!
>     > 
> > Best regards,
> > Ciyong
> > 
> > -Original Message-
> > From: Chen, Ciyong  
> > Sent: Monday, July 6, 2020 10:48 PM
> > To: d...@mxnet.apache.org
> > Cc: Bob Paulin ; Henri Yandell ; 
> Jason Dai ; Markus Weimer ; Michael 
> Wall 
> > Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> > 
> > For the language bindings and windows platform, may I have your support 
> to help verify these features? Thanks!
> > 
> > @lanking520 to h

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Thanks Macro for raising up the concern of license and Sheng for the 
clarification, the current release process is only targeting source release.

Regards,
-Ciyong

-Original Message-
From: Marco de Abreu  
Sent: Monday, July 13, 2020 4:51 AM
To: dev@mxnet.incubator.apache.org
Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Okay, thanks for the clarification!

-Marco

On Sun, Jul 12, 2020, 5:58 PM Sheng Zha  wrote:

> Hi Marco,
>
> Since the license issues apply to binary distribution, we should still 
> be able to make official source releases.
>
> Regards,
> Sheng
>
> > On Jul 12, 2020, at 1:10 AM, Marco de Abreu 
> > 
> wrote:
> >
> > Are we in the position to make a release given that we have open 
> > license issues with the ipmc and Apache board? I want to avoid 
> > giving the impression that we are ignoring their requests - my 
> > current understanding is that we are non compliant.
> >
> > -Marco
> >
> >> On Sat, Jul 11, 2020, 9:46 AM Tong He  wrote:
> >>
> >> My +1 on the R binding.
> >>
> >> Tested with
> >>
> >> - Build from source
> >> - Install the R package and check it passed all tests.
> >>
> >>> On 2020/07/10 18:31:27, Patrick Mu  wrote:
> >>> Hi Ciyong,
> >>>
> >>> I just discovered an issue with the 1.7, which causes the Yolo 
> >>> training
> >> with latest Gluon CV Yolo to fail.
> >>>
> >>> The PR that causes the failure is
> >> https://github.com/apache/incubator-mxnet/pull/18358, which 
> >> modifies basic blocks of Gluon to fix a memory leak issue.
> >>>
> >>> Talked with Leonard, the author of the PR, and he said he found 
> >>> the
> root
> >> cause, but patching that PR would modifies those Gluon basic blocks 
> >> further, which might be risky towards existing models and various
> customer
> >> models.
> >>>
> >>> So my 2-cents is reverting this PR in 1.7, and try patching the PR 
> >>> in
> >> 1.x and 2.0, meaning that the 1.7 won't have memory usage optimized 
> >> by
> that
> >> feature.
> >>>
> >>> I'd like to hear what you think about this issue.
> >>>
> >>> Thanks,
> >>> Ziyi
> >>>
> >>>
> >>> On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote:
> >>>> Hi Community,
> >>>>
> >>>> I would like to call for action to test/validate/vote for the 
> >>>> release
> >> candidate (1.7.0.rc0)
> >>>> As there's not any voting result during the scheduled time 
> >>>> window, I
> >> would like to extend the time windows to July 13, 23:59:59 PST.
> >>>> Please prepare your time and provide feedback if you've tried 
> >>>> with the
> >> pre-release code bases, thanks!
> >>>>
> >>>> Best regards,
> >>>> Ciyong
> >>>>
> >>>> -Original Message-
> >>>> From: Chen, Ciyong 
> >>>> Sent: Monday, July 6, 2020 10:48 PM
> >>>> To: d...@mxnet.apache.org
> >>>> Cc: Bob Paulin ; Henri Yandell 
> >>>> ;
> >> Jason Dai ; Markus Weimer ; 
> >> Michael Wall 
> >>>> Subject: RE: [VOTE] Release Apache MXNet (incubating) version
> 1.7.0.rc0
> >>>>
> >>>> For the language bindings and windows platform, may I have your
> >> support to help verify these features? Thanks!
> >>>>
> >>>> @lanking520 to help verify the Scala/Java @gigasquid to help 
> >>>> verify
> >> the Clojure
> >>>> @hetong007 to help verify the R
> >>>> @yajiedesign to help verify the windows platform
> >>>>
> >>>> Best regards,
> >>>> Ciyong Chen
> >>>>
> >>>> -Original Message-
> >>>> From: Chen, Ciyong 
> >>>> Sent: Monday, July 6, 2020 10:39 PM
> >>>> To: d...@mxnet.apache.org
> >>>> Cc: Bob Paulin ; Henri Yandell 
> >>>> ;
> >> Jason Dai ; Markus Weimer ; 
> >> Michael Wall 
> >>>> Subject: [VOTE] Release Apache MXNet (incubating) version 
> >>>> 1.7.0.rc0
> >>>>
> >>>> Dear MXNet community,
> >>>>
> >>>> This is the vote to release Apache MXNet (incubating) version 1.7.0.
>

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Hi Ziyi, Xingjian,

Thanks for reporting the issues from GluonCV/AutoGluon perspective.
I just did a quick try by reverting the 
https://github.com/apache/incubator-mxnet/pull/18358, then the behavior is same 
as 1.6.0 with the cases in the gist 
(https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).

Considering there's many end-users using Gluon based API/models, and 
introducing a new patch to fix this issue could be risky, so I agree that 
reverting this PR (#18358) might be the best option for the 1.7.0 release.
But I'm considering is there any other test cases to cover this feature, which 
could be helpful to track this kind of code changes in future, or can you help 
to verify if this revert do resolve the broken issue at your side?

> Thus, the real issue is: Should we supporting pickling a Gluon Block? If not, 
> should we support combining multiprocessing.pool with the Gluon Block?
Seems it's more like a new feature for MXNet Gluon Block, probably we can make 
it available in the next patch/minor release?

Thanks,
-Ciyong

-Original Message-
From: Xingjian SHI  
Sent: Saturday, July 11, 2020 4:27 AM
To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Thanks Ziyi,

I've discovered the same issue when I'm trying to use AutoGluon with 1.7.0rc0 
and would like to share my finding:

Basically, I don't think Gluon Block is designed to be pickleble. But pickling 
do work for some cases in the old version:

I've included two cases in the gist 
(https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).

- Case1: we construct a gluon block, hybridize it and feed one NDArray to help 
initialize the block. After that, it will no longer be pickleble. 
- Case2: we just construct a gluon block and it will be pickleble in 1.6.0, but 
won't be pickleble in 1.7.0.

Thus, the real issue is: Should we supporting pickling a Gluon Block? If not, 
should we support combining multiprocessing.pool with the Gluon Block? For 
reference, PyTorch supports pickling the nn.Module as shown in: 
https://gist.github.com/sxjscience/90b812a66d445e759c55eedc3ef93668 and also in 
the doc (https://pytorch.org/tutorials/beginner/saving_loading_models.html). 

Best,
Xingjian


On 7/10/20, 11:31 AM, "Patrick Mu"  wrote:

Hi Ciyong, 

I just discovered an issue with the 1.7, which causes the Yolo training 
with latest Gluon CV Yolo to fail.

The PR that causes the failure is 
https://github.com/apache/incubator-mxnet/pull/18358, which modifies  basic 
blocks of Gluon to fix a memory leak issue.

Talked with Leonard, the author of the PR, and he said he found the root 
cause, but patching that PR would modifies those Gluon basic blocks further, 
which might be risky towards existing models and various customer models.

So my 2-cents is reverting this PR in 1.7, and try patching the PR in 1.x 
and 2.0, meaning that the 1.7 won't have memory usage optimized by that feature.

I'd like to hear what you think about this issue.

Thanks,
Ziyi


On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote: 
> Hi Community,
> 
> I would like to call for action to test/validate/vote for the release 
candidate (1.7.0.rc0)
> As there's not any voting result during the scheduled time window, I 
would like to extend the time windows to July 13, 23:59:59 PST.
> Please prepare your time and provide feedback if you've tried with the 
pre-release code bases, thanks!
> 
> Best regards,
> Ciyong
> 
> -Original Message-
> From: Chen, Ciyong  
> Sent: Monday, July 6, 2020 10:48 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; 
Jason Dai ; Markus Weimer ; Michael 
Wall 
> Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> For the language bindings and windows platform, may I have your support 
to help verify these features? Thanks!
> 
> @lanking520 to help verify the Scala/Java @gigasquid to help verify the 
Clojure
> @hetong007 to help verify the R
> @yajiedesign to help verify the windows platform
> 
> Best regards,
> Ciyong Chen
> 
> -Original Message-
> From: Chen, Ciyong 
> Sent: Monday, July 6, 2020 10:39 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; 
Jason Dai ; Markus Weimer ; Michael 
Wall 
> Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> Dear MXNet community,
> 
> This is the vote to release Apache MXNet (incubating) version 1.7.0. 
Voting will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.
> 
> Link to release notes:
> https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
> 
> Link

Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Skalicky, Sam]

+1

Tested:
- Make flow building from source: example/extensions all work correctly
- staticbuild flow cpu & cu102 variants with custom extension library

Sam

On 7/12/20, 1:52 PM, "Marco de Abreu"  wrote:

CAUTION: This email originated from outside of the organization. Do not 
click links or open attachments unless you can confirm the sender and know the 
content is safe.



Okay, thanks for the clarification!

-Marco

On Sun, Jul 12, 2020, 5:58 PM Sheng Zha  wrote:

> Hi Marco,
>
> Since the license issues apply to binary distribution, we should still be
> able to make official source releases.
>
> Regards,
> Sheng
>
> > On Jul 12, 2020, at 1:10 AM, Marco de Abreu 
> wrote:
> >
> > Are we in the position to make a release given that we have open license
> > issues with the ipmc and Apache board? I want to avoid giving the
> > impression that we are ignoring their requests - my current 
understanding
> > is that we are non compliant.
> >
> > -Marco
> >
> >> On Sat, Jul 11, 2020, 9:46 AM Tong He  wrote:
> >>
> >> My +1 on the R binding.
> >>
> >> Tested with
> >>
> >> - Build from source
> >> - Install the R package and check it passed all tests.
> >>
> >>> On 2020/07/10 18:31:27, Patrick Mu  wrote:
> >>> Hi Ciyong,
> >>>
> >>> I just discovered an issue with the 1.7, which causes the Yolo 
training
> >> with latest Gluon CV Yolo to fail.
> >>>
> >>> The PR that causes the failure is
> >> https://github.com/apache/incubator-mxnet/pull/18358, which modifies
> >> basic blocks of Gluon to fix a memory leak issue.
> >>>
> >>> Talked with Leonard, the author of the PR, and he said he found the
> root
> >> cause, but patching that PR would modifies those Gluon basic blocks
> >> further, which might be risky towards existing models and various
> customer
> >> models.
> >>>
> >>> So my 2-cents is reverting this PR in 1.7, and try patching the PR in
> >> 1.x and 2.0, meaning that the 1.7 won't have memory usage optimized by
> that
> >> feature.
> >>>
> >>> I'd like to hear what you think about this issue.
> >>>
> >>> Thanks,
> >>> Ziyi
> >>>
> >>>
> >>> On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote:
> >>>> Hi Community,
> >>>>
> >>>> I would like to call for action to test/validate/vote for the release
> >> candidate (1.7.0.rc0)
> >>>> As there's not any voting result during the scheduled time window, I
> >> would like to extend the time windows to July 13, 23:59:59 PST.
> >>>> Please prepare your time and provide feedback if you've tried with 
the
> >> pre-release code bases, thanks!
> >>>>
> >>>> Best regards,
> >>>> Ciyong
> >>>>
> >>>> -Original Message-
> >>>> From: Chen, Ciyong 
> >>>> Sent: Monday, July 6, 2020 10:48 PM
> >>>> To: d...@mxnet.apache.org
> >>>> Cc: Bob Paulin ; Henri Yandell ;
> >> Jason Dai ; Markus Weimer ;
> >> Michael Wall 
> >>>> Subject: RE: [VOTE] Release Apache MXNet (incubating) version
> 1.7.0.rc0
    > >>>>
> >>>> For the language bindings and windows platform, may I have your
> >> support to help verify these features? Thanks!
> >>>>
> >>>> @lanking520 to help verify the Scala/Java @gigasquid to help verify
> >> the Clojure
> >>>> @hetong007 to help verify the R
> >>>> @yajiedesign to help verify the windows platform
> >>>>
> >>>> Best regards,
> >>>> Ciyong Chen
> >>>>
> >>>> -Original Message-
> >>>> From: Chen, Ciyong 
> >>>> Sent: Monday, July 6, 2020 10:39 PM
> >>>> To: d...@mxnet.apache.org
> >>>> Cc: Bob Paulin ; Henri Yandell ;
> >> Jason Dai ; Markus Weimer ;
> >> Michael Wall 
> >>>> Subject: [VOTE] Release Apache MXNet (incubating) version 1.

Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Marco de Abreu]

Okay, thanks for the clarification!

-Marco

On Sun, Jul 12, 2020, 5:58 PM Sheng Zha  wrote:

> Hi Marco,
>
> Since the license issues apply to binary distribution, we should still be
> able to make official source releases.
>
> Regards,
> Sheng
>
> > On Jul 12, 2020, at 1:10 AM, Marco de Abreu 
> wrote:
> >
> > Are we in the position to make a release given that we have open license
> > issues with the ipmc and Apache board? I want to avoid giving the
> > impression that we are ignoring their requests - my current understanding
> > is that we are non compliant.
> >
> > -Marco
> >
> >> On Sat, Jul 11, 2020, 9:46 AM Tong He  wrote:
> >>
> >> My +1 on the R binding.
> >>
> >> Tested with
> >>
> >> - Build from source
> >> - Install the R package and check it passed all tests.
> >>
> >>> On 2020/07/10 18:31:27, Patrick Mu  wrote:
> >>> Hi Ciyong,
> >>>
> >>> I just discovered an issue with the 1.7, which causes the Yolo training
> >> with latest Gluon CV Yolo to fail.
> >>>
> >>> The PR that causes the failure is
> >> https://github.com/apache/incubator-mxnet/pull/18358, which modifies
> >> basic blocks of Gluon to fix a memory leak issue.
> >>>
> >>> Talked with Leonard, the author of the PR, and he said he found the
> root
> >> cause, but patching that PR would modifies those Gluon basic blocks
> >> further, which might be risky towards existing models and various
> customer
> >> models.
> >>>
> >>> So my 2-cents is reverting this PR in 1.7, and try patching the PR in
> >> 1.x and 2.0, meaning that the 1.7 won't have memory usage optimized by
> that
> >> feature.
> >>>
> >>> I'd like to hear what you think about this issue.
> >>>
> >>> Thanks,
> >>> Ziyi
> >>>
> >>>
> >>> On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote:
> >>>> Hi Community,
> >>>>
> >>>> I would like to call for action to test/validate/vote for the release
> >> candidate (1.7.0.rc0)
> >>>> As there's not any voting result during the scheduled time window, I
> >> would like to extend the time windows to July 13, 23:59:59 PST.
> >>>> Please prepare your time and provide feedback if you've tried with the
> >> pre-release code bases, thanks!
> >>>>
> >>>> Best regards,
> >>>> Ciyong
> >>>>
> >>>> -Original Message-
> >>>> From: Chen, Ciyong 
> >>>> Sent: Monday, July 6, 2020 10:48 PM
> >>>> To: d...@mxnet.apache.org
> >>>> Cc: Bob Paulin ; Henri Yandell ;
> >> Jason Dai ; Markus Weimer ;
> >> Michael Wall 
> >>>> Subject: RE: [VOTE] Release Apache MXNet (incubating) version
> 1.7.0.rc0
> >>>>
> >>>> For the language bindings and windows platform, may I have your
> >> support to help verify these features? Thanks!
> >>>>
> >>>> @lanking520 to help verify the Scala/Java @gigasquid to help verify
> >> the Clojure
> >>>> @hetong007 to help verify the R
> >>>> @yajiedesign to help verify the windows platform
> >>>>
> >>>> Best regards,
> >>>> Ciyong Chen
> >>>>
> >>>> -Original Message-
> >>>> From: Chen, Ciyong 
> >>>> Sent: Monday, July 6, 2020 10:39 PM
> >>>> To: d...@mxnet.apache.org
> >>>> Cc: Bob Paulin ; Henri Yandell ;
> >> Jason Dai ; Markus Weimer ;
> >> Michael Wall 
> >>>> Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> >>>>
> >>>> Dear MXNet community,
> >>>>
> >>>> This is the vote to release Apache MXNet (incubating) version 1.7.0.
> >> Voting will start July 6, 23:59:59 PST and close on July 9, 23:59:59
> PST.
> >>>>
> >>>> Link to release notes:
> >>>> https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
> >>>>
> >>>> Link to release candidate:
> >>>> https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0
> >>>>
> >>>> Link to source and signatures on apache dist server:
> >>>> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<
> >> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>
> >>>>
> >>>> Please remember to TEST first before voting accordingly:
> >>>> +1 = approve
> >>>> +0 = no opinion
> >>>> -1 = disapprove (provide reason)
> >>>>
> >>>> Additional notes:
> >>>>
> >>>>  *   There was an issue and discussion[1] regarding on a few numpy
> >> operators failed due to numpy 1.19.0 released on Jun 20, 2020, which
> exists
> >> in all branches (works with numpy <= 1.18.5). As numpy operator is
> still an
> >> experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0
> >> release, so I decided to not block the voting and instead let the
> Community
> >> decide whether this is a blocker for the release.
> >>>>
> >>>> [1] https://github.com/apache/incubator-mxnet/issues/18600
> >>>>
> >>>> Best regards,
> >>>> Ciyong Chen
> >>>>
> >>>>
> >>>
> >>
>

Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Sheng Zha]

Hi Marco,

Since the license issues apply to binary distribution, we should still be able 
to make official source releases.

Regards,
Sheng

> On Jul 12, 2020, at 1:10 AM, Marco de Abreu  wrote:
> 
> Are we in the position to make a release given that we have open license
> issues with the ipmc and Apache board? I want to avoid giving the
> impression that we are ignoring their requests - my current understanding
> is that we are non compliant.
> 
> -Marco
> 
>> On Sat, Jul 11, 2020, 9:46 AM Tong He  wrote:
>> 
>> My +1 on the R binding.
>> 
>> Tested with
>> 
>> - Build from source
>> - Install the R package and check it passed all tests.
>> 
>>> On 2020/07/10 18:31:27, Patrick Mu  wrote:
>>> Hi Ciyong,
>>> 
>>> I just discovered an issue with the 1.7, which causes the Yolo training
>> with latest Gluon CV Yolo to fail.
>>> 
>>> The PR that causes the failure is
>> https://github.com/apache/incubator-mxnet/pull/18358, which modifies
>> basic blocks of Gluon to fix a memory leak issue.
>>> 
>>> Talked with Leonard, the author of the PR, and he said he found the root
>> cause, but patching that PR would modifies those Gluon basic blocks
>> further, which might be risky towards existing models and various customer
>> models.
>>> 
>>> So my 2-cents is reverting this PR in 1.7, and try patching the PR in
>> 1.x and 2.0, meaning that the 1.7 won't have memory usage optimized by that
>> feature.
>>> 
>>> I'd like to hear what you think about this issue.
>>> 
>>> Thanks,
>>> Ziyi
>>> 
>>> 
>>> On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote:
>>>> Hi Community,
>>>> 
>>>> I would like to call for action to test/validate/vote for the release
>> candidate (1.7.0.rc0)
>>>> As there's not any voting result during the scheduled time window, I
>> would like to extend the time windows to July 13, 23:59:59 PST.
>>>> Please prepare your time and provide feedback if you've tried with the
>> pre-release code bases, thanks!
>>>> 
>>>> Best regards,
>>>> Ciyong
>>>> 
>>>> -Original Message-
>>>> From: Chen, Ciyong 
>>>> Sent: Monday, July 6, 2020 10:48 PM
>>>> To: d...@mxnet.apache.org
>>>> Cc: Bob Paulin ; Henri Yandell ;
>> Jason Dai ; Markus Weimer ;
>> Michael Wall 
>>>> Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
>>>> 
>>>> For the language bindings and windows platform, may I have your
>> support to help verify these features? Thanks!
>>>> 
>>>> @lanking520 to help verify the Scala/Java @gigasquid to help verify
>> the Clojure
>>>> @hetong007 to help verify the R
>>>> @yajiedesign to help verify the windows platform
>>>> 
>>>> Best regards,
>>>> Ciyong Chen
>>>> 
>>>> -Original Message-
>>>> From: Chen, Ciyong 
>>>> Sent: Monday, July 6, 2020 10:39 PM
>>>> To: d...@mxnet.apache.org
>>>> Cc: Bob Paulin ; Henri Yandell ;
>> Jason Dai ; Markus Weimer ;
>> Michael Wall 
>>>> Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
>>>> 
>>>> Dear MXNet community,
>>>> 
>>>> This is the vote to release Apache MXNet (incubating) version 1.7.0.
>> Voting will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.
>>>> 
>>>> Link to release notes:
>>>> https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
>>>> 
>>>> Link to release candidate:
>>>> https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0
>>>> 
>>>> Link to source and signatures on apache dist server:
>>>> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<
>> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>
>>>> 
>>>> Please remember to TEST first before voting accordingly:
>>>> +1 = approve
>>>> +0 = no opinion
>>>> -1 = disapprove (provide reason)
>>>> 
>>>> Additional notes:
>>>> 
>>>>  *   There was an issue and discussion[1] regarding on a few numpy
>> operators failed due to numpy 1.19.0 released on Jun 20, 2020, which exists
>> in all branches (works with numpy <= 1.18.5). As numpy operator is still an
>> experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0
>> release, so I decided to not block the voting and instead let the Community
>> decide whether this is a blocker for the release.
>>>> 
>>>> [1] https://github.com/apache/incubator-mxnet/issues/18600
>>>> 
>>>> Best regards,
>>>> Ciyong Chen
>>>> 
>>>> 
>>> 
>> 

Re: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Marco de Abreu]

Are we in the position to make a release given that we have open license
issues with the ipmc and Apache board? I want to avoid giving the
impression that we are ignoring their requests - my current understanding
is that we are non compliant.

-Marco

On Sat, Jul 11, 2020, 9:46 AM Tong He  wrote:

> My +1 on the R binding.
>
> Tested with
>
> - Build from source
> - Install the R package and check it passed all tests.
>
> On 2020/07/10 18:31:27, Patrick Mu  wrote:
> > Hi Ciyong,
> >
> > I just discovered an issue with the 1.7, which causes the Yolo training
> with latest Gluon CV Yolo to fail.
> >
> > The PR that causes the failure is
> https://github.com/apache/incubator-mxnet/pull/18358, which modifies
> basic blocks of Gluon to fix a memory leak issue.
> >
> > Talked with Leonard, the author of the PR, and he said he found the root
> cause, but patching that PR would modifies those Gluon basic blocks
> further, which might be risky towards existing models and various customer
> models.
> >
> > So my 2-cents is reverting this PR in 1.7, and try patching the PR in
> 1.x and 2.0, meaning that the 1.7 won't have memory usage optimized by that
> feature.
> >
> > I'd like to hear what you think about this issue.
> >
> > Thanks,
> > Ziyi
> >
> >
> > On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote:
> > > Hi Community,
> > >
> > > I would like to call for action to test/validate/vote for the release
> candidate (1.7.0.rc0)
> > > As there's not any voting result during the scheduled time window, I
> would like to extend the time windows to July 13, 23:59:59 PST.
> > > Please prepare your time and provide feedback if you've tried with the
> pre-release code bases, thanks!
> > >
> > > Best regards,
> > > Ciyong
> > >
> > > -Original Message-
> > > From: Chen, Ciyong 
> > > Sent: Monday, July 6, 2020 10:48 PM
> > > To: d...@mxnet.apache.org
> > > Cc: Bob Paulin ; Henri Yandell ;
> Jason Dai ; Markus Weimer ;
> Michael Wall 
> > > Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> > >
> > > For the language bindings and windows platform, may I have your
> support to help verify these features? Thanks!
> > >
> > > @lanking520 to help verify the Scala/Java @gigasquid to help verify
> the Clojure
> > > @hetong007 to help verify the R
> > > @yajiedesign to help verify the windows platform
> > >
> > > Best regards,
> > > Ciyong Chen
> > >
> > > -Original Message-
> > > From: Chen, Ciyong 
> > > Sent: Monday, July 6, 2020 10:39 PM
> > > To: d...@mxnet.apache.org
> > > Cc: Bob Paulin ; Henri Yandell ;
> Jason Dai ; Markus Weimer ;
> Michael Wall 
> > > Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> > >
> > > Dear MXNet community,
> > >
> > > This is the vote to release Apache MXNet (incubating) version 1.7.0.
> Voting will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.
> > >
> > > Link to release notes:
> > > https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
> > >
> > > Link to release candidate:
> > > https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0
> > >
> > > Link to source and signatures on apache dist server:
> > > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>
> > >
> > > Please remember to TEST first before voting accordingly:
> > > +1 = approve
> > > +0 = no opinion
> > > -1 = disapprove (provide reason)
> > >
> > > Additional notes:
> > >
> > >   *   There was an issue and discussion[1] regarding on a few numpy
> operators failed due to numpy 1.19.0 released on Jun 20, 2020, which exists
> in all branches (works with numpy <= 1.18.5). As numpy operator is still an
> experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0
> release, so I decided to not block the voting and instead let the Community
> decide whether this is a blocker for the release.
> > >
> > > [1] https://github.com/apache/incubator-mxnet/issues/18600
> > >
> > > Best regards,
> > > Ciyong Chen
> > >
> > >
> >
>

Re: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Tong He]

My +1 on the R binding.

Tested with

- Build from source
- Install the R package and check it passed all tests.

On 2020/07/10 18:31:27, Patrick Mu  wrote: 
> Hi Ciyong, 
> 
> I just discovered an issue with the 1.7, which causes the Yolo training with 
> latest Gluon CV Yolo to fail.
> 
> The PR that causes the failure is 
> https://github.com/apache/incubator-mxnet/pull/18358, which modifies  basic 
> blocks of Gluon to fix a memory leak issue.
> 
> Talked with Leonard, the author of the PR, and he said he found the root 
> cause, but patching that PR would modifies those Gluon basic blocks further, 
> which might be risky towards existing models and various customer models.
> 
> So my 2-cents is reverting this PR in 1.7, and try patching the PR in 1.x and 
> 2.0, meaning that the 1.7 won't have memory usage optimized by that feature.
> 
> I'd like to hear what you think about this issue.
> 
> Thanks,
> Ziyi
> 
> 
> On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote: 
> > Hi Community,
> > 
> > I would like to call for action to test/validate/vote for the release 
> > candidate (1.7.0.rc0)
> > As there's not any voting result during the scheduled time window, I would 
> > like to extend the time windows to July 13, 23:59:59 PST.
> > Please prepare your time and provide feedback if you've tried with the 
> > pre-release code bases, thanks!
> > 
> > Best regards,
> > Ciyong
> > 
> > -Original Message-
> > From: Chen, Ciyong  
> > Sent: Monday, July 6, 2020 10:48 PM
> > To: d...@mxnet.apache.org
> > Cc: Bob Paulin ; Henri Yandell ; Jason 
> > Dai ; Markus Weimer ; Michael Wall 
> > 
> > Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> > 
> > For the language bindings and windows platform, may I have your support to 
> > help verify these features? Thanks!
> > 
> > @lanking520 to help verify the Scala/Java @gigasquid to help verify the 
> > Clojure
> > @hetong007 to help verify the R
> > @yajiedesign to help verify the windows platform
> > 
> > Best regards,
> > Ciyong Chen
> > 
> > -----Original Message-
> > From: Chen, Ciyong 
> > Sent: Monday, July 6, 2020 10:39 PM
> > To: d...@mxnet.apache.org
> > Cc: Bob Paulin ; Henri Yandell ; Jason 
> > Dai ; Markus Weimer ; Michael Wall 
> > 
> > Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> > 
> > Dear MXNet community,
> > 
> > This is the vote to release Apache MXNet (incubating) version 1.7.0. Voting 
> > will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.
> > 
> > Link to release notes:
> > https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
> > 
> > Link to release candidate:
> > https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0
> > 
> > Link to source and signatures on apache dist server:
> > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>
> > 
> > Please remember to TEST first before voting accordingly:
> > +1 = approve
> > +0 = no opinion
> > -1 = disapprove (provide reason)
> > 
> > Additional notes:
> > 
> >   *   There was an issue and discussion[1] regarding on a few numpy 
> > operators failed due to numpy 1.19.0 released on Jun 20, 2020, which exists 
> > in all branches (works with numpy <= 1.18.5). As numpy operator is still an 
> > experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0 
> > release, so I decided to not block the voting and instead let the Community 
> > decide whether this is a blocker for the release.
> > 
> > [1] https://github.com/apache/incubator-mxnet/issues/18600
> > 
> > Best regards,
> > Ciyong Chen
> > 
> > 
> 

Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Xingjian SHI]

-1 (binding). 

It breaks the YOLO training in GluonCV and the basic image classification in 
AutoGluon. There is an open PR that reverts the `weakref` fix: 
https://github.com/apache/incubator-mxnet/pull/18692 and we need to see whether 
to revert that fix or find some other ways to solve the issue.

Given the current status, we should not release 1.7.0rc. Thus, I voted for -1.

Best,
Xingjian

On 7/10/20, 9:46 PM, "Aston Zhang"  wrote:

+1

Tested:

mxnet 1.7.0rc0-cu101 passed d2l-en v0.14.0

On Fri, Jul 10, 2020 at 12:27 PM Qing Lan  wrote:

> My +1 (binding) on 1.7.0
>
> Tested:
>
>   *   Build from Source with static build instruction
>   *   Tested Scala pacakge and passed all tests
>
> Thanks,
> Qing
>
> 
> From: Tao Lv 
> Sent: Friday, July 10, 2020 0:03
> To: dev@mxnet.incubator.apache.org 
> Cc: d...@mxnet.apache.org ; Bob Paulin <
> b...@apache.org>; Henri Yandell ; Jason Dai <
> jason...@apache.org>; Markus Weimer ; Michael Wall <
> mjw...@apache.org>
> Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
>
> +1 (binding)
>
> I did:
> - Verify the key and signature;
> - Untar the source code package;
> - Build from source code with makefile, USE_BLAS=mkl, USE_MKLDNN=1;
> - Check mx.__version__;
> - Run benchmark_score.py under examples/image-classification.
>
> -tao
>
> On Fri, Jul 10, 2020 at 2:18 PM Chen, Ciyong 
> wrote:
>
> > Hi Community,
> >
> > I would like to call for action to test/validate/vote for the release
> > candidate (1.7.0.rc0)
> > As there's not any voting result during the scheduled time window, I
> would
> > like to extend the time windows to July 13, 23:59:59 PST.
> > Please prepare your time and provide feedback if you've tried with the
> > pre-release code bases, thanks!
> >
> > Best regards,
> > Ciyong
> >
> > -Original Message-
> > From: Chen, Ciyong 
    > > Sent: Monday, July 6, 2020 10:48 PM
> > To: d...@mxnet.apache.org
> > Cc: Bob Paulin ; Henri Yandell ;
> Jason
> > Dai ; Markus Weimer ; Michael
> > Wall 
> > Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> >
> > For the language bindings and windows platform, may I have your support
> to
> > help verify these features? Thanks!
> >
> > @lanking520 to help verify the Scala/Java @gigasquid to help verify the
> > Clojure
> > @hetong007 to help verify the R
> > @yajiedesign to help verify the windows platform
> >
    > > Best regards,
    > > Ciyong Chen
    > >
> > -Original Message-
> > From: Chen, Ciyong 
> > Sent: Monday, July 6, 2020 10:39 PM
> > To: d...@mxnet.apache.org
> > Cc: Bob Paulin ; Henri Yandell ;
> Jason
> > Dai ; Markus Weimer ; Michael
> > Wall 
> > Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> >
> > Dear MXNet community,
> >
> > This is the vote to release Apache MXNet (incubating) version 1.7.0.
> > Voting will start July 6, 23:59:59 PST and close on July 9, 23:59:59 
PST.
> >
> > Link to release notes:
> > https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
> >
> > Link to release candidate:
> > https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0
> >
> > Link to source and signatures on apache dist server:
> > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<
> > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>
> >
> > Please remember to TEST first before voting accordingly:
> > +1 = approve
> > +0 = no opinion
> > -1 = disapprove (provide reason)
> >
> > Additional notes:
> >
> >   *   There was an issue and discussion[1] regarding on a few numpy
> > operators failed due to numpy 1.19.0 released on Jun 20, 2020, which
> exists
> > in all branches (works with numpy <= 1.18.5). As numpy operator is still
> an
> > experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0
> > release, so I decided to not block the voting and instead let the
> Community
> > decide whether this is a blocker for the release.
> >
> > [1] https://github.com/apache/incubator-mxnet/issues/18600
> >
> > Best regards,
> > Ciyong Chen
> >
> >
>

Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Aston Zhang]

+1

Tested:

mxnet 1.7.0rc0-cu101 passed d2l-en v0.14.0

On Fri, Jul 10, 2020 at 12:27 PM Qing Lan  wrote:

> My +1 (binding) on 1.7.0
>
> Tested:
>
>   *   Build from Source with static build instruction
>   *   Tested Scala pacakge and passed all tests
>
> Thanks,
> Qing
>
> 
> From: Tao Lv 
> Sent: Friday, July 10, 2020 0:03
> To: dev@mxnet.incubator.apache.org 
> Cc: d...@mxnet.apache.org ; Bob Paulin <
> b...@apache.org>; Henri Yandell ; Jason Dai <
> jason...@apache.org>; Markus Weimer ; Michael Wall <
> mjw...@apache.org>
> Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
>
> +1 (binding)
>
> I did:
> - Verify the key and signature;
> - Untar the source code package;
> - Build from source code with makefile, USE_BLAS=mkl, USE_MKLDNN=1;
> - Check mx.__version__;
> - Run benchmark_score.py under examples/image-classification.
>
> -tao
>
> On Fri, Jul 10, 2020 at 2:18 PM Chen, Ciyong 
> wrote:
>
> > Hi Community,
> >
> > I would like to call for action to test/validate/vote for the release
> > candidate (1.7.0.rc0)
> > As there's not any voting result during the scheduled time window, I
> would
> > like to extend the time windows to July 13, 23:59:59 PST.
> > Please prepare your time and provide feedback if you've tried with the
> > pre-release code bases, thanks!
> >
> > Best regards,
> > Ciyong
> >
> > -Original Message-
> > From: Chen, Ciyong 
> > Sent: Monday, July 6, 2020 10:48 PM
> > To: d...@mxnet.apache.org
> > Cc: Bob Paulin ; Henri Yandell ;
> Jason
> > Dai ; Markus Weimer ; Michael
> > Wall 
> > Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> >
> > For the language bindings and windows platform, may I have your support
> to
> > help verify these features? Thanks!
> >
> > @lanking520 to help verify the Scala/Java @gigasquid to help verify the
> > Clojure
> > @hetong007 to help verify the R
> > @yajiedesign to help verify the windows platform
> >
> > Best regards,
> > Ciyong Chen
> >
> > -Original Message-
> > From: Chen, Ciyong 
> > Sent: Monday, July 6, 2020 10:39 PM
> > To: d...@mxnet.apache.org
> > Cc: Bob Paulin ; Henri Yandell ;
> Jason
> > Dai ; Markus Weimer ; Michael
> > Wall 
> > Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> >
> > Dear MXNet community,
> >
> > This is the vote to release Apache MXNet (incubating) version 1.7.0.
> > Voting will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.
> >
> > Link to release notes:
> > https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
> >
> > Link to release candidate:
> > https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0
> >
> > Link to source and signatures on apache dist server:
> > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<
> > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>
> >
> > Please remember to TEST first before voting accordingly:
> > +1 = approve
> > +0 = no opinion
> > -1 = disapprove (provide reason)
> >
> > Additional notes:
> >
> >   *   There was an issue and discussion[1] regarding on a few numpy
> > operators failed due to numpy 1.19.0 released on Jun 20, 2020, which
> exists
> > in all branches (works with numpy <= 1.18.5). As numpy operator is still
> an
> > experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0
> > release, so I decided to not block the voting and instead let the
> Community
> > decide whether this is a blocker for the release.
> >
> > [1] https://github.com/apache/incubator-mxnet/issues/18600
> >
> > Best regards,
> > Ciyong Chen
> >
> >
>

Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Xingjian SHI]

Thanks Ziyi,

I've discovered the same issue when I'm trying to use AutoGluon with 1.7.0rc0 
and would like to share my finding:

Basically, I don't think Gluon Block is designed to be pickleble. But pickling 
do work for some cases in the old version:

I've included two cases in the gist 
(https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).

- Case1: we construct a gluon block, hybridize it and feed one NDArray to help 
initialize the block. After that, it will no longer be pickleble. 
- Case2: we just construct a gluon block and it will be pickleble in 1.6.0, but 
won't be pickleble in 1.7.0.

Thus, the real issue is: Should we supporting pickling a Gluon Block? If not, 
should we support combining multiprocessing.pool with the Gluon Block? For 
reference, PyTorch supports pickling the nn.Module as shown in: 
https://gist.github.com/sxjscience/90b812a66d445e759c55eedc3ef93668 and also in 
the doc (https://pytorch.org/tutorials/beginner/saving_loading_models.html). 

Best,
Xingjian


On 7/10/20, 11:31 AM, "Patrick Mu"  wrote:

Hi Ciyong, 

I just discovered an issue with the 1.7, which causes the Yolo training 
with latest Gluon CV Yolo to fail.

The PR that causes the failure is 
https://github.com/apache/incubator-mxnet/pull/18358, which modifies  basic 
blocks of Gluon to fix a memory leak issue.

Talked with Leonard, the author of the PR, and he said he found the root 
cause, but patching that PR would modifies those Gluon basic blocks further, 
which might be risky towards existing models and various customer models.

So my 2-cents is reverting this PR in 1.7, and try patching the PR in 1.x 
and 2.0, meaning that the 1.7 won't have memory usage optimized by that feature.

I'd like to hear what you think about this issue.

Thanks,
Ziyi


On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote: 
> Hi Community,
> 
> I would like to call for action to test/validate/vote for the release 
candidate (1.7.0.rc0)
> As there's not any voting result during the scheduled time window, I 
would like to extend the time windows to July 13, 23:59:59 PST.
> Please prepare your time and provide feedback if you've tried with the 
pre-release code bases, thanks!
> 
> Best regards,
> Ciyong
> 
> -Original Message-
> From: Chen, Ciyong  
> Sent: Monday, July 6, 2020 10:48 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; 
Jason Dai ; Markus Weimer ; Michael 
Wall 
> Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> For the language bindings and windows platform, may I have your support 
to help verify these features? Thanks!
> 
> @lanking520 to help verify the Scala/Java @gigasquid to help verify the 
Clojure
> @hetong007 to help verify the R
> @yajiedesign to help verify the windows platform
> 
> Best regards,
> Ciyong Chen
> 
> -Original Message-
> From: Chen, Ciyong 
> Sent: Monday, July 6, 2020 10:39 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; 
Jason Dai ; Markus Weimer ; Michael 
Wall 
> Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> Dear MXNet community,
> 
> This is the vote to release Apache MXNet (incubating) version 1.7.0. 
Voting will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.
> 
> Link to release notes:
> https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
> 
> Link to release candidate:
> https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0
> 
> Link to source and signatures on apache dist server:
> 
https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>
> 
> Please remember to TEST first before voting accordingly:
> +1 = approve
> +0 = no opinion
> -1 = disapprove (provide reason)
> 
> Additional notes:
> 
>   *   There was an issue and discussion[1] regarding on a few numpy 
operators failed due to numpy 1.19.0 released on Jun 20, 2020, which exists in 
all branches (works with numpy <= 1.18.5). As numpy operator is still an 
experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0 
release, so I decided to not block the voting and instead let the Community 
decide whether this is a blocker for the release.
> 
> [1] https://github.com/apache/incubator-mxnet/issues/18600
> 
> Best regards,
> Ciyong Chen
> 
> 

Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Qing Lan]

My +1 (binding) on 1.7.0

Tested:

  *   Build from Source with static build instruction
  *   Tested Scala pacakge and passed all tests

Thanks,
Qing


From: Tao Lv 
Sent: Friday, July 10, 2020 0:03
To: dev@mxnet.incubator.apache.org 
Cc: d...@mxnet.apache.org ; Bob Paulin 
; Henri Yandell ; Jason Dai 
; Markus Weimer ; Michael Wall 

Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

+1 (binding)

I did:
- Verify the key and signature;
- Untar the source code package;
- Build from source code with makefile, USE_BLAS=mkl, USE_MKLDNN=1;
- Check mx.__version__;
- Run benchmark_score.py under examples/image-classification.

-tao

On Fri, Jul 10, 2020 at 2:18 PM Chen, Ciyong  wrote:

> Hi Community,
>
> I would like to call for action to test/validate/vote for the release
> candidate (1.7.0.rc0)
> As there's not any voting result during the scheduled time window, I would
> like to extend the time windows to July 13, 23:59:59 PST.
> Please prepare your time and provide feedback if you've tried with the
> pre-release code bases, thanks!
>
> Best regards,
> Ciyong
>
> -Original Message-
> From: Chen, Ciyong 
> Sent: Monday, July 6, 2020 10:48 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; Jason
> Dai ; Markus Weimer ; Michael
> Wall 
> Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
>
> For the language bindings and windows platform, may I have your support to
> help verify these features? Thanks!
>
> @lanking520 to help verify the Scala/Java @gigasquid to help verify the
> Clojure
> @hetong007 to help verify the R
> @yajiedesign to help verify the windows platform
>
> Best regards,
> Ciyong Chen
>
> -Original Message-
> From: Chen, Ciyong 
> Sent: Monday, July 6, 2020 10:39 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; Jason
> Dai ; Markus Weimer ; Michael
> Wall 
> Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
>
> Dear MXNet community,
>
> This is the vote to release Apache MXNet (incubating) version 1.7.0.
> Voting will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.
>
> Link to release notes:
> https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
>
> Link to release candidate:
> https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0
>
> Link to source and signatures on apache dist server:
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>
>
> Please remember to TEST first before voting accordingly:
> +1 = approve
> +0 = no opinion
> -1 = disapprove (provide reason)
>
> Additional notes:
>
>   *   There was an issue and discussion[1] regarding on a few numpy
> operators failed due to numpy 1.19.0 released on Jun 20, 2020, which exists
> in all branches (works with numpy <= 1.18.5). As numpy operator is still an
> experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0
> release, so I decided to not block the voting and instead let the Community
> decide whether this is a blocker for the release.
>
> [1] https://github.com/apache/incubator-mxnet/issues/18600
>
> Best regards,
> Ciyong Chen
>
>

Re: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Patrick Mu]

Hi Ciyong, 

I just discovered an issue with the 1.7, which causes the Yolo training with 
latest Gluon CV Yolo to fail.

The PR that causes the failure is 
https://github.com/apache/incubator-mxnet/pull/18358, which modifies  basic 
blocks of Gluon to fix a memory leak issue.

Talked with Leonard, the author of the PR, and he said he found the root cause, 
but patching that PR would modifies those Gluon basic blocks further, which 
might be risky towards existing models and various customer models.

So my 2-cents is reverting this PR in 1.7, and try patching the PR in 1.x and 
2.0, meaning that the 1.7 won't have memory usage optimized by that feature.

I'd like to hear what you think about this issue.

Thanks,
Ziyi


On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote: 
> Hi Community,
> 
> I would like to call for action to test/validate/vote for the release 
> candidate (1.7.0.rc0)
> As there's not any voting result during the scheduled time window, I would 
> like to extend the time windows to July 13, 23:59:59 PST.
> Please prepare your time and provide feedback if you've tried with the 
> pre-release code bases, thanks!
> 
> Best regards,
> Ciyong
> 
> -Original Message-
> From: Chen, Ciyong  
> Sent: Monday, July 6, 2020 10:48 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; Jason 
> Dai ; Markus Weimer ; Michael Wall 
> 
> Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> For the language bindings and windows platform, may I have your support to 
> help verify these features? Thanks!
> 
> @lanking520 to help verify the Scala/Java @gigasquid to help verify the 
> Clojure
> @hetong007 to help verify the R
> @yajiedesign to help verify the windows platform
> 
> Best regards,
> Ciyong Chen
> 
> -Original Message-
> From: Chen, Ciyong 
> Sent: Monday, July 6, 2020 10:39 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; Jason 
> Dai ; Markus Weimer ; Michael Wall 
> 
> Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> Dear MXNet community,
> 
> This is the vote to release Apache MXNet (incubating) version 1.7.0. Voting 
> will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.
> 
> Link to release notes:
> https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
> 
> Link to release candidate:
> https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0
> 
> Link to source and signatures on apache dist server:
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>
> 
> Please remember to TEST first before voting accordingly:
> +1 = approve
> +0 = no opinion
> -1 = disapprove (provide reason)
> 
> Additional notes:
> 
>   *   There was an issue and discussion[1] regarding on a few numpy operators 
> failed due to numpy 1.19.0 released on Jun 20, 2020, which exists in all 
> branches (works with numpy <= 1.18.5). As numpy operator is still an 
> experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0 
> release, so I decided to not block the voting and instead let the Community 
> decide whether this is a blocker for the release.
> 
> [1] https://github.com/apache/incubator-mxnet/issues/18600
> 
> Best regards,
> Ciyong Chen
> 
> 

Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Tao Lv]

+1 (binding)

I did:
- Verify the key and signature;
- Untar the source code package;
- Build from source code with makefile, USE_BLAS=mkl, USE_MKLDNN=1;
- Check mx.__version__;
- Run benchmark_score.py under examples/image-classification.

-tao

On Fri, Jul 10, 2020 at 2:18 PM Chen, Ciyong  wrote:

> Hi Community,
>
> I would like to call for action to test/validate/vote for the release
> candidate (1.7.0.rc0)
> As there's not any voting result during the scheduled time window, I would
> like to extend the time windows to July 13, 23:59:59 PST.
> Please prepare your time and provide feedback if you've tried with the
> pre-release code bases, thanks!
>
> Best regards,
> Ciyong
>
> -Original Message-
> From: Chen, Ciyong 
> Sent: Monday, July 6, 2020 10:48 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; Jason
> Dai ; Markus Weimer ; Michael
> Wall 
> Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
>
> For the language bindings and windows platform, may I have your support to
> help verify these features? Thanks!
>
> @lanking520 to help verify the Scala/Java @gigasquid to help verify the
> Clojure
> @hetong007 to help verify the R
> @yajiedesign to help verify the windows platform
>
> Best regards,
> Ciyong Chen
>
> -Original Message-
> From: Chen, Ciyong 
> Sent: Monday, July 6, 2020 10:39 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; Jason
> Dai ; Markus Weimer ; Michael
> Wall 
> Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
>
> Dear MXNet community,
>
> This is the vote to release Apache MXNet (incubating) version 1.7.0.
> Voting will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.
>
> Link to release notes:
> https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
>
> Link to release candidate:
> https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0
>
> Link to source and signatures on apache dist server:
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>
>
> Please remember to TEST first before voting accordingly:
> +1 = approve
> +0 = no opinion
> -1 = disapprove (provide reason)
>
> Additional notes:
>
>   *   There was an issue and discussion[1] regarding on a few numpy
> operators failed due to numpy 1.19.0 released on Jun 20, 2020, which exists
> in all branches (works with numpy <= 1.18.5). As numpy operator is still an
> experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0
> release, so I decided to not block the voting and instead let the Community
> decide whether this is a blocker for the release.
>
> [1] https://github.com/apache/incubator-mxnet/issues/18600
>
> Best regards,
> Ciyong Chen
>
>

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Hi Community,

I would like to call for action to test/validate/vote for the release candidate 
(1.7.0.rc0)
As there's not any voting result during the scheduled time window, I would like 
to extend the time windows to July 13, 23:59:59 PST.
Please prepare your time and provide feedback if you've tried with the 
pre-release code bases, thanks!

Best regards,
Ciyong

-Original Message-
From: Chen, Ciyong  
Sent: Monday, July 6, 2020 10:48 PM
To: d...@mxnet.apache.org
Cc: Bob Paulin ; Henri Yandell ; Jason Dai 
; Markus Weimer ; Michael Wall 

Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

For the language bindings and windows platform, may I have your support to help 
verify these features? Thanks!

@lanking520 to help verify the Scala/Java @gigasquid to help verify the Clojure
@hetong007 to help verify the R
@yajiedesign to help verify the windows platform

Best regards,
Ciyong Chen

-Original Message-
From: Chen, Ciyong 
Sent: Monday, July 6, 2020 10:39 PM
To: d...@mxnet.apache.org
Cc: Bob Paulin ; Henri Yandell ; Jason Dai 
; Markus Weimer ; Michael Wall 

Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Dear MXNet community,

This is the vote to release Apache MXNet (incubating) version 1.7.0. Voting 
will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.

Link to release notes:
https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes

Link to release candidate:
https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0

Link to source and signatures on apache dist server:
https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>

Please remember to TEST first before voting accordingly:
+1 = approve
+0 = no opinion
-1 = disapprove (provide reason)

Additional notes:

  *   There was an issue and discussion[1] regarding on a few numpy operators 
failed due to numpy 1.19.0 released on Jun 20, 2020, which exists in all 
branches (works with numpy <= 1.18.5). As numpy operator is still an 
experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0 
release, so I decided to not block the voting and instead let the Community 
decide whether this is a blocker for the release.

[1] https://github.com/apache/incubator-mxnet/issues/18600

Best regards,
Ciyong Chen

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

For the language bindings and windows platform, may I have your support to help 
verify these features? Thanks!

@lanking520 to help verify the Scala/Java
@gigasquid to help verify the Clojure
@hetong007 to help verify the R
@yajiedesign to help verify the windows platform

Best regards,
Ciyong Chen

-Original Message-
From: Chen, Ciyong  
Sent: Monday, July 6, 2020 10:39 PM
To: d...@mxnet.apache.org
Cc: Bob Paulin ; Henri Yandell ; Jason Dai 
; Markus Weimer ; Michael Wall 

Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Dear MXNet community,

This is the vote to release Apache MXNet (incubating) version 1.7.0. Voting 
will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.

Link to release notes:
https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes

Link to release candidate:
https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0

Link to source and signatures on apache dist server:
https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>

Please remember to TEST first before voting accordingly:
+1 = approve
+0 = no opinion
-1 = disapprove (provide reason)

Additional notes:

  *   There was an issue and discussion[1] regarding on a few numpy operators 
failed due to numpy 1.19.0 released on Jun 20, 2020, which exists in all 
branches (works with numpy <= 1.18.5). As numpy operator is still an 
experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0 
release, so I decided to not block the voting and instead let the Community 
decide whether this is a blocker for the release.

[1] https://github.com/apache/incubator-mxnet/issues/18600

Best regards,
Ciyong Chen

[VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Dear MXNet community,

This is the vote to release Apache MXNet (incubating) version 1.7.0. Voting 
will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.

Link to release notes:
https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes

Link to release candidate:
https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0

Link to source and signatures on apache dist server:
https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0

Please remember to TEST first before voting accordingly:
+1 = approve
+0 = no opinion
-1 = disapprove (provide reason)

Additional notes:

  *   There was an issue and discussion[1] regarding on a few numpy operators 
failed due to numpy 1.19.0 released on Jun 20, 2020, which exists in all 
branches (works with numpy <= 1.18.5). As numpy operator is still an 
experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0 
release, so I decided to not block the voting and instead let the Community 
decide whether this is a blocker for the release.

[1] https://github.com/apache/incubator-mxnet/issues/18600

Best regards,
Ciyong Chen