Re: MXNet CD pipelines cost savings

2020-02-14 Thread Marco de Abreu 
Excellent, thanks a lot for your efforts, sheng!

-Marco

Sheng Zha  schrieb am Mi., 12. Feb. 2020, 22:57:

> Hi Marco,
>
> I've taken up the task to fix the CD pipeline [1] and it's pending CI
> checks and merge. I also made the adjustments to the namespace layout of
> the mxnet s3 bucket and updated the static page which is now accessible
> from https://repo.mxnet.io/dist/index.html.
>
> Meanwhile, the access from the CodeBuild to publish to the mxnet s3 bucket
> has been revoked so its status should no longer be relevant. I will update
> the installation doc to reflect only the artifacts published by Jenkins CD.
> I hope this brings conclusion to this situation.
>
> Let me know if there's any further question.
>
> -sz
>
> [1] https://github.com/apache/incubator-mxnet/pull/17551
>
> On 2020/02/10 23:57:47, Marco de Abreu  wrote:
> > Thanks for bringing this to our attention.
> >
> > I'm quite devastated that our two vetoes have been ignored and the
> > CodeBuild pipeline is actually supplying our user-facing binaries.
> > Suggesting to turn of the Jenkins based CD now adds insult to injury.
> >
> > I'd like to hear a plan how to make the project compliant again. I
> already
> > announced that I will remove any mentions of that publishing method
> (speak,
> > all links on our website pointing to the bucket) if the sourcing system
> is
> > not our Jenkins CD. So far I believed that this was actually done, but
> > seems like we got played here.
> >
> > For the sake of our users, I'm giving one week (until 2/17) to come up
> with
> > a proposal and until the end of the month 2/29 to have CodeBuild turned
> off
> > and Jenkins CD fixed. Considering we have been tricked last time, I want
> to
> > have confirmation that CodeBuild has been turned off and a description
> how
> > we can verify that all artifacts are now coming from Jenkins CD.
> >
> > Best regards
> > Marco
> >
> > Zha, Sheng  schrieb am Mo., 10. Feb. 2020,
> > 19:40:
> >
> > > +dev@
> > >
> > > -sz
> > >
> > > On Feb 10, 2020, at 1:35 PM, Zha, Sheng  wrote:
> > >
> > >  As already stated in the public threads, I’ve vetoed the CodeBuild
> > > solution from becoming the long term solution as it’s not publicly
> > > manageable.
> > >
> > > As communicated before, the team should have put efforts in maintaining
> > > and fixing the Jenkins CD pipeline but has neglected to do so.
> Promoting
> > > the CodeBuild solution this way is a step in the wrong direction that
> has
> > > to be stopped.
> > >
> > > -sz
> > >
> > > On Feb 10, 2020, at 1:13 PM, Davydenko, Denis  wrote:
> > >
> > > 
> > > Hello guys,
> > >
> > > I would like to start this discussion so that we can align on handling
> CD
> > > pipelines we currently have. There are two of them: one in Jenkins<
> > > http://jenkins.mxnet-ci.amazon-ml.com/job/restricted-mxnet-cd/> and
> one
> > > in CodeBuild. The one in
> > > Jenkins is currently functioning but its runs are always failing. The
> one
> > > in CodeBuild is currently functioning and publishing artifacts to S3
> bucket<
> > > https://tiny.amazon.com/39negmk0/IsenLink>.
> > >
> > > MXNet Engineering team proposal is to shut down Jenkins based CD
> > > completely as it is currently just a waste of resources and use
> CodeBuild
> > > based setup to continue publishing nightly builds to S3 bucket, which
> > > provides public access to all binaries stored in it. This doesn’t
> affect a
> > > discussion of whether to publish binaries to S3 or to pypi – once that
> > > concludes (if ever) we can switch destination of CodeBuild projects so
> that
> > > they would upload MXNet nightly binaries to pypi instead of S3.
> > >
> > > This is an effort to get alignment internally, if possible, before
> > > bringing this as a proposal for community discussion.
> > >
> > > --
> > > Thanks,
> > > Denis
> > >
> >
>

Re: MXNet CD pipelines cost savings

2020-02-12 Thread Sheng Zha 
Hi Marco,

I've taken up the task to fix the CD pipeline [1] and it's pending CI checks 
and merge. I also made the adjustments to the namespace layout of the mxnet s3 
bucket and updated the static page which is now accessible from 
https://repo.mxnet.io/dist/index.html.

Meanwhile, the access from the CodeBuild to publish to the mxnet s3 bucket has 
been revoked so its status should no longer be relevant. I will update the 
installation doc to reflect only the artifacts published by Jenkins CD. I hope 
this brings conclusion to this situation.

Let me know if there's any further question.

-sz

[1] https://github.com/apache/incubator-mxnet/pull/17551

On 2020/02/10 23:57:47, Marco de Abreu  wrote: 
> Thanks for bringing this to our attention.
> 
> I'm quite devastated that our two vetoes have been ignored and the
> CodeBuild pipeline is actually supplying our user-facing binaries.
> Suggesting to turn of the Jenkins based CD now adds insult to injury.
> 
> I'd like to hear a plan how to make the project compliant again. I already
> announced that I will remove any mentions of that publishing method (speak,
> all links on our website pointing to the bucket) if the sourcing system is
> not our Jenkins CD. So far I believed that this was actually done, but
> seems like we got played here.
> 
> For the sake of our users, I'm giving one week (until 2/17) to come up with
> a proposal and until the end of the month 2/29 to have CodeBuild turned off
> and Jenkins CD fixed. Considering we have been tricked last time, I want to
> have confirmation that CodeBuild has been turned off and a description how
> we can verify that all artifacts are now coming from Jenkins CD.
> 
> Best regards
> Marco
> 
> Zha, Sheng  schrieb am Mo., 10. Feb. 2020,
> 19:40:
> 
> > +dev@
> >
> > -sz
> >
> > On Feb 10, 2020, at 1:35 PM, Zha, Sheng  wrote:
> >
> >  As already stated in the public threads, I’ve vetoed the CodeBuild
> > solution from becoming the long term solution as it’s not publicly
> > manageable.
> >
> > As communicated before, the team should have put efforts in maintaining
> > and fixing the Jenkins CD pipeline but has neglected to do so. Promoting
> > the CodeBuild solution this way is a step in the wrong direction that has
> > to be stopped.
> >
> > -sz
> >
> > On Feb 10, 2020, at 1:13 PM, Davydenko, Denis  wrote:
> >
> > 
> > Hello guys,
> >
> > I would like to start this discussion so that we can align on handling CD
> > pipelines we currently have. There are two of them: one in Jenkins<
> > http://jenkins.mxnet-ci.amazon-ml.com/job/restricted-mxnet-cd/> and one
> > in CodeBuild. The one in
> > Jenkins is currently functioning but its runs are always failing. The one
> > in CodeBuild is currently functioning and publishing artifacts to S3 bucket<
> > https://tiny.amazon.com/39negmk0/IsenLink>.
> >
> > MXNet Engineering team proposal is to shut down Jenkins based CD
> > completely as it is currently just a waste of resources and use CodeBuild
> > based setup to continue publishing nightly builds to S3 bucket, which
> > provides public access to all binaries stored in it. This doesn’t affect a
> > discussion of whether to publish binaries to S3 or to pypi – once that
> > concludes (if ever) we can switch destination of CodeBuild projects so that
> > they would upload MXNet nightly binaries to pypi instead of S3.
> >
> > This is an effort to get alignment internally, if possible, before
> > bringing this as a proposal for community discussion.
> >
> > --
> > Thanks,
> > Denis
> >
>

Re: MXNet CD pipelines cost savings

2020-02-10 Thread Marco de Abreu 
Thanks for bringing this to our attention.

I'm quite devastated that our two vetoes have been ignored and the
CodeBuild pipeline is actually supplying our user-facing binaries.
Suggesting to turn of the Jenkins based CD now adds insult to injury.

I'd like to hear a plan how to make the project compliant again. I already
announced that I will remove any mentions of that publishing method (speak,
all links on our website pointing to the bucket) if the sourcing system is
not our Jenkins CD. So far I believed that this was actually done, but
seems like we got played here.

For the sake of our users, I'm giving one week (until 2/17) to come up with
a proposal and until the end of the month 2/29 to have CodeBuild turned off
and Jenkins CD fixed. Considering we have been tricked last time, I want to
have confirmation that CodeBuild has been turned off and a description how
we can verify that all artifacts are now coming from Jenkins CD.

Best regards
Marco

Zha, Sheng  schrieb am Mo., 10. Feb. 2020,
19:40:

> +dev@
>
> -sz
>
> On Feb 10, 2020, at 1:35 PM, Zha, Sheng  wrote:
>
>  As already stated in the public threads, I’ve vetoed the CodeBuild
> solution from becoming the long term solution as it’s not publicly
> manageable.
>
> As communicated before, the team should have put efforts in maintaining
> and fixing the Jenkins CD pipeline but has neglected to do so. Promoting
> the CodeBuild solution this way is a step in the wrong direction that has
> to be stopped.
>
> -sz
>
> On Feb 10, 2020, at 1:13 PM, Davydenko, Denis  wrote:
>
> 
> Hello guys,
>
> I would like to start this discussion so that we can align on handling CD
> pipelines we currently have. There are two of them: one in Jenkins<
> http://jenkins.mxnet-ci.amazon-ml.com/job/restricted-mxnet-cd/> and one
> in CodeBuild. The one in
> Jenkins is currently functioning but its runs are always failing. The one
> in CodeBuild is currently functioning and publishing artifacts to S3 bucket<
> https://tiny.amazon.com/39negmk0/IsenLink>.
>
> MXNet Engineering team proposal is to shut down Jenkins based CD
> completely as it is currently just a waste of resources and use CodeBuild
> based setup to continue publishing nightly builds to S3 bucket, which
> provides public access to all binaries stored in it. This doesn’t affect a
> discussion of whether to publish binaries to S3 or to pypi – once that
> concludes (if ever) we can switch destination of CodeBuild projects so that
> they would upload MXNet nightly binaries to pypi instead of S3.
>
> This is an effort to get alignment internally, if possible, before
> bringing this as a proposal for community discussion.
>
> --
> Thanks,
> Denis
>

Re: MXNet CD pipelines cost savings

2020-02-10 Thread Zha, Sheng 
+dev@

-sz

On Feb 10, 2020, at 1:35 PM, Zha, Sheng  wrote:

 As already stated in the public threads, I’ve vetoed the CodeBuild solution 
from becoming the long term solution as it’s not publicly manageable.

As communicated before, the team should have put efforts in maintaining and 
fixing the Jenkins CD pipeline but has neglected to do so. Promoting the 
CodeBuild solution this way is a step in the wrong direction that has to be 
stopped.

-sz

On Feb 10, 2020, at 1:13 PM, Davydenko, Denis  wrote:


Hello guys,

I would like to start this discussion so that we can align on handling CD 
pipelines we currently have. There are two of them: one in 
Jenkins and one 
in CodeBuild. The one in Jenkins is 
currently functioning but its runs are always failing. The one in CodeBuild is 
currently functioning and publishing artifacts to S3 
bucket.

MXNet Engineering team proposal is to shut down Jenkins based CD completely as 
it is currently just a waste of resources and use CodeBuild based setup to 
continue publishing nightly builds to S3 bucket, which provides public access 
to all binaries stored in it. This doesn’t affect a discussion of whether to 
publish binaries to S3 or to pypi – once that concludes (if ever) we can switch 
destination of CodeBuild projects so that they would upload MXNet nightly 
binaries to pypi instead of S3.

This is an effort to get alignment internally, if possible, before bringing 
this as a proposal for community discussion.

--
Thanks,
Denis