Re: Commiter access to Jenkins Sevrer

[Marco de Abreu]

I agree with Pedros points. We've got a test-system running at
http://jenkins.mxnet-ci-dev.amazon-ml.com and all changes should be tested
there before they are deployed to prod. For now, we can move forward as-is
and I would kindly ask the committers to check back with me if any changes
are required. Please don't forget that the whole setup is configured as
infrastructure-as-code, this means that any changes you make without
checking back will be lost after a redeployment, so please sync up.

After successful review, I've enabled SSO via GitHub on our prod system at
http://jenkins.mxnet-ci.amazon-ml.com. Next time you press 'log in' on the
top right corner, you will be redirected to a GitHub login page which will
look like the following: https://imgur.com/Bg8LGWD
You can safely press 'Authorize marcoabreu'. The section about the
organizations can be safely ignored, do NOT press 'grant' besides an
organization as the CI system does not require access to the private data
of your organizations. This permission is only required in order to
identify which organizations you're in. Jenkins GitHub SSO requires this
information in order to allow organization-based access rights. Although we
are not using this feature, it can not be disabled unfortunately - the
system will crash if the read-org-permission is not granted.

For now, I've added the following people as administrators:
- Meghna Baijal
- Gautam Kumar
- Kellen Sunderland
- Pedro Larroy
- Marco de Abreu
- Chris Olivier

Also, I've tried to add all committers listed at
http://incubator.apache.org/projects/mxnet.html. Please drop me quick
message if it doesn't work for you, it might have been possible that I have
used the wrong account name. It seems like that the Apache alias sometimes
does not match the GitHub alias. Also, I haven't been able to link Jian
Zhangs alias 'jianzhangzju'.

Please try to log in and let me know if any issues arise.

Best regards,
Marco

On Mon, Jan 8, 2018 at 12:41 PM, Pedro Larroy 
wrote:

> Regarding the proposed permissions, I would like stricter permissions.
> I think a committer should be able to stop-start-cancel jobs. But I
> think only admins should be able to create new jobs, otherwise we run
> the risk of the CI becoming a mess of jobs that nobody owns and
> maintains. Please let's only have the jobs that are needed, supported,
> maintained and have an owner on CI and agreed on the mailing list.
>
> Pedro.
>
> On Fri, Jan 5, 2018 at 11:54 PM, Marco de Abreu
>  wrote:
> > I'm proposing following permissions: https://i.imgur.com/uiFBtuW.png.
> The
> > meaning of every permission is explained at https://wiki.jenkins.io/
> > display/JENKINS/Matrix-based+security.
> >
> > Any objections?
> >
> > On Fri, Jan 5, 2018 at 11:03 PM, Marco de Abreu <
> > marco.g.ab...@googlemail.com> wrote:
> >
> >> I'm currently working on a prototype of SSO based on GitHub and a few
> >> issues arose:
> >>
> >> We are not able to use the permission strategy which determines the
> access
> >> rights based on the read/write permission to a project as the
> >> Jenkins-plugin is not able to resolve the link between Jenkins-jobs and
> >> GitHub-repositories. Instead I would propose to use a role-based
> approach
> >> using https://wiki.jenkins.io/display/JENKINS/Role+Strategy+Plugin. In
> >> this case we would have three roles: Anonymous, Administrator and
> >> Committer. While everybody would authenticate using their regular GitHub
> >> account, the role assignment would have to happen manually. Considering
> >> that the amount of administrators and committers doesn't change that
> >> frequently, this shouldn't be too much of an issue - auto populating the
> >> status is not possible unfortunately.
> >>
> >> Reason for splitting Administrators and Committers into two separate
> roles
> >> has a security reason. At the moment, we're using Chris Oliviers GitHub
> >> credentials to populate the commit status. If all committers would gain
> >> full admin rights, they would have access to these credentials. Chris is
> >> not fine with this approach and would like to limit the amount of people
> >> with access to his credentials as much as possible.
> >>
> >> In order to address his concerns, I propose to add Chris to the
> committer
> >> as well as to the admin role, while all other committers will only
> receive
> >> the committer role without read access to the credentials. In a later
> >> email, I will make a proposal for the detailed committer role rights.
> You
> >> can check all available options at https://wiki.jenkins.io/
> >> display/JENKINS/Matrix-based+security.
> >>
> >> All people who have access to the underlying AWS account would be
> granted
> >> the Administrator role with full access. At the moment, this would be
> >> Meghna Baijal, Gautam Kumar and myself.
> >>
> >> An alternative solution would be to create a bot account specifically
> for
> >> MXNet CI and use its 

Re: Commiter access to Jenkins Sevrer

[Pedro Larroy]

Regarding the proposed permissions, I would like stricter permissions.
I think a committer should be able to stop-start-cancel jobs. But I
think only admins should be able to create new jobs, otherwise we run
the risk of the CI becoming a mess of jobs that nobody owns and
maintains. Please let's only have the jobs that are needed, supported,
maintained and have an owner on CI and agreed on the mailing list.

Pedro.

On Fri, Jan 5, 2018 at 11:54 PM, Marco de Abreu
 wrote:
> I'm proposing following permissions: https://i.imgur.com/uiFBtuW.png. The
> meaning of every permission is explained at https://wiki.jenkins.io/
> display/JENKINS/Matrix-based+security.
>
> Any objections?
>
> On Fri, Jan 5, 2018 at 11:03 PM, Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
>> I'm currently working on a prototype of SSO based on GitHub and a few
>> issues arose:
>>
>> We are not able to use the permission strategy which determines the access
>> rights based on the read/write permission to a project as the
>> Jenkins-plugin is not able to resolve the link between Jenkins-jobs and
>> GitHub-repositories. Instead I would propose to use a role-based approach
>> using https://wiki.jenkins.io/display/JENKINS/Role+Strategy+Plugin. In
>> this case we would have three roles: Anonymous, Administrator and
>> Committer. While everybody would authenticate using their regular GitHub
>> account, the role assignment would have to happen manually. Considering
>> that the amount of administrators and committers doesn't change that
>> frequently, this shouldn't be too much of an issue - auto populating the
>> status is not possible unfortunately.
>>
>> Reason for splitting Administrators and Committers into two separate roles
>> has a security reason. At the moment, we're using Chris Oliviers GitHub
>> credentials to populate the commit status. If all committers would gain
>> full admin rights, they would have access to these credentials. Chris is
>> not fine with this approach and would like to limit the amount of people
>> with access to his credentials as much as possible.
>>
>> In order to address his concerns, I propose to add Chris to the committer
>> as well as to the admin role, while all other committers will only receive
>> the committer role without read access to the credentials. In a later
>> email, I will make a proposal for the detailed committer role rights. You
>> can check all available options at https://wiki.jenkins.io/
>> display/JENKINS/Matrix-based+security.
>>
>> All people who have access to the underlying AWS account would be granted
>> the Administrator role with full access. At the moment, this would be
>> Meghna Baijal, Gautam Kumar and myself.
>>
>> An alternative solution would be to create a bot account specifically for
>> MXNet CI and use its credentials instead of Chris'. This account requires
>> write permission to the repository, but would give us the advantage that
>> these credentials would be shared within the committers and thus making the
>> restrictions regarding credentials obsolete (and Chris would be happy not
>> the see his face within every single PR :P ). I've asked around and
>> received the feedback from multiple people that Apache Infra does not want
>> to grant bot accounts write permission to a repository, but I would like to
>> confirm back considering that AppVeyor, for example, has a bot account with
>> write permission. I would like to check back with a mentor and create an
>> Apache Infra ticket to request details and permission.
>>
>> I would propose to take both approaches at the same time, meaning we can
>> start with Chris in the committer AND admin role while trying to get
>> permission for a bot account in the meantime.
>>
>> wdyt?
>>
>> On Fri, Jan 5, 2018 at 8:21 PM, Chris Olivier 
>> wrote:
>>
>>> I am fine without a vote unless a vote is required?  Any objections,
>>> anyone?  You're sort of adding functionality here, not changing or
>>> restricting...  We can always change to Apache later.
>>>
>>> On Fri, Jan 5, 2018 at 11:18 AM, Marco de Abreu <
>>> marco.g.ab...@googlemail.com> wrote:
>>>
>>> > I'd be in favour of GitHub. Shall we open a vote or would you like me to
>>> > create a POC with GitHub first and afterwards we can check if that's
>>> > enough?
>>> >
>>> > -Marco
>>> >
>>> > On Fri, Jan 5, 2018 at 8:13 PM, Chris Olivier 
>>> > wrote:
>>> >
>>> > > Apparently Apache supports OATH, so I am open to either.
>>> > > Good idea for the docker thing.
>>> > >
>>> > > On Fri, Jan 5, 2018 at 11:02 AM, Marco de Abreu <
>>> > > marco.g.ab...@googlemail.com> wrote:
>>> > >
>>> > > > GitHub SSO allows the neat feature that login and permission can be
>>> > > > selected depending on the access rights a user has to a project.
>>> > Somebody
>>> > > > with write access (committers) would be get different permissions
>>> than
>>> > > > somebody with only read access.
>>> > > >
>>> > > > We could 

Re: Commiter access to Jenkins Sevrer

[Marco de Abreu]

Sorry, I meant Naveen Swamy and not Suneel.

-Marco

On Sat, Jan 6, 2018 at 3:29 PM, Marco de Abreu  wrote:

> @Kellen: From my experience it was a bit unclear which exact steps have to
> be executed in order to create a valid environment to run ci_build.sh with
> tests. For example, the script expects the source-code in a specific
> directory which is then getting softlinked into the docker container. The
> build artefacts are getting copied into another specific softlinked
> directory as a result of the build process. In order to get to the test
> stage, these specific directories have to be in place. In general, I've got
> the feeling that too many undocumented requirements exists before the
> ci_build.sh can be executed properly - which makes sense at some point as
> it is supposed to only be used on Jenkins-slave.
> I'd like to see the scripted revamped in such a way that it can be run out
> of the box on a developer computer as well as on CI, telling the user if
> anything is missing or expected.
>
> @Pedro: Thank you! I've already had the possibility to let Eric, Sheng and
> Suneel test the authentication mechanism and so far everything worked
> flawlessly. At the moment, the roles have to be assigned manually, but I'd
> like to invite everybody to try it out themselves on our test-system,
> available at http://jenkins.mxnet-ci-dev.amazon-ml.com/. Feel free to
> break it and let me know if anything is missing. If this system passed
> review and test, I'd like to migrate it to prod.
>
> -Marco
>
> On Sat, Jan 6, 2018 at 3:12 PM, Pedro Larroy  > wrote:
>
>> Agree that comitters should have access to Jenkins.
>>
>> I would like to as ask for some patience due to the ongoing progress
>> on the CI work and thank Amazon for providing the resources for
>> running the new CI and the great job done by Marco and the infra team.
>>
>> Are there some volunteers in helping with the authentication mechanism
>> for committers?
>>
>> Pedro.
>>
>> On Sat, Jan 6, 2018 at 1:15 PM, Marco de Abreu
>>  wrote:
>> > While compile errors may be reproduced that way, it's hard to run any
>> tests
>> > due to the required softlink to the compiled binaries. It is possible,
>> but
>> > very inconvenient to use and thus it renders reproducing test results
>> very
>> > hard.
>> >
>> > I have been thinking about giving the possibility to download the
>> generated
>> > artefacts during build stage - for debugging reasons only! This way,
>> they
>> > can be used in conjunction with unit tests to reproduce a test failure,
>> but
>> > this still needs some discussions and a security review.
>> >
>> > -Marco
>> >
>> > Am 06.01.2018 12:59 nachm. schrieb "kellen sunderland" <
>> > kellen.sunderl...@gmail.com>:
>> >
>> >> Regarding the comments around reproducibility, what parts of the CI are
>> >> people having trouble reproducing now?  I'm in favour of making our
>> AMIs
>> >> public for transparency reasons (and so that people can provide
>> suggestions
>> >> on how to improve them), but I'm not sure it would help in terms of
>> >> reproducibility for any system other than Windows.  When I have an
>> error in
>> >> CI I generally just do a `make clean` in my mxnet root source dire,
>> then
>> >> copy the failing command from CI, i.e. `tests/ci_build/ci_build.sh cpu
>> >> --dockerbinary docker make DEV=1 USE_PROFILER=1 USE_CPP_PACKAGE=1
>> >> USE_BLAS=openblas -j$(nproc)`.  Are there CI tasks (other than Windows)
>> >> that don't work for people?  If so maybe we can help fix those?
>> >>
>> >>
>> >> On Sat, Jan 6, 2018 at 11:50 AM, kellen sunderland <
>> >> kellen.sunderl...@gmail.com> wrote:
>> >>
>> >> > +1, thanks for the work Marco.
>> >> >
>> >> > On Sat, Jan 6, 2018 at 12:24 AM, Naveen Swamy 
>> >> wrote:
>> >> >
>> >> >> this sounds fine to me as long as there is at least one MXNet
>> committer
>> >> >> who
>> >> >> is also an admin.
>> >> >>
>> >> >> thanks Marco for making this happen :)
>> >> >>
>> >> >>  - Naveen
>> >> >>
>> >> >> On Fri, Jan 5, 2018 at 2:54 PM, Marco de Abreu <
>> >> >> marco.g.ab...@googlemail.com
>> >> >> > wrote:
>> >> >>
>> >> >> > I'm proposing following permissions:
>> https://i.imgur.com/uiFBtuW.png.
>> >> >> The
>> >> >> > meaning of every permission is explained at
>> https://wiki.jenkins.io/
>> >> >> > display/JENKINS/Matrix-based+security.
>> >> >> >
>> >> >> > Any objections?
>> >> >> >
>> >> >> > On Fri, Jan 5, 2018 at 11:03 PM, Marco de Abreu <
>> >> >> > marco.g.ab...@googlemail.com> wrote:
>> >> >> >
>> >> >> > > I'm currently working on a prototype of SSO based on GitHub and
>> a
>> >> few
>> >> >> > > issues arose:
>> >> >> > >
>> >> >> > > We are not able to use the permission strategy which determines
>> the
>> >> >> > access
>> >> >> > > rights based on the read/write permission to a project as the
>> >> >> > > Jenkins-plugin is not able to resolve the link between
>> 

Re: Commiter access to Jenkins Sevrer

[Marco de Abreu]

@Kellen: From my experience it was a bit unclear which exact steps have to
be executed in order to create a valid environment to run ci_build.sh with
tests. For example, the script expects the source-code in a specific
directory which is then getting softlinked into the docker container. The
build artefacts are getting copied into another specific softlinked
directory as a result of the build process. In order to get to the test
stage, these specific directories have to be in place. In general, I've got
the feeling that too many undocumented requirements exists before the
ci_build.sh can be executed properly - which makes sense at some point as
it is supposed to only be used on Jenkins-slave.
I'd like to see the scripted revamped in such a way that it can be run out
of the box on a developer computer as well as on CI, telling the user if
anything is missing or expected.

@Pedro: Thank you! I've already had the possibility to let Eric, Sheng and
Suneel test the authentication mechanism and so far everything worked
flawlessly. At the moment, the roles have to be assigned manually, but I'd
like to invite everybody to try it out themselves on our test-system,
available at http://jenkins.mxnet-ci-dev.amazon-ml.com/. Feel free to break
it and let me know if anything is missing. If this system passed review and
test, I'd like to migrate it to prod.

-Marco

On Sat, Jan 6, 2018 at 3:12 PM, Pedro Larroy 
wrote:

> Agree that comitters should have access to Jenkins.
>
> I would like to as ask for some patience due to the ongoing progress
> on the CI work and thank Amazon for providing the resources for
> running the new CI and the great job done by Marco and the infra team.
>
> Are there some volunteers in helping with the authentication mechanism
> for committers?
>
> Pedro.
>
> On Sat, Jan 6, 2018 at 1:15 PM, Marco de Abreu
>  wrote:
> > While compile errors may be reproduced that way, it's hard to run any
> tests
> > due to the required softlink to the compiled binaries. It is possible,
> but
> > very inconvenient to use and thus it renders reproducing test results
> very
> > hard.
> >
> > I have been thinking about giving the possibility to download the
> generated
> > artefacts during build stage - for debugging reasons only! This way, they
> > can be used in conjunction with unit tests to reproduce a test failure,
> but
> > this still needs some discussions and a security review.
> >
> > -Marco
> >
> > Am 06.01.2018 12:59 nachm. schrieb "kellen sunderland" <
> > kellen.sunderl...@gmail.com>:
> >
> >> Regarding the comments around reproducibility, what parts of the CI are
> >> people having trouble reproducing now?  I'm in favour of making our AMIs
> >> public for transparency reasons (and so that people can provide
> suggestions
> >> on how to improve them), but I'm not sure it would help in terms of
> >> reproducibility for any system other than Windows.  When I have an
> error in
> >> CI I generally just do a `make clean` in my mxnet root source dire, then
> >> copy the failing command from CI, i.e. `tests/ci_build/ci_build.sh cpu
> >> --dockerbinary docker make DEV=1 USE_PROFILER=1 USE_CPP_PACKAGE=1
> >> USE_BLAS=openblas -j$(nproc)`.  Are there CI tasks (other than Windows)
> >> that don't work for people?  If so maybe we can help fix those?
> >>
> >>
> >> On Sat, Jan 6, 2018 at 11:50 AM, kellen sunderland <
> >> kellen.sunderl...@gmail.com> wrote:
> >>
> >> > +1, thanks for the work Marco.
> >> >
> >> > On Sat, Jan 6, 2018 at 12:24 AM, Naveen Swamy 
> >> wrote:
> >> >
> >> >> this sounds fine to me as long as there is at least one MXNet
> committer
> >> >> who
> >> >> is also an admin.
> >> >>
> >> >> thanks Marco for making this happen :)
> >> >>
> >> >>  - Naveen
> >> >>
> >> >> On Fri, Jan 5, 2018 at 2:54 PM, Marco de Abreu <
> >> >> marco.g.ab...@googlemail.com
> >> >> > wrote:
> >> >>
> >> >> > I'm proposing following permissions: https://i.imgur.com/uiFBtuW.
> png.
> >> >> The
> >> >> > meaning of every permission is explained at
> https://wiki.jenkins.io/
> >> >> > display/JENKINS/Matrix-based+security.
> >> >> >
> >> >> > Any objections?
> >> >> >
> >> >> > On Fri, Jan 5, 2018 at 11:03 PM, Marco de Abreu <
> >> >> > marco.g.ab...@googlemail.com> wrote:
> >> >> >
> >> >> > > I'm currently working on a prototype of SSO based on GitHub and a
> >> few
> >> >> > > issues arose:
> >> >> > >
> >> >> > > We are not able to use the permission strategy which determines
> the
> >> >> > access
> >> >> > > rights based on the read/write permission to a project as the
> >> >> > > Jenkins-plugin is not able to resolve the link between
> Jenkins-jobs
> >> >> and
> >> >> > > GitHub-repositories. Instead I would propose to use a role-based
> >> >> approach
> >> >> > > using https://wiki.jenkins.io/display/JENKINS/Role+Strategy+
> Plugin.
> >> >> In
> >> >> > > this case we would have three roles: Anonymous, Administrator and
> >> >> > 

Re: Commiter access to Jenkins Sevrer

[Pedro Larroy]

Agree that comitters should have access to Jenkins.

I would like to as ask for some patience due to the ongoing progress
on the CI work and thank Amazon for providing the resources for
running the new CI and the great job done by Marco and the infra team.

Are there some volunteers in helping with the authentication mechanism
for committers?

Pedro.

On Sat, Jan 6, 2018 at 1:15 PM, Marco de Abreu
 wrote:
> While compile errors may be reproduced that way, it's hard to run any tests
> due to the required softlink to the compiled binaries. It is possible, but
> very inconvenient to use and thus it renders reproducing test results very
> hard.
>
> I have been thinking about giving the possibility to download the generated
> artefacts during build stage - for debugging reasons only! This way, they
> can be used in conjunction with unit tests to reproduce a test failure, but
> this still needs some discussions and a security review.
>
> -Marco
>
> Am 06.01.2018 12:59 nachm. schrieb "kellen sunderland" <
> kellen.sunderl...@gmail.com>:
>
>> Regarding the comments around reproducibility, what parts of the CI are
>> people having trouble reproducing now?  I'm in favour of making our AMIs
>> public for transparency reasons (and so that people can provide suggestions
>> on how to improve them), but I'm not sure it would help in terms of
>> reproducibility for any system other than Windows.  When I have an error in
>> CI I generally just do a `make clean` in my mxnet root source dire, then
>> copy the failing command from CI, i.e. `tests/ci_build/ci_build.sh cpu
>> --dockerbinary docker make DEV=1 USE_PROFILER=1 USE_CPP_PACKAGE=1
>> USE_BLAS=openblas -j$(nproc)`.  Are there CI tasks (other than Windows)
>> that don't work for people?  If so maybe we can help fix those?
>>
>>
>> On Sat, Jan 6, 2018 at 11:50 AM, kellen sunderland <
>> kellen.sunderl...@gmail.com> wrote:
>>
>> > +1, thanks for the work Marco.
>> >
>> > On Sat, Jan 6, 2018 at 12:24 AM, Naveen Swamy 
>> wrote:
>> >
>> >> this sounds fine to me as long as there is at least one MXNet committer
>> >> who
>> >> is also an admin.
>> >>
>> >> thanks Marco for making this happen :)
>> >>
>> >>  - Naveen
>> >>
>> >> On Fri, Jan 5, 2018 at 2:54 PM, Marco de Abreu <
>> >> marco.g.ab...@googlemail.com
>> >> > wrote:
>> >>
>> >> > I'm proposing following permissions: https://i.imgur.com/uiFBtuW.png.
>> >> The
>> >> > meaning of every permission is explained at https://wiki.jenkins.io/
>> >> > display/JENKINS/Matrix-based+security.
>> >> >
>> >> > Any objections?
>> >> >
>> >> > On Fri, Jan 5, 2018 at 11:03 PM, Marco de Abreu <
>> >> > marco.g.ab...@googlemail.com> wrote:
>> >> >
>> >> > > I'm currently working on a prototype of SSO based on GitHub and a
>> few
>> >> > > issues arose:
>> >> > >
>> >> > > We are not able to use the permission strategy which determines the
>> >> > access
>> >> > > rights based on the read/write permission to a project as the
>> >> > > Jenkins-plugin is not able to resolve the link between Jenkins-jobs
>> >> and
>> >> > > GitHub-repositories. Instead I would propose to use a role-based
>> >> approach
>> >> > > using https://wiki.jenkins.io/display/JENKINS/Role+Strategy+Plugin.
>> >> In
>> >> > > this case we would have three roles: Anonymous, Administrator and
>> >> > > Committer. While everybody would authenticate using their regular
>> >> GitHub
>> >> > > account, the role assignment would have to happen manually.
>> >> Considering
>> >> > > that the amount of administrators and committers doesn't change that
>> >> > > frequently, this shouldn't be too much of an issue - auto populating
>> >> the
>> >> > > status is not possible unfortunately.
>> >> > >
>> >> > > Reason for splitting Administrators and Committers into two separate
>> >> > roles
>> >> > > has a security reason. At the moment, we're using Chris Oliviers
>> >> GitHub
>> >> > > credentials to populate the commit status. If all committers would
>> >> gain
>> >> > > full admin rights, they would have access to these credentials.
>> Chris
>> >> is
>> >> > > not fine with this approach and would like to limit the amount of
>> >> people
>> >> > > with access to his credentials as much as possible.
>> >> > >
>> >> > > In order to address his concerns, I propose to add Chris to the
>> >> committer
>> >> > > as well as to the admin role, while all other committers will only
>> >> > receive
>> >> > > the committer role without read access to the credentials. In a
>> later
>> >> > > email, I will make a proposal for the detailed committer role
>> rights.
>> >> You
>> >> > > can check all available options at https://wiki.jenkins.io/
>> >> > > display/JENKINS/Matrix-based+security.
>> >> > >
>> >> > > All people who have access to the underlying AWS account would be
>> >> granted
>> >> > > the Administrator role with full access. At the moment, this would
>> be
>> >> > > Meghna Baijal, Gautam Kumar and myself.
>> >> > >
>> >> > > An 

Re: Commiter access to Jenkins Sevrer

[kellen sunderland]

Can you give a little more info around which binaries would you be missing
locally after building?

On Sat, Jan 6, 2018 at 1:15 PM, Marco de Abreu  wrote:

> While compile errors may be reproduced that way, it's hard to run any tests
> due to the required softlink to the compiled binaries. It is possible, but
> very inconvenient to use and thus it renders reproducing test results very
> hard.
>
> I have been thinking about giving the possibility to download the generated
> artefacts during build stage - for debugging reasons only! This way, they
> can be used in conjunction with unit tests to reproduce a test failure, but
> this still needs some discussions and a security review.
>
> -Marco
>
> Am 06.01.2018 12:59 nachm. schrieb "kellen sunderland" <
> kellen.sunderl...@gmail.com>:
>
> > Regarding the comments around reproducibility, what parts of the CI are
> > people having trouble reproducing now?  I'm in favour of making our AMIs
> > public for transparency reasons (and so that people can provide
> suggestions
> > on how to improve them), but I'm not sure it would help in terms of
> > reproducibility for any system other than Windows.  When I have an error
> in
> > CI I generally just do a `make clean` in my mxnet root source dire, then
> > copy the failing command from CI, i.e. `tests/ci_build/ci_build.sh cpu
> > --dockerbinary docker make DEV=1 USE_PROFILER=1 USE_CPP_PACKAGE=1
> > USE_BLAS=openblas -j$(nproc)`.  Are there CI tasks (other than Windows)
> > that don't work for people?  If so maybe we can help fix those?
> >
> >
> > On Sat, Jan 6, 2018 at 11:50 AM, kellen sunderland <
> > kellen.sunderl...@gmail.com> wrote:
> >
> > > +1, thanks for the work Marco.
> > >
> > > On Sat, Jan 6, 2018 at 12:24 AM, Naveen Swamy 
> > wrote:
> > >
> > >> this sounds fine to me as long as there is at least one MXNet
> committer
> > >> who
> > >> is also an admin.
> > >>
> > >> thanks Marco for making this happen :)
> > >>
> > >>  - Naveen
> > >>
> > >> On Fri, Jan 5, 2018 at 2:54 PM, Marco de Abreu <
> > >> marco.g.ab...@googlemail.com
> > >> > wrote:
> > >>
> > >> > I'm proposing following permissions: https://i.imgur.com/uiFBtuW.
> png.
> > >> The
> > >> > meaning of every permission is explained at
> https://wiki.jenkins.io/
> > >> > display/JENKINS/Matrix-based+security.
> > >> >
> > >> > Any objections?
> > >> >
> > >> > On Fri, Jan 5, 2018 at 11:03 PM, Marco de Abreu <
> > >> > marco.g.ab...@googlemail.com> wrote:
> > >> >
> > >> > > I'm currently working on a prototype of SSO based on GitHub and a
> > few
> > >> > > issues arose:
> > >> > >
> > >> > > We are not able to use the permission strategy which determines
> the
> > >> > access
> > >> > > rights based on the read/write permission to a project as the
> > >> > > Jenkins-plugin is not able to resolve the link between
> Jenkins-jobs
> > >> and
> > >> > > GitHub-repositories. Instead I would propose to use a role-based
> > >> approach
> > >> > > using https://wiki.jenkins.io/display/JENKINS/Role+Strategy+
> Plugin.
> > >> In
> > >> > > this case we would have three roles: Anonymous, Administrator and
> > >> > > Committer. While everybody would authenticate using their regular
> > >> GitHub
> > >> > > account, the role assignment would have to happen manually.
> > >> Considering
> > >> > > that the amount of administrators and committers doesn't change
> that
> > >> > > frequently, this shouldn't be too much of an issue - auto
> populating
> > >> the
> > >> > > status is not possible unfortunately.
> > >> > >
> > >> > > Reason for splitting Administrators and Committers into two
> separate
> > >> > roles
> > >> > > has a security reason. At the moment, we're using Chris Oliviers
> > >> GitHub
> > >> > > credentials to populate the commit status. If all committers would
> > >> gain
> > >> > > full admin rights, they would have access to these credentials.
> > Chris
> > >> is
> > >> > > not fine with this approach and would like to limit the amount of
> > >> people
> > >> > > with access to his credentials as much as possible.
> > >> > >
> > >> > > In order to address his concerns, I propose to add Chris to the
> > >> committer
> > >> > > as well as to the admin role, while all other committers will only
> > >> > receive
> > >> > > the committer role without read access to the credentials. In a
> > later
> > >> > > email, I will make a proposal for the detailed committer role
> > rights.
> > >> You
> > >> > > can check all available options at https://wiki.jenkins.io/
> > >> > > display/JENKINS/Matrix-based+security.
> > >> > >
> > >> > > All people who have access to the underlying AWS account would be
> > >> granted
> > >> > > the Administrator role with full access. At the moment, this would
> > be
> > >> > > Meghna Baijal, Gautam Kumar and myself.
> > >> > >
> > >> > > An alternative solution would be to create a bot account
> > specifically
> > >> for
> > >> > > MXNet CI and use its credentials 

Re: Commiter access to Jenkins Sevrer

[kellen sunderland]

Regarding the comments around reproducibility, what parts of the CI are
people having trouble reproducing now?  I'm in favour of making our AMIs
public for transparency reasons (and so that people can provide suggestions
on how to improve them), but I'm not sure it would help in terms of
reproducibility for any system other than Windows.  When I have an error in
CI I generally just do a `make clean` in my mxnet root source dire, then
copy the failing command from CI, i.e. `tests/ci_build/ci_build.sh cpu
--dockerbinary docker make DEV=1 USE_PROFILER=1 USE_CPP_PACKAGE=1
USE_BLAS=openblas -j$(nproc)`.  Are there CI tasks (other than Windows)
that don't work for people?  If so maybe we can help fix those?


On Sat, Jan 6, 2018 at 11:50 AM, kellen sunderland <
kellen.sunderl...@gmail.com> wrote:

> +1, thanks for the work Marco.
>
> On Sat, Jan 6, 2018 at 12:24 AM, Naveen Swamy  wrote:
>
>> this sounds fine to me as long as there is at least one MXNet committer
>> who
>> is also an admin.
>>
>> thanks Marco for making this happen :)
>>
>>  - Naveen
>>
>> On Fri, Jan 5, 2018 at 2:54 PM, Marco de Abreu <
>> marco.g.ab...@googlemail.com
>> > wrote:
>>
>> > I'm proposing following permissions: https://i.imgur.com/uiFBtuW.png.
>> The
>> > meaning of every permission is explained at https://wiki.jenkins.io/
>> > display/JENKINS/Matrix-based+security.
>> >
>> > Any objections?
>> >
>> > On Fri, Jan 5, 2018 at 11:03 PM, Marco de Abreu <
>> > marco.g.ab...@googlemail.com> wrote:
>> >
>> > > I'm currently working on a prototype of SSO based on GitHub and a few
>> > > issues arose:
>> > >
>> > > We are not able to use the permission strategy which determines the
>> > access
>> > > rights based on the read/write permission to a project as the
>> > > Jenkins-plugin is not able to resolve the link between Jenkins-jobs
>> and
>> > > GitHub-repositories. Instead I would propose to use a role-based
>> approach
>> > > using https://wiki.jenkins.io/display/JENKINS/Role+Strategy+Plugin.
>> In
>> > > this case we would have three roles: Anonymous, Administrator and
>> > > Committer. While everybody would authenticate using their regular
>> GitHub
>> > > account, the role assignment would have to happen manually.
>> Considering
>> > > that the amount of administrators and committers doesn't change that
>> > > frequently, this shouldn't be too much of an issue - auto populating
>> the
>> > > status is not possible unfortunately.
>> > >
>> > > Reason for splitting Administrators and Committers into two separate
>> > roles
>> > > has a security reason. At the moment, we're using Chris Oliviers
>> GitHub
>> > > credentials to populate the commit status. If all committers would
>> gain
>> > > full admin rights, they would have access to these credentials. Chris
>> is
>> > > not fine with this approach and would like to limit the amount of
>> people
>> > > with access to his credentials as much as possible.
>> > >
>> > > In order to address his concerns, I propose to add Chris to the
>> committer
>> > > as well as to the admin role, while all other committers will only
>> > receive
>> > > the committer role without read access to the credentials. In a later
>> > > email, I will make a proposal for the detailed committer role rights.
>> You
>> > > can check all available options at https://wiki.jenkins.io/
>> > > display/JENKINS/Matrix-based+security.
>> > >
>> > > All people who have access to the underlying AWS account would be
>> granted
>> > > the Administrator role with full access. At the moment, this would be
>> > > Meghna Baijal, Gautam Kumar and myself.
>> > >
>> > > An alternative solution would be to create a bot account specifically
>> for
>> > > MXNet CI and use its credentials instead of Chris'. This account
>> requires
>> > > write permission to the repository, but would give us the advantage
>> that
>> > > these credentials would be shared within the committers and thus
>> making
>> > the
>> > > restrictions regarding credentials obsolete (and Chris would be happy
>> not
>> > > the see his face within every single PR :P ). I've asked around and
>> > > received the feedback from multiple people that Apache Infra does not
>> > want
>> > > to grant bot accounts write permission to a repository, but I would
>> like
>> > to
>> > > confirm back considering that AppVeyor, for example, has a bot account
>> > with
>> > > write permission. I would like to check back with a mentor and create
>> an
>> > > Apache Infra ticket to request details and permission.
>> > >
>> > > I would propose to take both approaches at the same time, meaning we
>> can
>> > > start with Chris in the committer AND admin role while trying to get
>> > > permission for a bot account in the meantime.
>> > >
>> > > wdyt?
>> > >
>> > > On Fri, Jan 5, 2018 at 8:21 PM, Chris Olivier 
>> > > wrote:
>> > >
>> > >> I am fine without a vote unless a vote is required?  Any objections,
>> > >> anyone?  You're sort of adding 

Re: Commiter access to Jenkins Sevrer

[Naveen Swamy]

this sounds fine to me as long as there is at least one MXNet committer who
is also an admin.

thanks Marco for making this happen :)

 - Naveen

On Fri, Jan 5, 2018 at 2:54 PM, Marco de Abreu  wrote:

> I'm proposing following permissions: https://i.imgur.com/uiFBtuW.png. The
> meaning of every permission is explained at https://wiki.jenkins.io/
> display/JENKINS/Matrix-based+security.
>
> Any objections?
>
> On Fri, Jan 5, 2018 at 11:03 PM, Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
> > I'm currently working on a prototype of SSO based on GitHub and a few
> > issues arose:
> >
> > We are not able to use the permission strategy which determines the
> access
> > rights based on the read/write permission to a project as the
> > Jenkins-plugin is not able to resolve the link between Jenkins-jobs and
> > GitHub-repositories. Instead I would propose to use a role-based approach
> > using https://wiki.jenkins.io/display/JENKINS/Role+Strategy+Plugin. In
> > this case we would have three roles: Anonymous, Administrator and
> > Committer. While everybody would authenticate using their regular GitHub
> > account, the role assignment would have to happen manually. Considering
> > that the amount of administrators and committers doesn't change that
> > frequently, this shouldn't be too much of an issue - auto populating the
> > status is not possible unfortunately.
> >
> > Reason for splitting Administrators and Committers into two separate
> roles
> > has a security reason. At the moment, we're using Chris Oliviers GitHub
> > credentials to populate the commit status. If all committers would gain
> > full admin rights, they would have access to these credentials. Chris is
> > not fine with this approach and would like to limit the amount of people
> > with access to his credentials as much as possible.
> >
> > In order to address his concerns, I propose to add Chris to the committer
> > as well as to the admin role, while all other committers will only
> receive
> > the committer role without read access to the credentials. In a later
> > email, I will make a proposal for the detailed committer role rights. You
> > can check all available options at https://wiki.jenkins.io/
> > display/JENKINS/Matrix-based+security.
> >
> > All people who have access to the underlying AWS account would be granted
> > the Administrator role with full access. At the moment, this would be
> > Meghna Baijal, Gautam Kumar and myself.
> >
> > An alternative solution would be to create a bot account specifically for
> > MXNet CI and use its credentials instead of Chris'. This account requires
> > write permission to the repository, but would give us the advantage that
> > these credentials would be shared within the committers and thus making
> the
> > restrictions regarding credentials obsolete (and Chris would be happy not
> > the see his face within every single PR :P ). I've asked around and
> > received the feedback from multiple people that Apache Infra does not
> want
> > to grant bot accounts write permission to a repository, but I would like
> to
> > confirm back considering that AppVeyor, for example, has a bot account
> with
> > write permission. I would like to check back with a mentor and create an
> > Apache Infra ticket to request details and permission.
> >
> > I would propose to take both approaches at the same time, meaning we can
> > start with Chris in the committer AND admin role while trying to get
> > permission for a bot account in the meantime.
> >
> > wdyt?
> >
> > On Fri, Jan 5, 2018 at 8:21 PM, Chris Olivier 
> > wrote:
> >
> >> I am fine without a vote unless a vote is required?  Any objections,
> >> anyone?  You're sort of adding functionality here, not changing or
> >> restricting...  We can always change to Apache later.
> >>
> >> On Fri, Jan 5, 2018 at 11:18 AM, Marco de Abreu <
> >> marco.g.ab...@googlemail.com> wrote:
> >>
> >> > I'd be in favour of GitHub. Shall we open a vote or would you like me
> to
> >> > create a POC with GitHub first and afterwards we can check if that's
> >> > enough?
> >> >
> >> > -Marco
> >> >
> >> > On Fri, Jan 5, 2018 at 8:13 PM, Chris Olivier 
> >> > wrote:
> >> >
> >> > > Apparently Apache supports OATH, so I am open to either.
> >> > > Good idea for the docker thing.
> >> > >
> >> > > On Fri, Jan 5, 2018 at 11:02 AM, Marco de Abreu <
> >> > > marco.g.ab...@googlemail.com> wrote:
> >> > >
> >> > > > GitHub SSO allows the neat feature that login and permission can
> be
> >> > > > selected depending on the access rights a user has to a project.
> >> > Somebody
> >> > > > with write access (committers) would be get different permissions
> >> than
> >> > > > somebody with only read access.
> >> > > >
> >> > > > We could check back with Apache for SSO, but this would involve
> >> Apache
> >> > > > infra. We could put it up to a vote whether to use GitHub or
> Apache
> >> > SSO.
> >> > > >

Re: Commiter access to Jenkins Sevrer

[Marco de Abreu]

I'm proposing following permissions: https://i.imgur.com/uiFBtuW.png. The
meaning of every permission is explained at https://wiki.jenkins.io/
display/JENKINS/Matrix-based+security.

Any objections?

On Fri, Jan 5, 2018 at 11:03 PM, Marco de Abreu <
marco.g.ab...@googlemail.com> wrote:

> I'm currently working on a prototype of SSO based on GitHub and a few
> issues arose:
>
> We are not able to use the permission strategy which determines the access
> rights based on the read/write permission to a project as the
> Jenkins-plugin is not able to resolve the link between Jenkins-jobs and
> GitHub-repositories. Instead I would propose to use a role-based approach
> using https://wiki.jenkins.io/display/JENKINS/Role+Strategy+Plugin. In
> this case we would have three roles: Anonymous, Administrator and
> Committer. While everybody would authenticate using their regular GitHub
> account, the role assignment would have to happen manually. Considering
> that the amount of administrators and committers doesn't change that
> frequently, this shouldn't be too much of an issue - auto populating the
> status is not possible unfortunately.
>
> Reason for splitting Administrators and Committers into two separate roles
> has a security reason. At the moment, we're using Chris Oliviers GitHub
> credentials to populate the commit status. If all committers would gain
> full admin rights, they would have access to these credentials. Chris is
> not fine with this approach and would like to limit the amount of people
> with access to his credentials as much as possible.
>
> In order to address his concerns, I propose to add Chris to the committer
> as well as to the admin role, while all other committers will only receive
> the committer role without read access to the credentials. In a later
> email, I will make a proposal for the detailed committer role rights. You
> can check all available options at https://wiki.jenkins.io/
> display/JENKINS/Matrix-based+security.
>
> All people who have access to the underlying AWS account would be granted
> the Administrator role with full access. At the moment, this would be
> Meghna Baijal, Gautam Kumar and myself.
>
> An alternative solution would be to create a bot account specifically for
> MXNet CI and use its credentials instead of Chris'. This account requires
> write permission to the repository, but would give us the advantage that
> these credentials would be shared within the committers and thus making the
> restrictions regarding credentials obsolete (and Chris would be happy not
> the see his face within every single PR :P ). I've asked around and
> received the feedback from multiple people that Apache Infra does not want
> to grant bot accounts write permission to a repository, but I would like to
> confirm back considering that AppVeyor, for example, has a bot account with
> write permission. I would like to check back with a mentor and create an
> Apache Infra ticket to request details and permission.
>
> I would propose to take both approaches at the same time, meaning we can
> start with Chris in the committer AND admin role while trying to get
> permission for a bot account in the meantime.
>
> wdyt?
>
> On Fri, Jan 5, 2018 at 8:21 PM, Chris Olivier 
> wrote:
>
>> I am fine without a vote unless a vote is required?  Any objections,
>> anyone?  You're sort of adding functionality here, not changing or
>> restricting...  We can always change to Apache later.
>>
>> On Fri, Jan 5, 2018 at 11:18 AM, Marco de Abreu <
>> marco.g.ab...@googlemail.com> wrote:
>>
>> > I'd be in favour of GitHub. Shall we open a vote or would you like me to
>> > create a POC with GitHub first and afterwards we can check if that's
>> > enough?
>> >
>> > -Marco
>> >
>> > On Fri, Jan 5, 2018 at 8:13 PM, Chris Olivier 
>> > wrote:
>> >
>> > > Apparently Apache supports OATH, so I am open to either.
>> > > Good idea for the docker thing.
>> > >
>> > > On Fri, Jan 5, 2018 at 11:02 AM, Marco de Abreu <
>> > > marco.g.ab...@googlemail.com> wrote:
>> > >
>> > > > GitHub SSO allows the neat feature that login and permission can be
>> > > > selected depending on the access rights a user has to a project.
>> > Somebody
>> > > > with write access (committers) would be get different permissions
>> than
>> > > > somebody with only read access.
>> > > >
>> > > > We could check back with Apache for SSO, but this would involve
>> Apache
>> > > > infra. We could put it up to a vote whether to use GitHub or Apache
>> > SSO.
>> > > >
>> > > > In order to reproduce a build failure we have been thinking about
>> > > changing
>> > > > the ci_build.sh in such a way that it can be run manually without
>> > > Jenkins.
>> > > > The setup I took over binds the Jenkins work directory into the
>> docker
>> > > > containers and uses a few hacks which are hard to reproduce
>> locally. We
>> > > > plan to reengineer this script to make it easier to run manually.
>> > > > But making 

Re: Commiter access to Jenkins Sevrer

[Marco de Abreu]

I'm currently working on a prototype of SSO based on GitHub and a few
issues arose:

We are not able to use the permission strategy which determines the access
rights based on the read/write permission to a project as the
Jenkins-plugin is not able to resolve the link between Jenkins-jobs and
GitHub-repositories. Instead I would propose to use a role-based approach
using https://wiki.jenkins.io/display/JENKINS/Role+Strategy+Plugin. In this
case we would have three roles: Anonymous, Administrator and Committer.
While everybody would authenticate using their regular GitHub account, the
role assignment would have to happen manually. Considering that the amount
of administrators and committers doesn't change that frequently, this
shouldn't be too much of an issue - auto populating the status is not
possible unfortunately.

Reason for splitting Administrators and Committers into two separate roles
has a security reason. At the moment, we're using Chris Oliviers GitHub
credentials to populate the commit status. If all committers would gain
full admin rights, they would have access to these credentials. Chris is
not fine with this approach and would like to limit the amount of people
with access to his credentials as much as possible.

In order to address his concerns, I propose to add Chris to the committer
as well as to the admin role, while all other committers will only receive
the committer role without read access to the credentials. In a later
email, I will make a proposal for the detailed committer role rights. You
can check all available options at
https://wiki.jenkins.io/display/JENKINS/Matrix-based+security.

All people who have access to the underlying AWS account would be granted
the Administrator role with full access. At the moment, this would be
Meghna Baijal, Gautam Kumar and myself.

An alternative solution would be to create a bot account specifically for
MXNet CI and use its credentials instead of Chris'. This account requires
write permission to the repository, but would give us the advantage that
these credentials would be shared within the committers and thus making the
restrictions regarding credentials obsolete (and Chris would be happy not
the see his face within every single PR :P ). I've asked around and
received the feedback from multiple people that Apache Infra does not want
to grant bot accounts write permission to a repository, but I would like to
confirm back considering that AppVeyor, for example, has a bot account with
write permission. I would like to check back with a mentor and create an
Apache Infra ticket to request details and permission.

I would propose to take both approaches at the same time, meaning we can
start with Chris in the committer AND admin role while trying to get
permission for a bot account in the meantime.

wdyt?

On Fri, Jan 5, 2018 at 8:21 PM, Chris Olivier  wrote:

> I am fine without a vote unless a vote is required?  Any objections,
> anyone?  You're sort of adding functionality here, not changing or
> restricting...  We can always change to Apache later.
>
> On Fri, Jan 5, 2018 at 11:18 AM, Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
> > I'd be in favour of GitHub. Shall we open a vote or would you like me to
> > create a POC with GitHub first and afterwards we can check if that's
> > enough?
> >
> > -Marco
> >
> > On Fri, Jan 5, 2018 at 8:13 PM, Chris Olivier 
> > wrote:
> >
> > > Apparently Apache supports OATH, so I am open to either.
> > > Good idea for the docker thing.
> > >
> > > On Fri, Jan 5, 2018 at 11:02 AM, Marco de Abreu <
> > > marco.g.ab...@googlemail.com> wrote:
> > >
> > > > GitHub SSO allows the neat feature that login and permission can be
> > > > selected depending on the access rights a user has to a project.
> > Somebody
> > > > with write access (committers) would be get different permissions
> than
> > > > somebody with only read access.
> > > >
> > > > We could check back with Apache for SSO, but this would involve
> Apache
> > > > infra. We could put it up to a vote whether to use GitHub or Apache
> > SSO.
> > > >
> > > > In order to reproduce a build failure we have been thinking about
> > > changing
> > > > the ci_build.sh in such a way that it can be run manually without
> > > Jenkins.
> > > > The setup I took over binds the Jenkins work directory into the
> docker
> > > > containers and uses a few hacks which are hard to reproduce locally.
> We
> > > > plan to reengineer this script to make it easier to run manually.
> > > > But making the AMI public is a good idea! We plan to make the whole
> > > > infrastructure code (based on Terraform) completely public - at the
> > > moment
> > > > it's in a private repository as it contains credentials, but they
> will
> > be
> > > > moved to KMS soon. It would definitely be a good approach to just
> > supply
> > > > the AMI so everybody could recreate the environment in their own
> > account.
> > > >
> > > > -Marco
> > 

Re: Commiter access to Jenkins Sevrer

[Chris Olivier]

+1

On Fri, Jan 5, 2018 at 1:41 PM, Indhu  wrote:

> Objective is to make sure committers has access to CI. As far as that
> objective is met, it shouldn't matter whether the auth mechanism uses
> GitHub or Apache SSO.
>
> If GitHub SSO is easier to implement, let's go ahead and do that unless
> someone in the list objects.
>
>
> On Fri, Jan 5, 2018 at 11:21 AM Chris Olivier 
> wrote:
>
> > I am fine without a vote unless a vote is required?  Any objections,
> > anyone?  You're sort of adding functionality here, not changing or
> > restricting...  We can always change to Apache later.
> >
> > On Fri, Jan 5, 2018 at 11:18 AM, Marco de Abreu <
> > marco.g.ab...@googlemail.com> wrote:
> >
> > > I'd be in favour of GitHub. Shall we open a vote or would you like me
> to
> > > create a POC with GitHub first and afterwards we can check if that's
> > > enough?
> > >
> > > -Marco
> > >
> > > On Fri, Jan 5, 2018 at 8:13 PM, Chris Olivier 
> > > wrote:
> > >
> > > > Apparently Apache supports OATH, so I am open to either.
> > > > Good idea for the docker thing.
> > > >
> > > > On Fri, Jan 5, 2018 at 11:02 AM, Marco de Abreu <
> > > > marco.g.ab...@googlemail.com> wrote:
> > > >
> > > > > GitHub SSO allows the neat feature that login and permission can be
> > > > > selected depending on the access rights a user has to a project.
> > > Somebody
> > > > > with write access (committers) would be get different permissions
> > than
> > > > > somebody with only read access.
> > > > >
> > > > > We could check back with Apache for SSO, but this would involve
> > Apache
> > > > > infra. We could put it up to a vote whether to use GitHub or Apache
> > > SSO.
> > > > >
> > > > > In order to reproduce a build failure we have been thinking about
> > > > changing
> > > > > the ci_build.sh in such a way that it can be run manually without
> > > > Jenkins.
> > > > > The setup I took over binds the Jenkins work directory into the
> > docker
> > > > > containers and uses a few hacks which are hard to reproduce
> locally.
> > We
> > > > > plan to reengineer this script to make it easier to run manually.
> > > > > But making the AMI public is a good idea! We plan to make the whole
> > > > > infrastructure code (based on Terraform) completely public - at the
> > > > moment
> > > > > it's in a private repository as it contains credentials, but they
> > will
> > > be
> > > > > moved to KMS soon. It would definitely be a good approach to just
> > > supply
> > > > > the AMI so everybody could recreate the environment in their own
> > > account.
> > > > >
> > > > > -Marco
> > > > >
> > > > > Am 05.01.2018 7:51 nachm. schrieb "Chris Olivier" <
> > > cjolivie...@gmail.com
> > > > >:
> > > > >
> > > > > Well, login to the Jenkins server, I would imagine.
> > > > >
> > > > > github or Apache SSO (does Apache support OAUTH?) seems like a good
> > > idea
> > > > as
> > > > > long as there's a way to not let everyone with a github account log
> > in.
> > > > >
> > > > > Access to actual slave machines could be more restricted, I
> imagine.
> > > > >
> > > > > Eventually, a public current AMI for a build slave would be good in
> > > order
> > > > > to reproduce build or test problems that can't be reproduced
> locally.
> > > > >
> > > > > wdyt?
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Jan 5, 2018 at 10:41 AM, Marco de Abreu <
> > > > > marco.g.ab...@googlemail.com> wrote:
> > > > >
> > > > > > Would it be an acceptable solution if we add SSO or do you also
> > want
> > > > > access
> > > > > > to the actual AWS account and all machines?
> > > > > >
> > > > > > Yes, the build jobs are automatically getting created for new
> > > branches.
> > > > > >
> > > > > > -Marco
> > > > > >
> > > > > > Am 05.01.2018 7:35 nachm. schrieb "Marco de Abreu" <
> > > > > > marco.g.ab...@googlemail.com>:
> > > > > >
> > > > > > I totally agree, this is not the way it should work in an Apache
> > > > Project.
> > > > > > It's running on an isengard account, meaning it is only
> accessible
> > > for
> > > > > > Amazon employees. The problem is that a compromised account could
> > > cause
> > > > > > damage up to 170,000$ per day. There are alarms in place to
> notice
> > > > those
> > > > > > cases, but we still have to be very careful. These high limits
> have
> > > > been
> > > > > > chosen due to auto scaling being added within the next week's.
> > > > > >
> > > > > > I'd be happy to introduce a committer into the CI process and all
> > the
> > > > > > necessary steps as well as granting them permission. The only
> > > > restriction
> > > > > > being that it has to be and Amazon employee and access to
> console,
> > > > master
> > > > > > and slave only being possible from the Corp network.
> > > > > >
> > > > > > There is no open ticket. What would you like to request?
> > > > > >
> > > > > > -Marco
> > > > > >
> > > > > >
> > > > > > Am 05.01.2018 7:22 nachm. schrieb "Chris Olivier" <
> > > 

Re: Commiter access to Jenkins Sevrer

[Indhu]

Objective is to make sure committers has access to CI. As far as that
objective is met, it shouldn't matter whether the auth mechanism uses
GitHub or Apache SSO.

If GitHub SSO is easier to implement, let's go ahead and do that unless
someone in the list objects.


On Fri, Jan 5, 2018 at 11:21 AM Chris Olivier  wrote:

> I am fine without a vote unless a vote is required?  Any objections,
> anyone?  You're sort of adding functionality here, not changing or
> restricting...  We can always change to Apache later.
>
> On Fri, Jan 5, 2018 at 11:18 AM, Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
> > I'd be in favour of GitHub. Shall we open a vote or would you like me to
> > create a POC with GitHub first and afterwards we can check if that's
> > enough?
> >
> > -Marco
> >
> > On Fri, Jan 5, 2018 at 8:13 PM, Chris Olivier 
> > wrote:
> >
> > > Apparently Apache supports OATH, so I am open to either.
> > > Good idea for the docker thing.
> > >
> > > On Fri, Jan 5, 2018 at 11:02 AM, Marco de Abreu <
> > > marco.g.ab...@googlemail.com> wrote:
> > >
> > > > GitHub SSO allows the neat feature that login and permission can be
> > > > selected depending on the access rights a user has to a project.
> > Somebody
> > > > with write access (committers) would be get different permissions
> than
> > > > somebody with only read access.
> > > >
> > > > We could check back with Apache for SSO, but this would involve
> Apache
> > > > infra. We could put it up to a vote whether to use GitHub or Apache
> > SSO.
> > > >
> > > > In order to reproduce a build failure we have been thinking about
> > > changing
> > > > the ci_build.sh in such a way that it can be run manually without
> > > Jenkins.
> > > > The setup I took over binds the Jenkins work directory into the
> docker
> > > > containers and uses a few hacks which are hard to reproduce locally.
> We
> > > > plan to reengineer this script to make it easier to run manually.
> > > > But making the AMI public is a good idea! We plan to make the whole
> > > > infrastructure code (based on Terraform) completely public - at the
> > > moment
> > > > it's in a private repository as it contains credentials, but they
> will
> > be
> > > > moved to KMS soon. It would definitely be a good approach to just
> > supply
> > > > the AMI so everybody could recreate the environment in their own
> > account.
> > > >
> > > > -Marco
> > > >
> > > > Am 05.01.2018 7:51 nachm. schrieb "Chris Olivier" <
> > cjolivie...@gmail.com
> > > >:
> > > >
> > > > Well, login to the Jenkins server, I would imagine.
> > > >
> > > > github or Apache SSO (does Apache support OAUTH?) seems like a good
> > idea
> > > as
> > > > long as there's a way to not let everyone with a github account log
> in.
> > > >
> > > > Access to actual slave machines could be more restricted, I imagine.
> > > >
> > > > Eventually, a public current AMI for a build slave would be good in
> > order
> > > > to reproduce build or test problems that can't be reproduced locally.
> > > >
> > > > wdyt?
> > > >
> > > >
> > > >
> > > > On Fri, Jan 5, 2018 at 10:41 AM, Marco de Abreu <
> > > > marco.g.ab...@googlemail.com> wrote:
> > > >
> > > > > Would it be an acceptable solution if we add SSO or do you also
> want
> > > > access
> > > > > to the actual AWS account and all machines?
> > > > >
> > > > > Yes, the build jobs are automatically getting created for new
> > branches.
> > > > >
> > > > > -Marco
> > > > >
> > > > > Am 05.01.2018 7:35 nachm. schrieb "Marco de Abreu" <
> > > > > marco.g.ab...@googlemail.com>:
> > > > >
> > > > > I totally agree, this is not the way it should work in an Apache
> > > Project.
> > > > > It's running on an isengard account, meaning it is only accessible
> > for
> > > > > Amazon employees. The problem is that a compromised account could
> > cause
> > > > > damage up to 170,000$ per day. There are alarms in place to notice
> > > those
> > > > > cases, but we still have to be very careful. These high limits have
> > > been
> > > > > chosen due to auto scaling being added within the next week's.
> > > > >
> > > > > I'd be happy to introduce a committer into the CI process and all
> the
> > > > > necessary steps as well as granting them permission. The only
> > > restriction
> > > > > being that it has to be and Amazon employee and access to console,
> > > master
> > > > > and slave only being possible from the Corp network.
> > > > >
> > > > > There is no open ticket. What would you like to request?
> > > > >
> > > > > -Marco
> > > > >
> > > > >
> > > > > Am 05.01.2018 7:22 nachm. schrieb "Chris Olivier" <
> > > cjolivie...@gmail.com
> > > > >:
> > > > >
> > > > > Like John and other mentors were saying, it's not proper for CI to
> > be a
> > > > > closed/inaccessible environment.  Is it running on an Isengard
> > account
> > > or
> > > > > in PROD or CORP or just generic EC2?  I think that we should remedy
> > > this.
> > > > > It's very strange that no 

Re: Commiter access to Jenkins Sevrer

[Chris Olivier]

I am fine without a vote unless a vote is required?  Any objections,
anyone?  You're sort of adding functionality here, not changing or
restricting...  We can always change to Apache later.

On Fri, Jan 5, 2018 at 11:18 AM, Marco de Abreu <
marco.g.ab...@googlemail.com> wrote:

> I'd be in favour of GitHub. Shall we open a vote or would you like me to
> create a POC with GitHub first and afterwards we can check if that's
> enough?
>
> -Marco
>
> On Fri, Jan 5, 2018 at 8:13 PM, Chris Olivier 
> wrote:
>
> > Apparently Apache supports OATH, so I am open to either.
> > Good idea for the docker thing.
> >
> > On Fri, Jan 5, 2018 at 11:02 AM, Marco de Abreu <
> > marco.g.ab...@googlemail.com> wrote:
> >
> > > GitHub SSO allows the neat feature that login and permission can be
> > > selected depending on the access rights a user has to a project.
> Somebody
> > > with write access (committers) would be get different permissions than
> > > somebody with only read access.
> > >
> > > We could check back with Apache for SSO, but this would involve Apache
> > > infra. We could put it up to a vote whether to use GitHub or Apache
> SSO.
> > >
> > > In order to reproduce a build failure we have been thinking about
> > changing
> > > the ci_build.sh in such a way that it can be run manually without
> > Jenkins.
> > > The setup I took over binds the Jenkins work directory into the docker
> > > containers and uses a few hacks which are hard to reproduce locally. We
> > > plan to reengineer this script to make it easier to run manually.
> > > But making the AMI public is a good idea! We plan to make the whole
> > > infrastructure code (based on Terraform) completely public - at the
> > moment
> > > it's in a private repository as it contains credentials, but they will
> be
> > > moved to KMS soon. It would definitely be a good approach to just
> supply
> > > the AMI so everybody could recreate the environment in their own
> account.
> > >
> > > -Marco
> > >
> > > Am 05.01.2018 7:51 nachm. schrieb "Chris Olivier" <
> cjolivie...@gmail.com
> > >:
> > >
> > > Well, login to the Jenkins server, I would imagine.
> > >
> > > github or Apache SSO (does Apache support OAUTH?) seems like a good
> idea
> > as
> > > long as there's a way to not let everyone with a github account log in.
> > >
> > > Access to actual slave machines could be more restricted, I imagine.
> > >
> > > Eventually, a public current AMI for a build slave would be good in
> order
> > > to reproduce build or test problems that can't be reproduced locally.
> > >
> > > wdyt?
> > >
> > >
> > >
> > > On Fri, Jan 5, 2018 at 10:41 AM, Marco de Abreu <
> > > marco.g.ab...@googlemail.com> wrote:
> > >
> > > > Would it be an acceptable solution if we add SSO or do you also want
> > > access
> > > > to the actual AWS account and all machines?
> > > >
> > > > Yes, the build jobs are automatically getting created for new
> branches.
> > > >
> > > > -Marco
> > > >
> > > > Am 05.01.2018 7:35 nachm. schrieb "Marco de Abreu" <
> > > > marco.g.ab...@googlemail.com>:
> > > >
> > > > I totally agree, this is not the way it should work in an Apache
> > Project.
> > > > It's running on an isengard account, meaning it is only accessible
> for
> > > > Amazon employees. The problem is that a compromised account could
> cause
> > > > damage up to 170,000$ per day. There are alarms in place to notice
> > those
> > > > cases, but we still have to be very careful. These high limits have
> > been
> > > > chosen due to auto scaling being added within the next week's.
> > > >
> > > > I'd be happy to introduce a committer into the CI process and all the
> > > > necessary steps as well as granting them permission. The only
> > restriction
> > > > being that it has to be and Amazon employee and access to console,
> > master
> > > > and slave only being possible from the Corp network.
> > > >
> > > > There is no open ticket. What would you like to request?
> > > >
> > > > -Marco
> > > >
> > > >
> > > > Am 05.01.2018 7:22 nachm. schrieb "Chris Olivier" <
> > cjolivie...@gmail.com
> > > >:
> > > >
> > > > Like John and other mentors were saying, it's not proper for CI to
> be a
> > > > closed/inaccessible environment.  Is it running on an Isengard
> account
> > or
> > > > in PROD or CORP or just generic EC2?  I think that we should remedy
> > this.
> > > > It's very strange that no committers have access at all.  Is there a
> > > ticket
> > > > open to IPSEC?
> > > >
> > > > On Fri, Jan 5, 2018 at 10:17 AM, Marco de Abreu <
> > > > marco.g.ab...@googlemail.com> wrote:
> > > >
> > > > > Hello Chris,
> > > > >
> > > > > At the moment this is not possible due Amazon AppSec (Application
> > > > security)
> > > > > restrictions which does not permit user data and credentials on
> these
> > > > > machines.
> > > > >
> > > > > I have been thinking about adding single sign on bound to GitHub,
> but
> > > we
> > > > > would have to check back with AppSec.
> > > > >
> > > > > Is the 

Re: Commiter access to Jenkins Sevrer

[Marco de Abreu]

I'd be in favour of GitHub. Shall we open a vote or would you like me to
create a POC with GitHub first and afterwards we can check if that's enough?

-Marco

On Fri, Jan 5, 2018 at 8:13 PM, Chris Olivier  wrote:

> Apparently Apache supports OATH, so I am open to either.
> Good idea for the docker thing.
>
> On Fri, Jan 5, 2018 at 11:02 AM, Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
> > GitHub SSO allows the neat feature that login and permission can be
> > selected depending on the access rights a user has to a project. Somebody
> > with write access (committers) would be get different permissions than
> > somebody with only read access.
> >
> > We could check back with Apache for SSO, but this would involve Apache
> > infra. We could put it up to a vote whether to use GitHub or Apache SSO.
> >
> > In order to reproduce a build failure we have been thinking about
> changing
> > the ci_build.sh in such a way that it can be run manually without
> Jenkins.
> > The setup I took over binds the Jenkins work directory into the docker
> > containers and uses a few hacks which are hard to reproduce locally. We
> > plan to reengineer this script to make it easier to run manually.
> > But making the AMI public is a good idea! We plan to make the whole
> > infrastructure code (based on Terraform) completely public - at the
> moment
> > it's in a private repository as it contains credentials, but they will be
> > moved to KMS soon. It would definitely be a good approach to just supply
> > the AMI so everybody could recreate the environment in their own account.
> >
> > -Marco
> >
> > Am 05.01.2018 7:51 nachm. schrieb "Chris Olivier"  >:
> >
> > Well, login to the Jenkins server, I would imagine.
> >
> > github or Apache SSO (does Apache support OAUTH?) seems like a good idea
> as
> > long as there's a way to not let everyone with a github account log in.
> >
> > Access to actual slave machines could be more restricted, I imagine.
> >
> > Eventually, a public current AMI for a build slave would be good in order
> > to reproduce build or test problems that can't be reproduced locally.
> >
> > wdyt?
> >
> >
> >
> > On Fri, Jan 5, 2018 at 10:41 AM, Marco de Abreu <
> > marco.g.ab...@googlemail.com> wrote:
> >
> > > Would it be an acceptable solution if we add SSO or do you also want
> > access
> > > to the actual AWS account and all machines?
> > >
> > > Yes, the build jobs are automatically getting created for new branches.
> > >
> > > -Marco
> > >
> > > Am 05.01.2018 7:35 nachm. schrieb "Marco de Abreu" <
> > > marco.g.ab...@googlemail.com>:
> > >
> > > I totally agree, this is not the way it should work in an Apache
> Project.
> > > It's running on an isengard account, meaning it is only accessible for
> > > Amazon employees. The problem is that a compromised account could cause
> > > damage up to 170,000$ per day. There are alarms in place to notice
> those
> > > cases, but we still have to be very careful. These high limits have
> been
> > > chosen due to auto scaling being added within the next week's.
> > >
> > > I'd be happy to introduce a committer into the CI process and all the
> > > necessary steps as well as granting them permission. The only
> restriction
> > > being that it has to be and Amazon employee and access to console,
> master
> > > and slave only being possible from the Corp network.
> > >
> > > There is no open ticket. What would you like to request?
> > >
> > > -Marco
> > >
> > >
> > > Am 05.01.2018 7:22 nachm. schrieb "Chris Olivier" <
> cjolivie...@gmail.com
> > >:
> > >
> > > Like John and other mentors were saying, it's not proper for CI to be a
> > > closed/inaccessible environment.  Is it running on an Isengard account
> or
> > > in PROD or CORP or just generic EC2?  I think that we should remedy
> this.
> > > It's very strange that no committers have access at all.  Is there a
> > ticket
> > > open to IPSEC?
> > >
> > > On Fri, Jan 5, 2018 at 10:17 AM, Marco de Abreu <
> > > marco.g.ab...@googlemail.com> wrote:
> > >
> > > > Hello Chris,
> > > >
> > > > At the moment this is not possible due Amazon AppSec (Application
> > > security)
> > > > restrictions which does not permit user data and credentials on these
> > > > machines.
> > > >
> > > > I have been thinking about adding single sign on bound to GitHub, but
> > we
> > > > would have to check back with AppSec.
> > > >
> > > > Is the reason for your request still the ability to start and stop
> > > running
> > > > builds?
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > > > Am 05.01.2018 7:11 nachm. schrieb "Chris Olivier" <
> > cjolivie...@gmail.com
> > > >:
> > > >
> > > > Marco,
> > > >
> > > > Are all committers able to get login access to the Jenkins Server?
> If
> > > not,
> > > > why?
> > > >
> > > > -Chris
> > > >
> > >
> >
>

Re: Commiter access to Jenkins Sevrer

[Chris Olivier]

Apparently Apache supports OATH, so I am open to either.
Good idea for the docker thing.

On Fri, Jan 5, 2018 at 11:02 AM, Marco de Abreu <
marco.g.ab...@googlemail.com> wrote:

> GitHub SSO allows the neat feature that login and permission can be
> selected depending on the access rights a user has to a project. Somebody
> with write access (committers) would be get different permissions than
> somebody with only read access.
>
> We could check back with Apache for SSO, but this would involve Apache
> infra. We could put it up to a vote whether to use GitHub or Apache SSO.
>
> In order to reproduce a build failure we have been thinking about changing
> the ci_build.sh in such a way that it can be run manually without Jenkins.
> The setup I took over binds the Jenkins work directory into the docker
> containers and uses a few hacks which are hard to reproduce locally. We
> plan to reengineer this script to make it easier to run manually.
> But making the AMI public is a good idea! We plan to make the whole
> infrastructure code (based on Terraform) completely public - at the moment
> it's in a private repository as it contains credentials, but they will be
> moved to KMS soon. It would definitely be a good approach to just supply
> the AMI so everybody could recreate the environment in their own account.
>
> -Marco
>
> Am 05.01.2018 7:51 nachm. schrieb "Chris Olivier" :
>
> Well, login to the Jenkins server, I would imagine.
>
> github or Apache SSO (does Apache support OAUTH?) seems like a good idea as
> long as there's a way to not let everyone with a github account log in.
>
> Access to actual slave machines could be more restricted, I imagine.
>
> Eventually, a public current AMI for a build slave would be good in order
> to reproduce build or test problems that can't be reproduced locally.
>
> wdyt?
>
>
>
> On Fri, Jan 5, 2018 at 10:41 AM, Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
> > Would it be an acceptable solution if we add SSO or do you also want
> access
> > to the actual AWS account and all machines?
> >
> > Yes, the build jobs are automatically getting created for new branches.
> >
> > -Marco
> >
> > Am 05.01.2018 7:35 nachm. schrieb "Marco de Abreu" <
> > marco.g.ab...@googlemail.com>:
> >
> > I totally agree, this is not the way it should work in an Apache Project.
> > It's running on an isengard account, meaning it is only accessible for
> > Amazon employees. The problem is that a compromised account could cause
> > damage up to 170,000$ per day. There are alarms in place to notice those
> > cases, but we still have to be very careful. These high limits have been
> > chosen due to auto scaling being added within the next week's.
> >
> > I'd be happy to introduce a committer into the CI process and all the
> > necessary steps as well as granting them permission. The only restriction
> > being that it has to be and Amazon employee and access to console, master
> > and slave only being possible from the Corp network.
> >
> > There is no open ticket. What would you like to request?
> >
> > -Marco
> >
> >
> > Am 05.01.2018 7:22 nachm. schrieb "Chris Olivier"  >:
> >
> > Like John and other mentors were saying, it's not proper for CI to be a
> > closed/inaccessible environment.  Is it running on an Isengard account or
> > in PROD or CORP or just generic EC2?  I think that we should remedy this.
> > It's very strange that no committers have access at all.  Is there a
> ticket
> > open to IPSEC?
> >
> > On Fri, Jan 5, 2018 at 10:17 AM, Marco de Abreu <
> > marco.g.ab...@googlemail.com> wrote:
> >
> > > Hello Chris,
> > >
> > > At the moment this is not possible due Amazon AppSec (Application
> > security)
> > > restrictions which does not permit user data and credentials on these
> > > machines.
> > >
> > > I have been thinking about adding single sign on bound to GitHub, but
> we
> > > would have to check back with AppSec.
> > >
> > > Is the reason for your request still the ability to start and stop
> > running
> > > builds?
> > >
> > > Best regards,
> > > Marco
> > >
> > > Am 05.01.2018 7:11 nachm. schrieb "Chris Olivier" <
> cjolivie...@gmail.com
> > >:
> > >
> > > Marco,
> > >
> > > Are all committers able to get login access to the Jenkins Server?  If
> > not,
> > > why?
> > >
> > > -Chris
> > >
> >
>

Re: Commiter access to Jenkins Sevrer

[Marco de Abreu]

GitHub SSO allows the neat feature that login and permission can be
selected depending on the access rights a user has to a project. Somebody
with write access (committers) would be get different permissions than
somebody with only read access.

We could check back with Apache for SSO, but this would involve Apache
infra. We could put it up to a vote whether to use GitHub or Apache SSO.

In order to reproduce a build failure we have been thinking about changing
the ci_build.sh in such a way that it can be run manually without Jenkins.
The setup I took over binds the Jenkins work directory into the docker
containers and uses a few hacks which are hard to reproduce locally. We
plan to reengineer this script to make it easier to run manually.
But making the AMI public is a good idea! We plan to make the whole
infrastructure code (based on Terraform) completely public - at the moment
it's in a private repository as it contains credentials, but they will be
moved to KMS soon. It would definitely be a good approach to just supply
the AMI so everybody could recreate the environment in their own account.

-Marco

Am 05.01.2018 7:51 nachm. schrieb "Chris Olivier" :

Well, login to the Jenkins server, I would imagine.

github or Apache SSO (does Apache support OAUTH?) seems like a good idea as
long as there's a way to not let everyone with a github account log in.

Access to actual slave machines could be more restricted, I imagine.

Eventually, a public current AMI for a build slave would be good in order
to reproduce build or test problems that can't be reproduced locally.

wdyt?



On Fri, Jan 5, 2018 at 10:41 AM, Marco de Abreu <
marco.g.ab...@googlemail.com> wrote:

> Would it be an acceptable solution if we add SSO or do you also want
access
> to the actual AWS account and all machines?
>
> Yes, the build jobs are automatically getting created for new branches.
>
> -Marco
>
> Am 05.01.2018 7:35 nachm. schrieb "Marco de Abreu" <
> marco.g.ab...@googlemail.com>:
>
> I totally agree, this is not the way it should work in an Apache Project.
> It's running on an isengard account, meaning it is only accessible for
> Amazon employees. The problem is that a compromised account could cause
> damage up to 170,000$ per day. There are alarms in place to notice those
> cases, but we still have to be very careful. These high limits have been
> chosen due to auto scaling being added within the next week's.
>
> I'd be happy to introduce a committer into the CI process and all the
> necessary steps as well as granting them permission. The only restriction
> being that it has to be and Amazon employee and access to console, master
> and slave only being possible from the Corp network.
>
> There is no open ticket. What would you like to request?
>
> -Marco
>
>
> Am 05.01.2018 7:22 nachm. schrieb "Chris Olivier" :
>
> Like John and other mentors were saying, it's not proper for CI to be a
> closed/inaccessible environment.  Is it running on an Isengard account or
> in PROD or CORP or just generic EC2?  I think that we should remedy this.
> It's very strange that no committers have access at all.  Is there a
ticket
> open to IPSEC?
>
> On Fri, Jan 5, 2018 at 10:17 AM, Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
> > Hello Chris,
> >
> > At the moment this is not possible due Amazon AppSec (Application
> security)
> > restrictions which does not permit user data and credentials on these
> > machines.
> >
> > I have been thinking about adding single sign on bound to GitHub, but we
> > would have to check back with AppSec.
> >
> > Is the reason for your request still the ability to start and stop
> running
> > builds?
> >
> > Best regards,
> > Marco
> >
> > Am 05.01.2018 7:11 nachm. schrieb "Chris Olivier"  >:
> >
> > Marco,
> >
> > Are all committers able to get login access to the Jenkins Server?  If
> not,
> > why?
> >
> > -Chris
> >
>

Re: Commiter access to Jenkins Sevrer

[Naveen Swamy]

SSO is enough.

On Fri, Jan 5, 2018 at 10:51 AM, Chris Olivier 
wrote:

> Well, login to the Jenkins server, I would imagine.
>
> github or Apache SSO (does Apache support OAUTH?) seems like a good idea as
> long as there's a way to not let everyone with a github account log in.
>
> Access to actual slave machines could be more restricted, I imagine.
>
> Eventually, a public current AMI for a build slave would be good in order
> to reproduce build or test problems that can't be reproduced locally.
>
> wdyt?
>
>
>
> On Fri, Jan 5, 2018 at 10:41 AM, Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
> > Would it be an acceptable solution if we add SSO or do you also want
> access
> > to the actual AWS account and all machines?
> >
> > Yes, the build jobs are automatically getting created for new branches.
> >
> > -Marco
> >
> > Am 05.01.2018 7:35 nachm. schrieb "Marco de Abreu" <
> > marco.g.ab...@googlemail.com>:
> >
> > I totally agree, this is not the way it should work in an Apache Project.
> > It's running on an isengard account, meaning it is only accessible for
> > Amazon employees. The problem is that a compromised account could cause
> > damage up to 170,000$ per day. There are alarms in place to notice those
> > cases, but we still have to be very careful. These high limits have been
> > chosen due to auto scaling being added within the next week's.
> >
> > I'd be happy to introduce a committer into the CI process and all the
> > necessary steps as well as granting them permission. The only restriction
> > being that it has to be and Amazon employee and access to console, master
> > and slave only being possible from the Corp network.
> >
> > There is no open ticket. What would you like to request?
> >
> > -Marco
> >
> >
> > Am 05.01.2018 7:22 nachm. schrieb "Chris Olivier"  >:
> >
> > Like John and other mentors were saying, it's not proper for CI to be a
> > closed/inaccessible environment.  Is it running on an Isengard account or
> > in PROD or CORP or just generic EC2?  I think that we should remedy this.
> > It's very strange that no committers have access at all.  Is there a
> ticket
> > open to IPSEC?
> >
> > On Fri, Jan 5, 2018 at 10:17 AM, Marco de Abreu <
> > marco.g.ab...@googlemail.com> wrote:
> >
> > > Hello Chris,
> > >
> > > At the moment this is not possible due Amazon AppSec (Application
> > security)
> > > restrictions which does not permit user data and credentials on these
> > > machines.
> > >
> > > I have been thinking about adding single sign on bound to GitHub, but
> we
> > > would have to check back with AppSec.
> > >
> > > Is the reason for your request still the ability to start and stop
> > running
> > > builds?
> > >
> > > Best regards,
> > > Marco
> > >
> > > Am 05.01.2018 7:11 nachm. schrieb "Chris Olivier" <
> cjolivie...@gmail.com
> > >:
> > >
> > > Marco,
> > >
> > > Are all committers able to get login access to the Jenkins Server?  If
> > not,
> > > why?
> > >
> > > -Chris
> > >
> >
>

Re: Commiter access to Jenkins Sevrer

[Chris Olivier]

Well, login to the Jenkins server, I would imagine.

github or Apache SSO (does Apache support OAUTH?) seems like a good idea as
long as there's a way to not let everyone with a github account log in.

Access to actual slave machines could be more restricted, I imagine.

Eventually, a public current AMI for a build slave would be good in order
to reproduce build or test problems that can't be reproduced locally.

wdyt?



On Fri, Jan 5, 2018 at 10:41 AM, Marco de Abreu <
marco.g.ab...@googlemail.com> wrote:

> Would it be an acceptable solution if we add SSO or do you also want access
> to the actual AWS account and all machines?
>
> Yes, the build jobs are automatically getting created for new branches.
>
> -Marco
>
> Am 05.01.2018 7:35 nachm. schrieb "Marco de Abreu" <
> marco.g.ab...@googlemail.com>:
>
> I totally agree, this is not the way it should work in an Apache Project.
> It's running on an isengard account, meaning it is only accessible for
> Amazon employees. The problem is that a compromised account could cause
> damage up to 170,000$ per day. There are alarms in place to notice those
> cases, but we still have to be very careful. These high limits have been
> chosen due to auto scaling being added within the next week's.
>
> I'd be happy to introduce a committer into the CI process and all the
> necessary steps as well as granting them permission. The only restriction
> being that it has to be and Amazon employee and access to console, master
> and slave only being possible from the Corp network.
>
> There is no open ticket. What would you like to request?
>
> -Marco
>
>
> Am 05.01.2018 7:22 nachm. schrieb "Chris Olivier" :
>
> Like John and other mentors were saying, it's not proper for CI to be a
> closed/inaccessible environment.  Is it running on an Isengard account or
> in PROD or CORP or just generic EC2?  I think that we should remedy this.
> It's very strange that no committers have access at all.  Is there a ticket
> open to IPSEC?
>
> On Fri, Jan 5, 2018 at 10:17 AM, Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
> > Hello Chris,
> >
> > At the moment this is not possible due Amazon AppSec (Application
> security)
> > restrictions which does not permit user data and credentials on these
> > machines.
> >
> > I have been thinking about adding single sign on bound to GitHub, but we
> > would have to check back with AppSec.
> >
> > Is the reason for your request still the ability to start and stop
> running
> > builds?
> >
> > Best regards,
> > Marco
> >
> > Am 05.01.2018 7:11 nachm. schrieb "Chris Olivier"  >:
> >
> > Marco,
> >
> > Are all committers able to get login access to the Jenkins Server?  If
> not,
> > why?
> >
> > -Chris
> >
>

Re: Commiter access to Jenkins Sevrer

[Marco de Abreu]

Would it be an acceptable solution if we add SSO or do you also want access
to the actual AWS account and all machines?

Yes, the build jobs are automatically getting created for new branches.

-Marco

Am 05.01.2018 7:35 nachm. schrieb "Marco de Abreu" <
marco.g.ab...@googlemail.com>:

I totally agree, this is not the way it should work in an Apache Project.
It's running on an isengard account, meaning it is only accessible for
Amazon employees. The problem is that a compromised account could cause
damage up to 170,000$ per day. There are alarms in place to notice those
cases, but we still have to be very careful. These high limits have been
chosen due to auto scaling being added within the next week's.

I'd be happy to introduce a committer into the CI process and all the
necessary steps as well as granting them permission. The only restriction
being that it has to be and Amazon employee and access to console, master
and slave only being possible from the Corp network.

There is no open ticket. What would you like to request?

-Marco


Am 05.01.2018 7:22 nachm. schrieb "Chris Olivier" :

Like John and other mentors were saying, it's not proper for CI to be a
closed/inaccessible environment.  Is it running on an Isengard account or
in PROD or CORP or just generic EC2?  I think that we should remedy this.
It's very strange that no committers have access at all.  Is there a ticket
open to IPSEC?

On Fri, Jan 5, 2018 at 10:17 AM, Marco de Abreu <
marco.g.ab...@googlemail.com> wrote:

> Hello Chris,
>
> At the moment this is not possible due Amazon AppSec (Application
security)
> restrictions which does not permit user data and credentials on these
> machines.
>
> I have been thinking about adding single sign on bound to GitHub, but we
> would have to check back with AppSec.
>
> Is the reason for your request still the ability to start and stop running
> builds?
>
> Best regards,
> Marco
>
> Am 05.01.2018 7:11 nachm. schrieb "Chris Olivier" :
>
> Marco,
>
> Are all committers able to get login access to the Jenkins Server?  If
not,
> why?
>
> -Chris
>

Re: Commiter access to Jenkins Sevrer

[Marco de Abreu]

I totally agree, this is not the way it should work in an Apache Project.
It's running on an isengard account, meaning it is only accessible for
Amazon employees. The problem is that a compromised account could cause
damage up to 170,000$ per day. There are alarms in place to notice those
cases, but we still have to be very careful. These high limits have been
chosen due to auto scaling being added within the next week's.

I'd be happy to introduce a committer into the CI process and all the
necessary steps as well as granting them permission. The only restriction
being that it has to be and Amazon employee and access to console, master
and slave only being possible from the Corp network.

There is no open ticket. What would you like to request?

-Marco

Am 05.01.2018 7:22 nachm. schrieb "Chris Olivier" :

Like John and other mentors were saying, it's not proper for CI to be a
closed/inaccessible environment.  Is it running on an Isengard account or
in PROD or CORP or just generic EC2?  I think that we should remedy this.
It's very strange that no committers have access at all.  Is there a ticket
open to IPSEC?

On Fri, Jan 5, 2018 at 10:17 AM, Marco de Abreu <
marco.g.ab...@googlemail.com> wrote:

> Hello Chris,
>
> At the moment this is not possible due Amazon AppSec (Application
security)
> restrictions which does not permit user data and credentials on these
> machines.
>
> I have been thinking about adding single sign on bound to GitHub, but we
> would have to check back with AppSec.
>
> Is the reason for your request still the ability to start and stop running
> builds?
>
> Best regards,
> Marco
>
> Am 05.01.2018 7:11 nachm. schrieb "Chris Olivier" :
>
> Marco,
>
> Are all committers able to get login access to the Jenkins Server?  If
not,
> why?
>
> -Chris
>

Re: Commiter access to Jenkins Sevrer

[Naveen Swamy]

I would like this access as well. for start/stop and also replay parts of
the build. It was very convenient when i am trying to a make a small change
to the Jenkins file and be able to test that and not have to go through the
entire build cycle.

Also If I am working on a branch, do you automatically create build jobs
for the new branches?



On Fri, Jan 5, 2018 at 10:17 AM, Marco de Abreu <
marco.g.ab...@googlemail.com> wrote:

> Hello Chris,
>
> At the moment this is not possible due Amazon AppSec (Application security)
> restrictions which does not permit user data and credentials on these
> machines.
>
> I have been thinking about adding single sign on bound to GitHub, but we
> would have to check back with AppSec.
>
> Is the reason for your request still the ability to start and stop running
> builds?
>
> Best regards,
> Marco
>
> Am 05.01.2018 7:11 nachm. schrieb "Chris Olivier" :
>
> Marco,
>
> Are all committers able to get login access to the Jenkins Server?  If not,
> why?
>
> -Chris
>

Re: Commiter access to Jenkins Sevrer

[Marco de Abreu]

Hello Chris,

At the moment this is not possible due Amazon AppSec (Application security)
restrictions which does not permit user data and credentials on these
machines.

I have been thinking about adding single sign on bound to GitHub, but we
would have to check back with AppSec.

Is the reason for your request still the ability to start and stop running
builds?

Best regards,
Marco

Am 05.01.2018 7:11 nachm. schrieb "Chris Olivier" :

Marco,

Are all committers able to get login access to the Jenkins Server?  If not,
why?

-Chris