Re: [Gluster-infra] Restore github mirror sync for libgfapi-python and gluster-swift

[Prashanth Pai]

The sync is up now and the github mirrors are up-to-date.
I think the gerrit restart did the trick.

Regards,
 -Prashanth Pai

- Original Message -
> From: "Prashanth Pai" <p...@redhat.com>
> To: "Michael Scherer" <msche...@redhat.com>
> Cc: "gluster-infra" <gluster-infra@gluster.org>
> Sent: Friday, March 11, 2016 12:52:38 PM
> Subject: Re: [Gluster-infra] Restore github mirror sync for libgfapi-python 
> and gluster-swift
> 
> 
> 
> - Original Message -
> > From: "Michael Scherer" <msche...@redhat.com>
> > To: "gluster-infra" <gluster-infra@gluster.org>
> > Sent: Wednesday, March 9, 2016 3:01:01 PM
> > Subject: Re: [Gluster-infra] Restore github mirror sync for libgfapi-python
> > and gluster-swift
> > 
> > Le mercredi 09 mars 2016 à 12:44 +0530, Kaushal M a écrit :
> > > This happened because of the changes we did with ssh-keys used for
> > > replication. I'd forgotten that this had been done, and was expecting
> > > pushes to happen automatically.
> > > 
> > > Earlier, a single users key was used (key of one of the admins of the
> > > gluster organization, Avati initially) to push changes from gerrit to
> > > github.
> > > This enabled all repositories in gerrit to be automatically be pushed
> > > into to its replica in github ( in gerrit would be
> > > automatically pushed to github/).
> > > This key got changed/deleted which caused replication to fail a while
> > > back.
> > > 
> > > To avoid tying pushes to a single user like this, a separate key was
> > > setup for each github repo, and gerrit was setup to use the respective
> > > keys to push to each repo.
> > > This requires manual replication configuration. This change was
> > > implemented only for the glusterfs and glusterfs-specs repos.
> > > All other repos require manual configuration.
> > 
> > Yep. I did sent the process on the list, but didn't automate as i was
> > not expecting more repo to be added.
> > 
> > > We'll try to comeup with a better way to do this replication. If we
> > > can't, we'll need to manually set up the keys and encryption for each
> > > repo.
> > 
> > Yeah, so the ideal situation is that we have some automation. However,
> > doing that for gluster seems a bit more complicated (there is API, but
> > then you have to deal with oauth, did I already said how much SaaS
> > service make our life harder today ? ).
> > 
> > We did discuss on irc about it, and so there is basically 2 solutions:
> > 
> > - go back to a gluster bot account, but that requires a way to secure
> > the password properly (especially since no one will be checking the
> > audit log for it). We could use 2FA for that account using a remote
> > command line script on a secure server, but we still need a way to store
> > the password in a secure way. And of course, like all password
> > management, it requires that we decide who has access, etc, etc.
> > 
> > - make something that automate part of it, and let the part about adding
> > credentials to github to a admin. That's the part I plan to do later,
> > since that could be just some ansible playbook
> > 
> > - make something more like self service. Have people declare "I want to
> > sync that repo to that repo I am admin of", and have it done
> > automatically on their behalf. It would requires a lot more coding
> > around github api, but would be the cleanest.
> > 
> > 
> > Another issue that arise with the current scheme is that push are made
> > under my name as I am the one that added the keys. That was somewhat
> > unexpected, and I do not see a easy way out of it. (unfortunately, it
> > doesn't appear in my stats, so I can't have a endless commit streak..)
> > I am not sure if that's a problem in practice or not, since that's not
> > impacting the git repo, just the activity stream. (again, we can't fix
> > it to be correct, because we are depending on a proprietary SaaS
> > offering, but I already complained about it today)
> > 
> > Oh, and I did add the replication for the 2 repos for now.
> 
> This does not seem to have worked. I was hoping it would be triggered from
> Gerrit when a change on review is merged. At least one change on review was
> merged yesterday to both repos after the replication was re-established but
> I still don't see them on github. For now, I'll keep the github repo as is
> if it helps finding out why

Re: [Gluster-infra] Restore github mirror sync for libgfapi-python and gluster-swift

[Prashanth Pai]

- Original Message -
> From: "Michael Scherer" <msche...@redhat.com>
> To: "gluster-infra" <gluster-infra@gluster.org>
> Sent: Wednesday, March 9, 2016 3:01:01 PM
> Subject: Re: [Gluster-infra] Restore github mirror sync for libgfapi-python 
> and gluster-swift
> 
> Le mercredi 09 mars 2016 à 12:44 +0530, Kaushal M a écrit :
> > This happened because of the changes we did with ssh-keys used for
> > replication. I'd forgotten that this had been done, and was expecting
> > pushes to happen automatically.
> > 
> > Earlier, a single users key was used (key of one of the admins of the
> > gluster organization, Avati initially) to push changes from gerrit to
> > github.
> > This enabled all repositories in gerrit to be automatically be pushed
> > into to its replica in github ( in gerrit would be
> > automatically pushed to github/).
> > This key got changed/deleted which caused replication to fail a while back.
> > 
> > To avoid tying pushes to a single user like this, a separate key was
> > setup for each github repo, and gerrit was setup to use the respective
> > keys to push to each repo.
> > This requires manual replication configuration. This change was
> > implemented only for the glusterfs and glusterfs-specs repos.
> > All other repos require manual configuration.
> 
> Yep. I did sent the process on the list, but didn't automate as i was
> not expecting more repo to be added.
> 
> > We'll try to comeup with a better way to do this replication. If we
> > can't, we'll need to manually set up the keys and encryption for each
> > repo.
> 
> Yeah, so the ideal situation is that we have some automation. However,
> doing that for gluster seems a bit more complicated (there is API, but
> then you have to deal with oauth, did I already said how much SaaS
> service make our life harder today ? ).
> 
> We did discuss on irc about it, and so there is basically 2 solutions:
> 
> - go back to a gluster bot account, but that requires a way to secure
> the password properly (especially since no one will be checking the
> audit log for it). We could use 2FA for that account using a remote
> command line script on a secure server, but we still need a way to store
> the password in a secure way. And of course, like all password
> management, it requires that we decide who has access, etc, etc.
> 
> - make something that automate part of it, and let the part about adding
> credentials to github to a admin. That's the part I plan to do later,
> since that could be just some ansible playbook
> 
> - make something more like self service. Have people declare "I want to
> sync that repo to that repo I am admin of", and have it done
> automatically on their behalf. It would requires a lot more coding
> around github api, but would be the cleanest.
> 
> 
> Another issue that arise with the current scheme is that push are made
> under my name as I am the one that added the keys. That was somewhat
> unexpected, and I do not see a easy way out of it. (unfortunately, it
> doesn't appear in my stats, so I can't have a endless commit streak..)
> I am not sure if that's a problem in practice or not, since that's not
> impacting the git repo, just the activity stream. (again, we can't fix
> it to be correct, because we are depending on a proprietary SaaS
> offering, but I already complained about it today)
> 
> Oh, and I did add the replication for the 2 repos for now.

This does not seem to have worked. I was hoping it would be triggered from
Gerrit when a change on review is merged. At least one change on review was
merged yesterday to both repos after the replication was re-established but
I still don't see them on github. For now, I'll keep the github repo as is
if it helps finding out why the sync failed to happen. I did take a look at
the gerrit plugin[1] for replication but I don't have access to look at
existing config.

I'll push the changes manually to github repos later if this can't work.

[1]: 
https://gerrit.googlesource.com/plugins/replication/+doc/master/src/main/resources/Documentation/config.md

> --
> Michael Scherer
> Sysadmin, Community Infrastructure and Platform, OSAS
> 
> 
> 
> ___
> Gluster-infra mailing list
> Gluster-infra@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-infra
___
Gluster-infra mailing list
Gluster-infra@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-infra

Re: [Gluster-infra] Restore github mirror sync for libgfapi-python and gluster-swift

[Michael Scherer]

Le mercredi 09 mars 2016 à 12:44 +0530, Kaushal M a écrit :
> This happened because of the changes we did with ssh-keys used for
> replication. I'd forgotten that this had been done, and was expecting
> pushes to happen automatically.
> 
> Earlier, a single users key was used (key of one of the admins of the
> gluster organization, Avati initially) to push changes from gerrit to
> github.
> This enabled all repositories in gerrit to be automatically be pushed
> into to its replica in github ( in gerrit would be
> automatically pushed to github/).
> This key got changed/deleted which caused replication to fail a while back.
> 
> To avoid tying pushes to a single user like this, a separate key was
> setup for each github repo, and gerrit was setup to use the respective
> keys to push to each repo.
> This requires manual replication configuration. This change was
> implemented only for the glusterfs and glusterfs-specs repos.
> All other repos require manual configuration.

Yep. I did sent the process on the list, but didn't automate as i was
not expecting more repo to be added.

> We'll try to comeup with a better way to do this replication. If we
> can't, we'll need to manually set up the keys and encryption for each
> repo.

Yeah, so the ideal situation is that we have some automation. However,
doing that for gluster seems a bit more complicated (there is API, but
then you have to deal with oauth, did I already said how much SaaS
service make our life harder today ? ). 

We did discuss on irc about it, and so there is basically 2 solutions:

- go back to a gluster bot account, but that requires a way to secure
the password properly (especially since no one will be checking the
audit log for it). We could use 2FA for that account using a remote
command line script on a secure server, but we still need a way to store
the password in a secure way. And of course, like all password
management, it requires that we decide who has access, etc, etc.

- make something that automate part of it, and let the part about adding
credentials to github to a admin. That's the part I plan to do later,
since that could be just some ansible playbook

- make something more like self service. Have people declare "I want to
sync that repo to that repo I am admin of", and have it done
automatically on their behalf. It would requires a lot more coding
around github api, but would be the cleanest.


Another issue that arise with the current scheme is that push are made
under my name as I am the one that added the keys. That was somewhat
unexpected, and I do not see a easy way out of it. (unfortunately, it
doesn't appear in my stats, so I can't have a endless commit streak..)
I am not sure if that's a problem in practice or not, since that's not
impacting the git repo, just the activity stream. (again, we can't fix
it to be correct, because we are depending on a proprietary SaaS
offering, but I already complained about it today)

Oh, and I did add the replication for the 2 repos for now.
-- 
Michael Scherer
Sysadmin, Community Infrastructure and Platform, OSAS




signature.asc
Description: This is a digitally signed message part
___
Gluster-infra mailing list
Gluster-infra@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-infra

Re: [Gluster-infra] Restore github mirror sync for libgfapi-python and gluster-swift

[Kaushal M]

This happened because of the changes we did with ssh-keys used for
replication. I'd forgotten that this had been done, and was expecting
pushes to happen automatically.

Earlier, a single users key was used (key of one of the admins of the
gluster organization, Avati initially) to push changes from gerrit to
github.
This enabled all repositories in gerrit to be automatically be pushed
into to its replica in github ( in gerrit would be
automatically pushed to github/).
This key got changed/deleted which caused replication to fail a while back.

To avoid tying pushes to a single user like this, a separate key was
setup for each github repo, and gerrit was setup to use the respective
keys to push to each repo.
This requires manual replication configuration. This change was
implemented only for the glusterfs and glusterfs-specs repos.
All other repos require manual configuration.

We'll try to comeup with a better way to do this replication. If we
can't, we'll need to manually set up the keys and encryption for each
repo.

~kaushal

On Wed, Mar 9, 2016 at 12:22 PM, Prashanth Pai  wrote:
> Hi,
>
> The github mirror[1] sync is broken for libgfapi-python and gluster-swift 
> repos[2].
> Can someone please take a look ?
>
> Thanks.
>
> [1]: Github mirrors:
> https://github.com/gluster/libgfapi-python
> https://github.com/gluster/gluster-swift
>
> [2]: Gerrit repos:
> git://review.gluster.org/libgfapi-python
> git://review.gluster.org/gluster-swift
>
> Regards,
>  -Prashanth Pai
> ___
> Gluster-infra mailing list
> Gluster-infra@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-infra
___
Gluster-infra mailing list
Gluster-infra@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-infra