Re: [Gluster-infra] Restore github mirror sync for libgfapi-python and gluster-swift
The sync is up now and the github mirrors are up-to-date. I think the gerrit restart did the trick. Regards, -Prashanth Pai - Original Message - > From: "Prashanth Pai" > To: "Michael Scherer" > Cc: "gluster-infra" > Sent: Friday, March 11, 2016 12:52:38 PM > Subject: Re: [Gluster-infra] Restore github mirror sync for libgfapi-python > and gluster-swift > > > > - Original Message - > > From: "Michael Scherer" > > To: "gluster-infra" > > Sent: Wednesday, March 9, 2016 3:01:01 PM > > Subject: Re: [Gluster-infra] Restore github mirror sync for libgfapi-python > > and gluster-swift > > > > Le mercredi 09 mars 2016 à 12:44 +0530, Kaushal M a écrit : > > > This happened because of the changes we did with ssh-keys used for > > > replication. I'd forgotten that this had been done, and was expecting > > > pushes to happen automatically. > > > > > > Earlier, a single users key was used (key of one of the admins of the > > > gluster organization, Avati initially) to push changes from gerrit to > > > github. > > > This enabled all repositories in gerrit to be automatically be pushed > > > into to its replica in github ( in gerrit would be > > > automatically pushed to github/). > > > This key got changed/deleted which caused replication to fail a while > > > back. > > > > > > To avoid tying pushes to a single user like this, a separate key was > > > setup for each github repo, and gerrit was setup to use the respective > > > keys to push to each repo. > > > This requires manual replication configuration. This change was > > > implemented only for the glusterfs and glusterfs-specs repos. > > > All other repos require manual configuration. > > > > Yep. I did sent the process on the list, but didn't automate as i was > > not expecting more repo to be added. > > > > > We'll try to comeup with a better way to do this replication. If we > > > can't, we'll need to manually set up the keys and encryption for each > > > repo. > > > > Yeah, so the ideal situation is that we have some automation. However, > > doing that for gluster seems a bit more complicated (there is API, but > > then you have to deal with oauth, did I already said how much SaaS > > service make our life harder today ? ). > > > > We did discuss on irc about it, and so there is basically 2 solutions: > > > > - go back to a gluster bot account, but that requires a way to secure > > the password properly (especially since no one will be checking the > > audit log for it). We could use 2FA for that account using a remote > > command line script on a secure server, but we still need a way to store > > the password in a secure way. And of course, like all password > > management, it requires that we decide who has access, etc, etc. > > > > - make something that automate part of it, and let the part about adding > > credentials to github to a admin. That's the part I plan to do later, > > since that could be just some ansible playbook > > > > - make something more like self service. Have people declare "I want to > > sync that repo to that repo I am admin of", and have it done > > automatically on their behalf. It would requires a lot more coding > > around github api, but would be the cleanest. > > > > > > Another issue that arise with the current scheme is that push are made > > under my name as I am the one that added the keys. That was somewhat > > unexpected, and I do not see a easy way out of it. (unfortunately, it > > doesn't appear in my stats, so I can't have a endless commit streak..) > > I am not sure if that's a problem in practice or not, since that's not > > impacting the git repo, just the activity stream. (again, we can't fix > > it to be correct, because we are depending on a proprietary SaaS > > offering, but I already complained about it today) > > > > Oh, and I did add the replication for the 2 repos for now. > > This does not seem to have worked. I was hoping it would be triggered from > Gerrit when a change on review is merged. At least one change on review was > merged yesterday to both repos after the replication was re-established but > I still don't see them on github. For now, I'll keep the github repo as is > if it helps finding out why the sync failed to happen. I did take a look at > the gerrit plugin[1] for replication but I don't have access to look
Re: [Gluster-infra] Restore github mirror sync for libgfapi-python and gluster-swift
- Original Message - > From: "Michael Scherer" > To: "gluster-infra" > Sent: Wednesday, March 9, 2016 3:01:01 PM > Subject: Re: [Gluster-infra] Restore github mirror sync for libgfapi-python > and gluster-swift > > Le mercredi 09 mars 2016 à 12:44 +0530, Kaushal M a écrit : > > This happened because of the changes we did with ssh-keys used for > > replication. I'd forgotten that this had been done, and was expecting > > pushes to happen automatically. > > > > Earlier, a single users key was used (key of one of the admins of the > > gluster organization, Avati initially) to push changes from gerrit to > > github. > > This enabled all repositories in gerrit to be automatically be pushed > > into to its replica in github ( in gerrit would be > > automatically pushed to github/). > > This key got changed/deleted which caused replication to fail a while back. > > > > To avoid tying pushes to a single user like this, a separate key was > > setup for each github repo, and gerrit was setup to use the respective > > keys to push to each repo. > > This requires manual replication configuration. This change was > > implemented only for the glusterfs and glusterfs-specs repos. > > All other repos require manual configuration. > > Yep. I did sent the process on the list, but didn't automate as i was > not expecting more repo to be added. > > > We'll try to comeup with a better way to do this replication. If we > > can't, we'll need to manually set up the keys and encryption for each > > repo. > > Yeah, so the ideal situation is that we have some automation. However, > doing that for gluster seems a bit more complicated (there is API, but > then you have to deal with oauth, did I already said how much SaaS > service make our life harder today ? ). > > We did discuss on irc about it, and so there is basically 2 solutions: > > - go back to a gluster bot account, but that requires a way to secure > the password properly (especially since no one will be checking the > audit log for it). We could use 2FA for that account using a remote > command line script on a secure server, but we still need a way to store > the password in a secure way. And of course, like all password > management, it requires that we decide who has access, etc, etc. > > - make something that automate part of it, and let the part about adding > credentials to github to a admin. That's the part I plan to do later, > since that could be just some ansible playbook > > - make something more like self service. Have people declare "I want to > sync that repo to that repo I am admin of", and have it done > automatically on their behalf. It would requires a lot more coding > around github api, but would be the cleanest. > > > Another issue that arise with the current scheme is that push are made > under my name as I am the one that added the keys. That was somewhat > unexpected, and I do not see a easy way out of it. (unfortunately, it > doesn't appear in my stats, so I can't have a endless commit streak..) > I am not sure if that's a problem in practice or not, since that's not > impacting the git repo, just the activity stream. (again, we can't fix > it to be correct, because we are depending on a proprietary SaaS > offering, but I already complained about it today) > > Oh, and I did add the replication for the 2 repos for now. This does not seem to have worked. I was hoping it would be triggered from Gerrit when a change on review is merged. At least one change on review was merged yesterday to both repos after the replication was re-established but I still don't see them on github. For now, I'll keep the github repo as is if it helps finding out why the sync failed to happen. I did take a look at the gerrit plugin[1] for replication but I don't have access to look at existing config. I'll push the changes manually to github repos later if this can't work. [1]: https://gerrit.googlesource.com/plugins/replication/+doc/master/src/main/resources/Documentation/config.md > -- > Michael Scherer > Sysadmin, Community Infrastructure and Platform, OSAS > > > > ___ > Gluster-infra mailing list > Gluster-infra@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-infra ___ Gluster-infra mailing list Gluster-infra@gluster.org http://www.gluster.org/mailman/listinfo/gluster-infra
Re: [Gluster-infra] Restore github mirror sync for libgfapi-python and gluster-swift
Le mercredi 09 mars 2016 à 12:44 +0530, Kaushal M a écrit : > This happened because of the changes we did with ssh-keys used for > replication. I'd forgotten that this had been done, and was expecting > pushes to happen automatically. > > Earlier, a single users key was used (key of one of the admins of the > gluster organization, Avati initially) to push changes from gerrit to > github. > This enabled all repositories in gerrit to be automatically be pushed > into to its replica in github ( in gerrit would be > automatically pushed to github/). > This key got changed/deleted which caused replication to fail a while back. > > To avoid tying pushes to a single user like this, a separate key was > setup for each github repo, and gerrit was setup to use the respective > keys to push to each repo. > This requires manual replication configuration. This change was > implemented only for the glusterfs and glusterfs-specs repos. > All other repos require manual configuration. Yep. I did sent the process on the list, but didn't automate as i was not expecting more repo to be added. > We'll try to comeup with a better way to do this replication. If we > can't, we'll need to manually set up the keys and encryption for each > repo. Yeah, so the ideal situation is that we have some automation. However, doing that for gluster seems a bit more complicated (there is API, but then you have to deal with oauth, did I already said how much SaaS service make our life harder today ? ). We did discuss on irc about it, and so there is basically 2 solutions: - go back to a gluster bot account, but that requires a way to secure the password properly (especially since no one will be checking the audit log for it). We could use 2FA for that account using a remote command line script on a secure server, but we still need a way to store the password in a secure way. And of course, like all password management, it requires that we decide who has access, etc, etc. - make something that automate part of it, and let the part about adding credentials to github to a admin. That's the part I plan to do later, since that could be just some ansible playbook - make something more like self service. Have people declare "I want to sync that repo to that repo I am admin of", and have it done automatically on their behalf. It would requires a lot more coding around github api, but would be the cleanest. Another issue that arise with the current scheme is that push are made under my name as I am the one that added the keys. That was somewhat unexpected, and I do not see a easy way out of it. (unfortunately, it doesn't appear in my stats, so I can't have a endless commit streak..) I am not sure if that's a problem in practice or not, since that's not impacting the git repo, just the activity stream. (again, we can't fix it to be correct, because we are depending on a proprietary SaaS offering, but I already complained about it today) Oh, and I did add the replication for the 2 repos for now. -- Michael Scherer Sysadmin, Community Infrastructure and Platform, OSAS signature.asc Description: This is a digitally signed message part ___ Gluster-infra mailing list Gluster-infra@gluster.org http://www.gluster.org/mailman/listinfo/gluster-infra
Re: [Gluster-infra] Restore github mirror sync for libgfapi-python and gluster-swift
This happened because of the changes we did with ssh-keys used for replication. I'd forgotten that this had been done, and was expecting pushes to happen automatically. Earlier, a single users key was used (key of one of the admins of the gluster organization, Avati initially) to push changes from gerrit to github. This enabled all repositories in gerrit to be automatically be pushed into to its replica in github ( in gerrit would be automatically pushed to github/). This key got changed/deleted which caused replication to fail a while back. To avoid tying pushes to a single user like this, a separate key was setup for each github repo, and gerrit was setup to use the respective keys to push to each repo. This requires manual replication configuration. This change was implemented only for the glusterfs and glusterfs-specs repos. All other repos require manual configuration. We'll try to comeup with a better way to do this replication. If we can't, we'll need to manually set up the keys and encryption for each repo. ~kaushal On Wed, Mar 9, 2016 at 12:22 PM, Prashanth Pai wrote: > Hi, > > The github mirror[1] sync is broken for libgfapi-python and gluster-swift > repos[2]. > Can someone please take a look ? > > Thanks. > > [1]: Github mirrors: > https://github.com/gluster/libgfapi-python > https://github.com/gluster/gluster-swift > > [2]: Gerrit repos: > git://review.gluster.org/libgfapi-python > git://review.gluster.org/gluster-swift > > Regards, > -Prashanth Pai > ___ > Gluster-infra mailing list > Gluster-infra@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-infra ___ Gluster-infra mailing list Gluster-infra@gluster.org http://www.gluster.org/mailman/listinfo/gluster-infra