Re: Fwd: Fwd: git-daemon access-hook race condition
For now I'm trying to do the following: access-hook.bash has: delayed-notify.bash $@ delayed-notify.bash has: sleep 10 ... curl ... I'm expecting access-hook to spawn new process and return without waiting for it to finish to let the service to do its job. But when i do push - it sleeps for 10 seconds anyway. Am i missing something obvious here? Any help is much appreciated! Thanks, Eugene I found a following solution to make it happen while waiting for somebody to be generous enough to take on the --post-service-hook (unfortunately i'm not a C guy): It is using 'at' command. The access-hook script has: echo delayed-notify.bash $@ | at now while delayed-notify.bash has: sleep 10 curl ... This is not perfect and in certain situations can still fail because the delay is not long enough but this will at least resolve 90% of issues. I hope that might be helpful for someone. Thanks, Eugene -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Fwd: git-daemon access-hook race condition
Hi, We are serving repos in closed netwrok via git protocol. We are using git-daemon access hook (thank you very much for such a great feature) in order to create push notifications for Jenkins. I.e. upon the push the access-hook is called and then the curl command is created and executed. As we have several instances of Jenkins, that we need to notify (three), the execution of the access-hook can take some time. Sometimes we have a situation when the whole chain works fine but Jenkins git plugin doesn't recognize the changes. I think it happens because we hit a kind of race condition: 1. Incoming push triggers access-hook 2. notify jenkins 1 3. notify jenkins 2 4. jenkins 1 polls repo but sees no changes 5. notify Jenkins 3 6. the push data transfer finishes - consequent pushes will find changes w/o any problem The question is: Is there a way to avoid that? Is it possible to have access-hook to be executed after receive? Is it possible to introduce a parameter that would specify if it needs to be executed before receive or after? Thanks, Eugene -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Fwd: git-daemon access-hook race condition
Eugene Sajine eugu...@gmail.com writes: Is it possible to have access-hook to be executed after receive? The whole point of access-hook is to allow it to decide whether the access is allowed or not, so that is a non-starter. A notification _after_ successful push update is usually done via the post-receive hook in the receiving repository, I think. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Fwd: Fwd: git-daemon access-hook race condition
On Thu, Sep 12, 2013 at 3:15 PM, Junio C Hamano gits...@pobox.com wrote: Eugene Sajine eugu...@gmail.com writes: Is it possible to have access-hook to be executed after receive? The whole point of access-hook is to allow it to decide whether the access is allowed or not, so that is a non-starter. A notification _after_ successful push update is usually done via the post-receive hook in the receiving repository, I think. Junio, Thanks for the reply! This is interesting: i always thought about the access-hook as something to be executed when the repo is accessed, not just verification if access is allowed - your definition is much more limiting. we have about 1400 bare repos - so i would like to avoid the configuration of each one of them. I could probably find a way to automate it, but already having access-hook in current implementation makes me reluctant to go this way, because it is so much easier to use centralized manager. So are you really sure that it is a non-starter to have --before-service/--after-service options for access-hook? Thanks, Eugene -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Fwd: Fwd: git-daemon access-hook race condition
Eugene Sajine eugu...@gmail.com writes: So are you really sure that it is a non-starter to have --before-service/--after-service options for access-hook? Given the definition of --access-hook in git help daemon: --access-hook=path:: Every time a client connects, first run an external command specified by the path ... The external command can decide to decline the service by exiting with a non-zero status (or to allow it by exiting with a zero status) There is *NO* way in anywhere --after-service makes any sense (and by definition --before-service is redundant). What you _could_ propose is to define a *new* hook that is run when the spawned service has returned, with the same information that is fed to the access hook (possibly with its exit status). I do not offhand know if we retain the original service information that long after the main daemon process has spawned the service process, though. With the current system, the only thing it needs to know is the PID of the service processes that are to be culled by calls to waitpid(). So you may have to extend existing bookkeeping data structures a bit to keep those pieces of information around if you wanted to add such a new hook. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Fwd: Fwd: git-daemon access-hook race condition
Junio, Thanks for the clarification! Your solution does look better. For now though i think i will have to delay the notification somehow and let the service finish first then notify the server. Thanks again! Eugene On Thu, Sep 12, 2013 at 5:08 PM, Junio C Hamano gits...@pobox.com wrote: Eugene Sajine eugu...@gmail.com writes: So are you really sure that it is a non-starter to have --before-service/--after-service options for access-hook? Given the definition of --access-hook in git help daemon: --access-hook=path:: Every time a client connects, first run an external command specified by the path ... The external command can decide to decline the service by exiting with a non-zero status (or to allow it by exiting with a zero status) There is *NO* way in anywhere --after-service makes any sense (and by definition --before-service is redundant). What you _could_ propose is to define a *new* hook that is run when the spawned service has returned, with the same information that is fed to the access hook (possibly with its exit status). I do not offhand know if we retain the original service information that long after the main daemon process has spawned the service process, though. With the current system, the only thing it needs to know is the PID of the service processes that are to be culled by calls to waitpid(). So you may have to extend existing bookkeeping data structures a bit to keep those pieces of information around if you wanted to add such a new hook. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Fwd: Fwd: git-daemon access-hook race condition
On Thu, Sep 12, 2013 at 5:16 PM, Eugene Sajine eugu...@gmail.com wrote: Junio, Thanks for the clarification! Your solution does look better. For now though i think i will have to delay the notification somehow and let the service finish first then notify the server. Thanks again! Eugene On Thu, Sep 12, 2013 at 5:08 PM, Junio C Hamano gits...@pobox.com wrote: Eugene Sajine eugu...@gmail.com writes: So are you really sure that it is a non-starter to have --before-service/--after-service options for access-hook? Given the definition of --access-hook in git help daemon: --access-hook=path:: Every time a client connects, first run an external command specified by the path ... The external command can decide to decline the service by exiting with a non-zero status (or to allow it by exiting with a zero status) There is *NO* way in anywhere --after-service makes any sense (and by definition --before-service is redundant). What you _could_ propose is to define a *new* hook that is run when the spawned service has returned, with the same information that is fed to the access hook (possibly with its exit status). I do not offhand know if we retain the original service information that long after the main daemon process has spawned the service process, though. With the current system, the only thing it needs to know is the PID of the service processes that are to be culled by calls to waitpid(). So you may have to extend existing bookkeeping data structures a bit to keep those pieces of information around if you wanted to add such a new hook. For now I'm trying to do the following: access-hook.bash has: delayed-notify.bash $@ delayed-notify.bash has: sleep 10 ... curl ... I'm expecting access-hook to spawn new process and return without waiting for it to finish to let the service to do its job. But when i do push - it sleeps for 10 seconds anyway. Am i missing something obvious here? Any help is much appreciated! Thanks, Eugene -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Fwd: Fwd: git-daemon access-hook race condition
Junio C Hamano gits...@pobox.com writes: Eugene Sajine eugu...@gmail.com writes: So are you really sure that it is a non-starter to have --before-service/--after-service options for access-hook? Given the definition of --access-hook in git help daemon: --access-hook=path:: Every time a client connects, first run an external command specified by the path ... The external command can decide to decline the service by exiting with a non-zero status (or to allow it by exiting with a zero status) There is *NO* way in anywhere --after-service makes any sense (and by definition --before-service is redundant). What you _could_ propose is to define a *new* hook that is run when the spawned service has returned, with the same information that is fed to the access hook (possibly with its exit status). Scratch that exit status part, as I do not think it is useful. To a receive-pack and a send-pack that is talking to it, if a push results in a failure, it is a failure. Likewise for upload-pack and fetch-pack for a transfer in the reverse direction. And the way that failure is communicated from the receive-pack to the end-user via the send-pack is for the receive-pack to send a protocol message that tells the send-pack about the failure, and the send-pack showing the error message and signalling the failure with its exit status. Likewise for upload-pack and fetch-pack (hence fetch, which is conceptually a thin wrapper around it). Between the deamon and the receive-pack (or the fetch-pack) process, however, such a failed push (or fetch) is still a success. I correctly diagnosed and successfully sent a rejection notice to the other end is signalled by receive-pack to the daemon by exiting with success (i.e. 0) exit status. So even if we feed the exit status of the service process to the hook script specified by the --post-service-hook, it does not tell the script if the service succeeded in that sense. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Fwd: Fwd: git-daemon access-hook race condition
On Thu, Sep 12, 2013 at 6:20 PM, Junio C Hamano gits...@pobox.com wrote: Junio C Hamano gits...@pobox.com writes: Eugene Sajine eugu...@gmail.com writes: So are you really sure that it is a non-starter to have --before-service/--after-service options for access-hook? Given the definition of --access-hook in git help daemon: --access-hook=path:: Every time a client connects, first run an external command specified by the path ... The external command can decide to decline the service by exiting with a non-zero status (or to allow it by exiting with a zero status) There is *NO* way in anywhere --after-service makes any sense (and by definition --before-service is redundant). What you _could_ propose is to define a *new* hook that is run when the spawned service has returned, with the same information that is fed to the access hook (possibly with its exit status). Scratch that exit status part, as I do not think it is useful. To a receive-pack and a send-pack that is talking to it, if a push results in a failure, it is a failure. Likewise for upload-pack and fetch-pack for a transfer in the reverse direction. And the way that failure is communicated from the receive-pack to the end-user via the send-pack is for the receive-pack to send a protocol message that tells the send-pack about the failure, and the send-pack showing the error message and signalling the failure with its exit status. Likewise for upload-pack and fetch-pack (hence fetch, which is conceptually a thin wrapper around it). Between the deamon and the receive-pack (or the fetch-pack) process, however, such a failed push (or fetch) is still a success. I correctly diagnosed and successfully sent a rejection notice to the other end is signalled by receive-pack to the daemon by exiting with success (i.e. 0) exit status. So even if we feed the exit status of the service process to the hook script specified by the --post-service-hook, it does not tell the script if the service succeeded in that sense. I see what you're saying. In my particular use case I can work around that service status because even if it failed it will just trigger Jenkins to poll and in case of failure to transfer data there will be no new changes for Jenkins to work with. If we would want the --post-service-hook to know that data transfer succeeded or failed, then may be there should be some difference between service status and service process status? In this case the existing logic works with service process status while the --post-service-hook is fed with the service status (or name it data transfer status) Do i make any sense? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Fwd: Fwd: git-daemon access-hook race condition
Eugene Sajine eugu...@gmail.com writes: So even if we feed the exit status of the service process to the hook script specified by the --post-service-hook, it does not tell the script if the service succeeded in that sense. I see what you're saying. In my particular use case I can work around that service status because even if it failed it will just trigger Jenkins to poll and in case of failure to transfer data there will be no new changes for Jenkins to work with. If we would want the --post-service-hook to know that data transfer succeeded or failed, then may be there should be some difference between service status and service process status? In this case the existing logic works with service process status while the --post-service-hook is fed with the service status (or name it data transfer status) Do i make any sense? Almost; you missed that there is no channel to pass data transfer status from the service back to the daemon. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
