Re: [PATCH 0/2] thin-pack capability for send-pack/receive-pack
On Wed, 2013-11-06 at 15:42 -0800, Shawn Pearce wrote: On Wed, Nov 6, 2013 at 1:41 PM, Carlos Martín Nieto c...@elego.de wrote: On Wed, 2013-11-06 at 12:32 -0800, Junio C Hamano wrote: I'll queue these for now, but I doubt the wisdom of this series, given that the ship has already sailed long time ago. Currently, no third-party implementation of a receiving end can accept thin push, because thin push is not a capability that needs to be checked by the current clients. People will have to wait until the clients with 2/2 patch are widely deployed before starting to use such a receiving end that is incapable of thin push. Wouldn't the world be a better place if instead they used that time waiting to help such a third-party receiving end to implement thin push support? Support in the code isn't always enough. The particular case that brought this on is one where the index-pack implementation can deal with thin packs just fine. This particular service takes the pack which the client sent and does post-processing on it to store it elsewhere. During the receive-pack equivalent, there is no git object db that it can query for the missing base objects. I realise this is pretty a unusual situation. How... odd? At Google we have made effort to ensure servers can accept thin packs, even though its clearly easier to accept non-thin, because clients in the wild already send thin packs and changing the deployed clients is harder than implementing the existing protocol. It is harder, but IMO also more correct, as thin packs are an optimisation that was added somewhat later. Not to say it shouldn't be something you should attempt to do, but it's a trade-off between the complexity of the communication between the pieces and the potential amount of extra data you're willing to put up with. The Google (Code) servers don't just support thin packs, but for upload-pack, they force it upon the client, which is quite frustrating as it won't even tell you why it closes the connection but sends a 500 instead, but that's a different story. If the server can't complete the pack, I guess this also means the client cannot immediately fetch from the server it just pushed to? Not all the details have been worked out yet, but the new history should be converted into the target format before reporting success and closing the connection. The Git frontend/protocol is one way of putting data into the system, but that's not its native data storage format. The database where this is getting stored only has very limited knowledge of git. I'll reroll the series with no-thin as mentioned elsewhere in this thread. cmn -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/2] thin-pack capability for send-pack/receive-pack
Hi all, This comes as a result of the discussion starting at [0] about git-push assuming that a server will always support thin packs. Most out there in fact do, but this isn't necessarily the case. Some implementations may not have support for it yet, or the server might be running in an environment where it is not feasible for it to try to fill in the missing objects. Jonathan and Duy mentioned that separate patches for receive-pack and send-pack could let us work around adding this at such a late stage, so here they are. The second patch can maybe lie in waiting for a while. [0] http://thread.gmane.org/gmane.comp.version-control.git/235766/focus=236402 Carlos Martín Nieto (2): receive-pack: advertise thin-pack send-pack: only send a thin pack if the server supports it Documentation/technical/protocol-capabilities.txt | 20 +++- builtin/receive-pack.c| 2 +- send-pack.c | 2 ++ 3 files changed, 18 insertions(+), 6 deletions(-) -- 1.8.4.652.g0d6e0ce -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/2] thin-pack capability for send-pack/receive-pack
Carlos Martín Nieto c...@elego.de writes: Hi all, This comes as a result of the discussion starting at [0] about git-push assuming that a server will always support thin packs. Most out there in fact do, but this isn't necessarily the case. Some implementations may not have support for it yet, or the server might be running in an environment where it is not feasible for it to try to fill in the missing objects. Jonathan and Duy mentioned that separate patches for receive-pack and send-pack could let us work around adding this at such a late stage, so here they are. The second patch can maybe lie in waiting for a while. I'll queue these for now, but I doubt the wisdom of this series, given that the ship has already sailed long time ago. Currently, no third-party implementation of a receiving end can accept thin push, because thin push is not a capability that needs to be checked by the current clients. People will have to wait until the clients with 2/2 patch are widely deployed before starting to use such a receiving end that is incapable of thin push. Wouldn't the world be a better place if instead they used that time waiting to help such a third-party receiving end to implement thin push support? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/2] thin-pack capability for send-pack/receive-pack
On Wed, 2013-11-06 at 12:32 -0800, Junio C Hamano wrote: I'll queue these for now, but I doubt the wisdom of this series, given that the ship has already sailed long time ago. Currently, no third-party implementation of a receiving end can accept thin push, because thin push is not a capability that needs to be checked by the current clients. People will have to wait until the clients with 2/2 patch are widely deployed before starting to use such a receiving end that is incapable of thin push. Wouldn't the world be a better place if instead they used that time waiting to help such a third-party receiving end to implement thin push support? Support in the code isn't always enough. The particular case that brought this on is one where the index-pack implementation can deal with thin packs just fine. This particular service takes the pack which the client sent and does post-processing on it to store it elsewhere. During the receive-pack equivalent, there is no git object db that it can query for the missing base objects. I realise this is pretty a unusual situation. Cheers, cmn -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/2] thin-pack capability for send-pack/receive-pack
Carlos Martín Nieto c...@elego.de writes: On Wed, 2013-11-06 at 12:32 -0800, Junio C Hamano wrote: I'll queue these for now, but I doubt the wisdom of this series, given that the ship has already sailed long time ago. Currently, no third-party implementation of a receiving end can accept thin push, because thin push is not a capability that needs to be checked by the current clients. People will have to wait until the clients with 2/2 patch are widely deployed before starting to use such a receiving end that is incapable of thin push. Wouldn't the world be a better place if instead they used that time waiting to help such a third-party receiving end to implement thin push support? Support in the code isn't always enough. The particular case that brought this on is one where the index-pack implementation can deal with thin packs just fine. This particular service takes the pack which the client sent and does post-processing on it to store it elsewhere. During the receive-pack equivalent, there is no git object db that it can query for the missing base objects. I realise this is pretty a unusual situation. OK, I agree that it sounds quite niche-y, but it still is sensible. If a receiving end does not want to (this includes it is incapable of doing so, but does not have to be limited to) complete a thin pack, the series will give it such an option in the longer term. Thanks. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/2] thin-pack capability for send-pack/receive-pack
On Wed, Nov 06, 2013 at 02:25:50PM -0800, Junio C Hamano wrote: Support in the code isn't always enough. The particular case that brought this on is one where the index-pack implementation can deal with thin packs just fine. This particular service takes the pack which the client sent and does post-processing on it to store it elsewhere. During the receive-pack equivalent, there is no git object db that it can query for the missing base objects. I realise this is pretty a unusual situation. OK, I agree that it sounds quite niche-y, but it still is sensible. If a receiving end does not want to (this includes it is incapable of doing so, but does not have to be limited to) complete a thin pack, the series will give it such an option in the longer term. I wonder if we want to make the flag go in the opposite direction, then. Right now we have no flag, and we assume the other side can handle a thin pack. If we add a thin flag, then the timeline is roughly: 1. Receive-pack starts advertising thin. 2. Send-pack cannot assume lack of thin means the other side cannot handle thin (it might just be an older receive-pack), and keeps sending thin packs. [time passes] 3. Send-pack can safely assume that every server has learned thin and can assume that lack of thin means the server does not want a thin pack. In other words, the benefit happens at step 3, and we do not get any effect until some long assumption time passes. If we instead introduced no-thin, it is more like: 1. Receive-pack starts advertising no-thin (as dictated by circumstances, as Carlos describes). 2. Send-pack which does not understand no-thin will ignore it and send a thin pack. This is the same as now, and the same as step 2 above. 3. An upgraded send-pack will understand no-thin and do as the server asks. So an upgraded client and server can start cooperating immediately, and we do not have to wait for the long assumption time to pass before applying the second half. It is tempting to think about a thin flag because that would be the natural way to have implemented it from the very beginning. But it is not the beginning, and the negative flag is the only way at this point to say if you understand this, please behave differently than we used to (because the status quo is send a thin pack, whether I said it was OK or not). -Peff -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/2] thin-pack capability for send-pack/receive-pack
On Wed, Nov 6, 2013 at 1:41 PM, Carlos Martín Nieto c...@elego.de wrote: On Wed, 2013-11-06 at 12:32 -0800, Junio C Hamano wrote: I'll queue these for now, but I doubt the wisdom of this series, given that the ship has already sailed long time ago. Currently, no third-party implementation of a receiving end can accept thin push, because thin push is not a capability that needs to be checked by the current clients. People will have to wait until the clients with 2/2 patch are widely deployed before starting to use such a receiving end that is incapable of thin push. Wouldn't the world be a better place if instead they used that time waiting to help such a third-party receiving end to implement thin push support? Support in the code isn't always enough. The particular case that brought this on is one where the index-pack implementation can deal with thin packs just fine. This particular service takes the pack which the client sent and does post-processing on it to store it elsewhere. During the receive-pack equivalent, there is no git object db that it can query for the missing base objects. I realise this is pretty a unusual situation. How... odd? At Google we have made effort to ensure servers can accept thin packs, even though its clearly easier to accept non-thin, because clients in the wild already send thin packs and changing the deployed clients is harder than implementing the existing protocol. If the server can't complete the pack, I guess this also means the client cannot immediately fetch from the server it just pushed to? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/2] thin-pack capability for send-pack/receive-pack
On Wed, Nov 6, 2013 at 2:54 PM, Jeff King p...@peff.net wrote: If we instead introduced no-thin, it is more like: 1. Receive-pack starts advertising no-thin (as dictated by circumstances, as Carlos describes). 2. Send-pack which does not understand no-thin will ignore it and send a thin pack. This is the same as now, and the same as step 2 above. 3. An upgraded send-pack will understand no-thin and do as the server asks. So an upgraded client and server can start cooperating immediately, and we do not have to wait for the long assumption time to pass before applying the second half. It is tempting to think about a thin flag because that would be the natural way to have implemented it from the very beginning. But it is not the beginning, and the negative flag is the only way at this point to say if you understand this, please behave differently than we used to (because the status quo is send a thin pack, whether I said it was OK or not). I think the only sane option at this point is a no-thin flag, or just require servers that want to be wire compatible to accept thin packs. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
