Re: support git+mosh for unreliable connections
On Wed, 15 Apr 2015 21:25:44 +, Dennis Kaarsemaker wrote: ... It does not and cannot work. The way mosh works, is that it uses ssh to log in and launch a mosh-server daemon. This daemon and the mosh client then communicate via a custom UDP protocol. The SSH connection is closed after the mosh-server has been launched as it is no longer needed. The communication between the mosh client and server synchronizes terminal state, somewhat like what screen/tmux do. I object to the 'can not' part a bit. There is (1) the terminal state prediction and (2) the reliable-over-reconnects communication, and for a noninteractive usage you'd need only (2). Once upon a time I implemented a simple UDP server and client; the client to be used as a ProxyCommand in ssh, and the server just talks to the local ssh server. This pretty much does what the OP wants, and it works just as a transport for ssh, so all ssh features are there (but of course there is no terminal prediction). Unfortunately it needs to be ported to libev/libuv before it could be released. It's *much* simpler than mosh, although the use-ssh-to-start-server trick would be nice.) Andreas -- Totally trivial. Famous last words. From: Linus Torvalds torvalds@*.org Date: Fri, 22 Jan 2010 07:29:21 -0800 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: support git+mosh for unreliable connections
On Thursday 16 April 2015 01:56 AM, Ilari Liusvaara wrote: On Wed, Apr 15, 2015 at 08:13:51PM +0530, Pirate Praveen wrote: Q: Are the mosh principles relevant to other network applications? We think so. The design principles that Mosh stands for are conservative: warning the user if the state being displayed is out of date, serializing and checkpointing all transactions so that if there are no warnings, the user knows every prior transaction has succeeded, and handling expected events (like roaming from one WiFi network to another) gracefully. Can the ideas be used to resume a pull, push or clone operation? Especially serializing and checkpointing. Well, it is possible to write a remote helper and serverside program that internally handles connection unreliability, so Git itself (upload-archive, upload-pack, receive-pack, archive, fetch-pack and send-pack) sees a reliable (full-duplex, half-closeable, stream) channel. Suitably done, that can resume (from Git POV, nothing special happened) across things like IP address changes. However, that is quite difficult to do in practice. Not because interface to Git is complicated, but because the transport problem itself is complicated (however, it still seems way easier than making Git internally be able to resume interrupted operations). Mosh needs to solve at least most of that, it just doesn't provode the right kind of interface. I have requested mosh team to fix these issues https://github.com/keithw/mosh/issues/597 signature.asc Description: OpenPGP digital signature
Re: support git+mosh for unreliable connections
Trevor Saunders venit, vidit, dixit 15.04.2015 20:59: On Wed, Apr 15, 2015 at 07:46:15PM +0200, Johannes Schindelin wrote: Hi Trevor, On 2015-04-15 17:33, Trevor Saunders wrote: On Wed, Apr 15, 2015 at 04:41:42PM +0200, Johannes Schindelin wrote: On 2015-04-15 16:18, Pirate Praveen wrote: On Wednesday 15 April 2015 07:22 PM, Michael J Gruber wrote: What would that require git to do, beyond taking whatever you tell it (using GIT_SSH or _GIT_SSH_COMMAND) to use as a drop in replacement for ssh? May be support git+mosh as a protocol, since it is not a drop in replacement. It is redesigned remote shell. The ideas it uses for session resumption needs to be reimplemented. This will need support from git, because it needs server side to be modified. Use SSP to return the the current progress for a particular session (it uses AES session ids). It will need support from Git alright, but not as much as from mosh, see my other reply: Mosh was not designed for non-interactive use. That support would have to be added before we can go any further. is that really relevent? mosh doesn't support things like X forwarding or port forwarding, but it certainly does support ssh host command and then doing IO. Ah, so mosh's README lied to me! I wouldn't say it lied, its just not really clear what is interactive I'd say git's use of ssh is kind of interactive compared to things like port forwarding. If `mosh user@host command` works, then a simple `GIT_SSH=mosh` should work out of the box, too. Have you tried it? it does work, I just tried mosh $host cat and then typing stuff and having it printed back at me. However it clears the terminal before hand and prints a message on exit. I tried GIT_SSH=mosh git clone and it failed, but I haven't really dug into why. SO I suspect this can be made to work with some work on the mosh side, but I'm not sure exactly how ssh and mosh are behaving differently here. Trev First thing you see on mosh.mit.edu: Mosh is a replacement for SSH. I guess that needs a footnote... Michael -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: support git+mosh for unreliable connections
On wo, 2015-04-15 at 18:37 +0530, Pirate Praveen wrote: Hi, When working with big projects over a slow, unreliable connection, currently there is no way to resume a clone or pull when the connection breaks. mosh is a better replacement for ssh over unreliable connections. supporting git+mosh protocol will go a long way in supporting people who work with unreliable, mobile networks, especially in developed countries (I personally have to try many times when working with large projects as my 3g mobile connection keeps dropping. I recently discovered mosh and it works like a charm. More about mosh https://mosh.mit.edu/ mosh isn't a generic transport though, it's a udp-based session state synchronization protocol. I don't think it can be used as a git transport. -- Dennis Kaarsemaker www.kaarsemaker.net -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: support git+mosh for unreliable connections
On Wednesday 15 April 2015 07:22 PM, Michael J Gruber wrote: What would that require git to do, beyond taking whatever you tell it (using GIT_SSH or _GIT_SSH_COMMAND) to use as a drop in replacement for ssh? Michael May be support git+mosh as a protocol, since it is not a drop in replacement. It is redesigned remote shell. The ideas it uses for session resumption needs to be reimplemented. This will need support from git, because it needs server side to be modified. Use SSP to return the the current progress for a particular session (it uses AES session ids). So when a client connect with a session id, git server side can respond with the current state, how many objects received in that session, and client can continue from where it stopped. Client also will need to store session information. signature.asc Description: OpenPGP digital signature
Re: support git+mosh for unreliable connections
Pirate Praveen venit, vidit, dixit 15.04.2015 15:07: Hi, When working with big projects over a slow, unreliable connection, currently there is no way to resume a clone or pull when the connection breaks. mosh is a better replacement for ssh over unreliable connections. supporting git+mosh protocol will go a long way in supporting people who work with unreliable, mobile networks, especially in developed countries (I personally have to try many times when working with large projects as my 3g mobile connection keeps dropping. I recently discovered mosh and it works like a charm. More about mosh https://mosh.mit.edu/ What would that require git to do, beyond taking whatever you tell it (using GIT_SSH or _GIT_SSH_COMMAND) to use as a drop in replacement for ssh? Michael -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: support git+mosh for unreliable connections
Hi Praveen, On 2015-04-15 16:18, Pirate Praveen wrote: On Wednesday 15 April 2015 07:22 PM, Michael J Gruber wrote: What would that require git to do, beyond taking whatever you tell it (using GIT_SSH or _GIT_SSH_COMMAND) to use as a drop in replacement for ssh? May be support git+mosh as a protocol, since it is not a drop in replacement. It is redesigned remote shell. The ideas it uses for session resumption needs to be reimplemented. This will need support from git, because it needs server side to be modified. Use SSP to return the the current progress for a particular session (it uses AES session ids). It will need support from Git alright, but not as much as from mosh, see my other reply: Mosh was not designed for non-interactive use. That support would have to be added before we can go any further. So when a client connect with a session id, git server side can respond with the current state, how many objects received in that session, and client can continue from where it stopped. Client also will need to store session information. No, the protocol can stay exactly the same, once you have a way to communicate non-interactively via mosh. Ciao, Johannes -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: support git+mosh for unreliable connections
On Wed, Apr 15, 2015 at 04:41:42PM +0200, Johannes Schindelin wrote: Hi Praveen, On 2015-04-15 16:18, Pirate Praveen wrote: On Wednesday 15 April 2015 07:22 PM, Michael J Gruber wrote: What would that require git to do, beyond taking whatever you tell it (using GIT_SSH or _GIT_SSH_COMMAND) to use as a drop in replacement for ssh? May be support git+mosh as a protocol, since it is not a drop in replacement. It is redesigned remote shell. The ideas it uses for session resumption needs to be reimplemented. This will need support from git, because it needs server side to be modified. Use SSP to return the the current progress for a particular session (it uses AES session ids). It will need support from Git alright, but not as much as from mosh, see my other reply: Mosh was not designed for non-interactive use. That support would have to be added before we can go any further. is that really relevent? mosh doesn't support things like X forwarding or port forwarding, but it certainly does support ssh host command and then doing IO. It might not support not doing terminal emulation stuff, but that seems like a simple thing to change in principal at which point I think it would support enough of ssh's functionality its a drop in replacement as far as git is concerned. Seems to me mosh is close enough on its own its worth experimentation by someone who cares. Trev So when a client connect with a session id, git server side can respond with the current state, how many objects received in that session, and client can continue from where it stopped. Client also will need to store session information. No, the protocol can stay exactly the same, once you have a way to communicate non-interactively via mosh. Ciao, Johannes -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: support git+mosh for unreliable connections
Hi Praveen A, On 2015-04-15 15:07, Pirate Praveen wrote: When working with big projects over a slow, unreliable connection, currently there is no way to resume a clone or pull when the connection breaks. mosh is a better replacement for ssh over unreliable connections. supporting git+mosh protocol will go a long way in supporting people who work with unreliable, mobile networks, especially in developed countries (I personally have to try many times when working with large projects as my 3g mobile connection keeps dropping. I recently discovered mosh and it works like a charm. More about mosh https://mosh.mit.edu/ From https://github.com/keithw/mosh: Mosh does not support X forwarding or the non-interactive uses of SSH, including port forwarding. In particular it does not support [...] the non-interactive uses of SSH, which the git+mosh transport would require, though. That means that you would have to invest quite a bit of effort into enhancing mosh to *support* the non-interactive uses of SSH before you could start implementing `git-remote-mosh`... Ciao, Johannes -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: support git+mosh for unreliable connections
On Wednesday 15 April 2015 07:52 PM, Johannes Schindelin wrote: From https://github.com/keithw/mosh: Mosh does not support X forwarding or the non-interactive uses of SSH, including port forwarding. In particular it does not support [...] the non-interactive uses of SSH, which the git+mosh transport would require, though. That means that you would have to invest quite a bit of effort into enhancing mosh to *support* the non-interactive uses of SSH before you could start implementing `git-remote-mosh`... Ciao, Johannes Q: Are the mosh principles relevant to other network applications? We think so. The design principles that Mosh stands for are conservative: warning the user if the state being displayed is out of date, serializing and checkpointing all transactions so that if there are no warnings, the user knows every prior transaction has succeeded, and handling expected events (like roaming from one WiFi network to another) gracefully. Can the ideas be used to resume a pull, push or clone operation? Especially serializing and checkpointing. signature.asc Description: OpenPGP digital signature
Re: support git+mosh for unreliable connections
Hi Trevor, On 2015-04-15 17:33, Trevor Saunders wrote: On Wed, Apr 15, 2015 at 04:41:42PM +0200, Johannes Schindelin wrote: On 2015-04-15 16:18, Pirate Praveen wrote: On Wednesday 15 April 2015 07:22 PM, Michael J Gruber wrote: What would that require git to do, beyond taking whatever you tell it (using GIT_SSH or _GIT_SSH_COMMAND) to use as a drop in replacement for ssh? May be support git+mosh as a protocol, since it is not a drop in replacement. It is redesigned remote shell. The ideas it uses for session resumption needs to be reimplemented. This will need support from git, because it needs server side to be modified. Use SSP to return the the current progress for a particular session (it uses AES session ids). It will need support from Git alright, but not as much as from mosh, see my other reply: Mosh was not designed for non-interactive use. That support would have to be added before we can go any further. is that really relevent? mosh doesn't support things like X forwarding or port forwarding, but it certainly does support ssh host command and then doing IO. Ah, so mosh's README lied to me! If `mosh user@host command` works, then a simple `GIT_SSH=mosh` should work out of the box, too. Have you tried it? Ciao, Johannes It might not support not doing terminal emulation stuff, but that seems like a simple thing to change in principal at which point I think it would support enough of ssh's functionality its a drop in replacement as far as git is concerned. Seems to me mosh is close enough on its own its worth experimentation by someone who cares. Trev So when a client connect with a session id, git server side can respond with the current state, how many objects received in that session, and client can continue from where it stopped. Client also will need to store session information. No, the protocol can stay exactly the same, once you have a way to communicate non-interactively via mosh. Ciao, Johannes -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: support git+mosh for unreliable connections
On Wed, Apr 15, 2015 at 08:13:51PM +0530, Pirate Praveen wrote: Q: Are the mosh principles relevant to other network applications? We think so. The design principles that Mosh stands for are conservative: warning the user if the state being displayed is out of date, serializing and checkpointing all transactions so that if there are no warnings, the user knows every prior transaction has succeeded, and handling expected events (like roaming from one WiFi network to another) gracefully. Can the ideas be used to resume a pull, push or clone operation? Especially serializing and checkpointing. Well, it is possible to write a remote helper and serverside program that internally handles connection unreliability, so Git itself (upload-archive, upload-pack, receive-pack, archive, fetch-pack and send-pack) sees a reliable (full-duplex, half-closeable, stream) channel. Suitably done, that can resume (from Git POV, nothing special happened) across things like IP address changes. However, that is quite difficult to do in practice. Not because interface to Git is complicated, but because the transport problem itself is complicated (however, it still seems way easier than making Git internally be able to resume interrupted operations). Mosh needs to solve at least most of that, it just doesn't provode the right kind of interface. -Ilari -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: support git+mosh for unreliable connections
On Wed, Apr 15, 2015 at 07:46:15PM +0200, Johannes Schindelin wrote: Hi Trevor, On 2015-04-15 17:33, Trevor Saunders wrote: On Wed, Apr 15, 2015 at 04:41:42PM +0200, Johannes Schindelin wrote: On 2015-04-15 16:18, Pirate Praveen wrote: On Wednesday 15 April 2015 07:22 PM, Michael J Gruber wrote: What would that require git to do, beyond taking whatever you tell it (using GIT_SSH or _GIT_SSH_COMMAND) to use as a drop in replacement for ssh? May be support git+mosh as a protocol, since it is not a drop in replacement. It is redesigned remote shell. The ideas it uses for session resumption needs to be reimplemented. This will need support from git, because it needs server side to be modified. Use SSP to return the the current progress for a particular session (it uses AES session ids). It will need support from Git alright, but not as much as from mosh, see my other reply: Mosh was not designed for non-interactive use. That support would have to be added before we can go any further. is that really relevent? mosh doesn't support things like X forwarding or port forwarding, but it certainly does support ssh host command and then doing IO. Ah, so mosh's README lied to me! I wouldn't say it lied, its just not really clear what is interactive I'd say git's use of ssh is kind of interactive compared to things like port forwarding. If `mosh user@host command` works, then a simple `GIT_SSH=mosh` should work out of the box, too. Have you tried it? it does work, I just tried mosh $host cat and then typing stuff and having it printed back at me. However it clears the terminal before hand and prints a message on exit. I tried GIT_SSH=mosh git clone and it failed, but I haven't really dug into why. SO I suspect this can be made to work with some work on the mosh side, but I'm not sure exactly how ssh and mosh are behaving differently here. Trev Ciao, Johannes It might not support not doing terminal emulation stuff, but that seems like a simple thing to change in principal at which point I think it would support enough of ssh's functionality its a drop in replacement as far as git is concerned. Seems to me mosh is close enough on its own its worth experimentation by someone who cares. Trev So when a client connect with a session id, git server side can respond with the current state, how many objects received in that session, and client can continue from where it stopped. Client also will need to store session information. No, the protocol can stay exactly the same, once you have a way to communicate non-interactively via mosh. Ciao, Johannes -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: support git+mosh for unreliable connections
On wo, 2015-04-15 at 19:46 +0200, Johannes Schindelin wrote: On 2015-04-15 17:33, Trevor Saunders wrote: but it certainly does support ssh host command and then doing IO. Yes, in interactive sessions. mosh synchronizes terminal state, it doesn't allow random I/O between client and server. Ah, so mosh's README lied to me! If `mosh user@host command` works, then a simple `GIT_SSH=mosh` should work out of the box, too. Have you tried it? It does not and cannot work. The way mosh works, is that it uses ssh to log in and launch a mosh-server daemon. This daemon and the mosh client then communicate via a custom UDP protocol. The SSH connection is closed after the mosh-server has been launched as it is no longer needed. The communication between the mosh client and server synchronizes terminal state, somewhat like what screen/tmux do. -- Dennis Kaarsemaker www.kaarsemaker.net -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
