Thanks Jim, That really clears things up for me.
So but you're saying that Keith's idea of having the outgoing datagram's source IP remapped is not possible? I totally agree about the traditional approach of the "bastion host", it's basically like having SSH act as the "load balancer" - as I'll have some shell script open up a connection to a random server on the bastion's CLI. The only problem there is that it doesn't take advantage of traditional cluster architecture so well. For instance I'll need to come up with a custom method of keeping a live list of the available internal IP addresses to which I can forward the session. Any idea how many connections an average 512MB VM could handle like this? I suspect so many that you'd only ever need one VM? Tom On 13 July 2018 at 12:54, Jim Cheetham <jim.cheet...@otago.ac.nz> wrote: > Excerpts from Thomas Buckley-Houston's message of July 13, 2018 3:58 pm: >> So >> because UDP is a connectionless protocol, every datagram has to >> contain the source IP in order for the receiver to send responses? > > Every packet does have the source IP address in it, but not because it's a > UDP packet, it's because that's what all *IP* packets have. UDP is a protocol > built on top of IP, as is TCP - we're used to the "full name" of TCP being > TCP/IP, but it's unusual to say "UDP/IP" :-) > >> It's not enough for either end to assume an IP based on the *initial* >> datagram's IP? > > Because UDP is connectionless, there's no concept of an initial packet in the > first place; all packets that arrive are completely self-contained entities > with no relation to any other packets that might arrive before or after. It's > a bit like HTTP in that respect. > > The *application* that's using UDP may have a concept of a long-term session > that involved multiple packets, but UDP itself doesn't. Again looking at > HTTP, that's like using cookies to describe a session. > >> So based on that assumption, the best way for a >> datagram to get its IP:port is to query the OS. Which in our scenario >> is a privately networked container and so will be unreachable from the >> outside. > > You have a middle layer load balancer, which will be keeping track of the > state of sessions going through; this is easy for TCP because the session > identifiers are basically in every packet. However, in UDP they are not > present at all - you have to be aware of the actual application using UDP; > you have to be explicitly aware of mosh itself in order to be able to > load-balance/proxy it correctly. > > A simple assumption for most protocols that want to have cheap lightweight > sessions is to assume that the source IP address stays the same throughout. > This happens to have been true for most of the network's history, but has > never been guaranteed. > > In the case of mosh, this is explicitly not true; the protocol *expects* that > the source IP address will change unannounced throughout the application's > session. > > It is this issue that you need to be looking in to. You need a proxy/LB layer > that looks at a UDP packet coming through, recognises that it is using SSP > (State Synchronization Protocol), and keeps track of which backend system the > packet should be forwarded to. > > Unfortunately, this will be very difficult; datagrams are encrypted, and your > proxy will not be aware of the key in use. The only way I can see to make > this happen will be to alter the mosh server to inform the proxy directly, > and that's a lot of engineering that will introduce a number of security > tradeoffs that might not be worthwhile. > > Overall, your better approach is to have a traditional "bastion host" in the > middle, use mosh to connect to that, and from there make inbound ssh > connections to your service hosts (because you can get away with ssh on an > internal "reliable" network, and because you're using TCP for this, you can > go through a load balancer if required). > > -jim > > -- > Jim Cheetham, Information Security, University of Otago, Dunedin, N.Z. > ✉ jim.cheet...@otago.ac.nz ☏ +64 3 470 4670 ☏ m +64 21 279 4670 > ⚷ OpenPGP: B50F BE3B D49B 3A8A 9CC3 8966 9374 82CD C982 0605 _______________________________________________ mosh-devel mailing list mosh-devel@mit.edu http://mailman.mit.edu/mailman/listinfo/mosh-devel
