Re: [9fans] A simple experiment
Doesn't make it past my Vonage modem/router. It would otherwise probably make it past the Cable modem. On Thu, Apr 29, 2010 at 12:03 PM, erik quanstrom quans...@quanstro.net wrote: if you pick a random protocol type that's not tcp or udp and the chances your little packets will make it across the intertubes will be lowered quite a bit. this isn't the ip designer's fault. although for giggles i just set this up: minooka; aux/listen1 -tv /net.alt/il!*!4000 /386/bin/cat listen started incoming call for /net.alt/il!*!4000 from 74.166.29.253 in /net.alt/il/1 - erik
Re: [9fans] A simple experiment
On Fri, 30 Apr 2010 12:13:59 EDT erik quanstrom quans...@labs.coraid.com wrote: i don't see any reason why 9p couldn't use some of the same congestion control ideas. the trick would be to feed back packet loss detection and retransmission info to the point where file io gets turned into rpcsâthe mount driver Agreed -- sort of what I meant by I hope 9p evolves. Though I'd probably stick this in a layer below 9p. i suggested putting it above 9p. i don't understand your suggestion. how would a layer below know that there is more data? I was just musing aloud -- just some vague ideas. 9p already allows 2^16 outstanding messages so in theory one can turn a read() or write() into a sequence of 9p messsages and only wait for acks at the end (but process any acks as they arrive). Perhaps ack receipt time can be used to measure latency bandwidth to transparently control how many 9p messages can be blasted. The other vague thought was that one can do this: char buf[SOME_LARGE_CONSTANT]; int count = read(src, buf, sizeof buf); if (count 0) write(dst, buf, count); But read will wait until buf is filled up or file ends write will wait until the entire buf is sent or there is an error. Now may be this is good enough in that a reading thread can read in big gulps and pass on filled buffers to a writing thread. And the writing thread writes out and sends empty buffers back to the reader. But one can also think of a lower level facility that in effect offload actual transfer to a coprocessor. This allows read()/write() to return much quicker (but obviously some errors will be delayed). One can implement something like this using mmap(). Now I am well aware of issues with mmap (we don't have to go through that discussion again) but it can be an option for improving throughput. Neither of these ideas requires changes to 9p per se. To take this any further, I would want to prototype something. I have coded some bits in Scheme but no time really to do much. In turn some examples to explain your suggestion would be helpful. It can backpressure a 9p client if it tries to send too much (either by blocking or returning with equiv of EWOULDBLOCK). isn't that what you just complained aboutâfcp having to do the kernel's work? Blocking would backpressure in that fcp wouldn't be able to send more data! fcp doesn't have to do anything more.
Re: [9fans] A simple experiment
erik said: why is using nat to make many hosts look like one a bad thing? I suspect the reaction is based on being forced to use it when you'd rather not, like many residential ISPs require. It's particularly upsetting when the CPE doesn't even have a globally routable address. How much is many?
Re: [9fans] A simple experiment
I suspect the reaction is based on being forced to use it when you'd rather not, like many residential ISPs require. It's particularly upsetting when the CPE doesn't even have a globally routable address. How much is many? Um, some? I don't have any sort of global count. I've had 7 broadband ISPs. 4 of the first 5 (I've stopped asking) wouldn't provide multiple IP addresses without moving to a business-class service, which is usually bundled with one or more things I didn't want (or want to pay for), like symmetric links, QoS terms, c. 2 of the 7 (in the first three; maybe this practice has finally been discredited?) didn't provide the CPE with globally-routable IPs. The sample size is small, but is distributed across two countries and three US states. I've also gotten ample confirmation from others about the first part. And, of course, this doesn't address things like hotels and wifi hotspots, where the mandated use of nat is so common as to just be assumed. PGP.sig Description: This is a digitally signed message part
Re: [9fans] A simple experiment
On Fri, 30 Apr 2010 01:01:59 EDT erik quanstrom quans...@quanstro.net wrote: If the sender-receiver pipe can hold N bytes and the sender is streaming (that is, keeping the pipe full), the sender *will* be ahead of the receiver by N bytes. So a *streaming* protcol has to allow it to be N bytes ahead. i don't think this is the normal definition. Even if you have one ack per message, in a sliding window of N messages (or bytes), the sender is allowed to get ahead of the receiver by upto N more messages (or bytes). Here I am not concerned about in-order delivery (though typically people assume this when they talk about sliding windows). a sliding window is just a particular implementation what's required is many messages in flight. I was using sliding window as a shorthand for allowing the sender to be N bytes or messages ahead without regard to sequencing but yes, the definition I used is not the generally accepted defn so I will use yours! i don't see any reason why 9p couldn't use some of the same congestion control ideas. the trick would be to feed back packet loss detection and retransmission info to the point where file io gets turned into rpcsâthe mount driver Agreed -- sort of what I meant by I hope 9p evolves. Though I'd probably stick this in a layer below 9p. It can backpressure a 9p client if it tries to send too much (either by blocking or returning with equiv of EWOULDBLOCK).
Re: [9fans] A simple experiment
i don't see any reason why 9p couldn't use some of the same congestion control ideas. the trick would be to feed back packet loss detection and retransmission info to the point where file io gets turned into rpcs→the mount driver Agreed -- sort of what I meant by I hope 9p evolves. Though I'd probably stick this in a layer below 9p. i suggested putting it above 9p. i don't understand your suggestion. how would a layer below know that there is more data? It can backpressure a 9p client if it tries to send too much (either by blocking or returning with equiv of EWOULDBLOCK). isn't that what you just complained about—fcp having to do the kernel's work? i think EWOULDBLOCK would be a big mistake. io in plan 9 is synchronous. and this is very much on purpose. - erik
Re: [9fans] A simple experiment
On 28 April 2010 19:42, ron minnich rminn...@gmail.com wrote: We did a simple experiment recently: added a new 9p type called Tstream, because this issue of streams vs. transactions has been bugging me for years. The semantics are simple: it's a lot like Tread (almost same packet) but a single Tstream results in any number of Rstreams, until you hit no more data (/dev/mouse) or maybe EOF (/usr/rminnich/movie). Andrey tossed a sample implementation into newsham's 9p library. We saw a 27x improvement in performance from calgary to sandia for a big file. Fcp did not come close. what happens if the consumer is slow and the Rstream writer blocks? how do you stop all the other replies on the connection waiting for the consumer to get on with it? in fact, how do you stop it deadlocking if the consumer is in fact waiting for one of those replies? i suppose this comes down to what the API looks like. isochronous might be easier, as a slow reader is a bad reader so you could just throw away some data.
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 5:40 AM, roger peppe rogpe...@gmail.com wrote: On 28 April 2010 19:42, ron minnich rminn...@gmail.com wrote: We did a simple experiment recently: added a new 9p type called Tstream, because this issue of streams vs. transactions has been bugging me for years. The semantics are simple: it's a lot like Tread (almost same packet) but a single Tstream results in any number of Rstreams, until you hit no more data (/dev/mouse) or maybe EOF (/usr/rminnich/movie). Andrey tossed a sample implementation into newsham's 9p library. We saw a 27x improvement in performance from calgary to sandia for a big file. Fcp did not come close. what happens if the consumer is slow and the Rstream writer blocks? how do you stop all the other replies on the connection waiting for the consumer to get on with it? in fact, how do you stop it deadlocking if the consumer is in fact waiting for one of those replies? i suppose this comes down to what the API looks like. isochronous might be easier, as a slow reader is a bad reader so you could just throw away some data. It sounds ok on the surface so far. Does RStream signal the end of the stream chunks, or does the TStreamer already know that answer? Is there any sort of realtime constraint for handling incoming RStream chunks? I would think this could be ok, even if it forces the client to put the streamed blocks somewhere handy while processing is going on. Streaming audio and video over plan 9 links this way might be nice. But then I start to wonder why we feel we want to compete with HTTP when it already works, and is still fairly simple. Nothing wrong with improving 9P I suppose, but what's so wrong with HTTP transfers that it warrants changing our beloved resource sharing protocol? Maybe I'm being too practical, and not feeling adventurous or something :-) Dave
Re: [9fans] A simple experiment
On Wed, Apr 28, 2010 at 3:51 PM, ron minnich rminn...@gmail.com wrote: On Wed, Apr 28, 2010 at 1:36 PM, Francisco J Ballesteros n...@lsub.org wrote: That's similar to a Tget in op with unlimited replies. The difference adds on quickly. neat, I need to study op more than I did :-) For the record, I kept telling you that. -eric
Re: [9fans] A simple experiment
But then I start to wonder why we feel we want to compete with HTTP when it already works, and is still fairly simple. Nothing wrong with improving 9P I suppose, but what's so wrong with HTTP transfers that it warrants changing our beloved resource sharing protocol? Maybe I'm being too practical, and not feeling adventurous or something :-) do we put a http bag on the side of every /n/remoteslowlink fileserver, since 9p can't take care of it's own business? - erik
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 7:22 AM, erik quanstrom quans...@quanstro.netwrote: what happens if the consumer is slow and the Rstream writer blocks? how do you stop all the other replies on the connection waiting for the consumer to get on with it? the tcp window closes. and the producer blocks. in fact, how do you stop it deadlocking if the consumer is in fact waiting for one of those replies? i don't think i understand this. i think Tstream would be very dependant on the transport layer. that's the part that makes me nervous. all the world's a tcp? What does 9P require to function? If TStream has the same or lesser requirements, then there's no problem right? This comes back to my wondering why we don't just use 9P to set up HTTP streams. - erik
Re: [9fans] A simple experiment
What does 9P require to function? If TStream has the same or lesser requirements, then there's no problem right? This comes back to my wondering why we don't just use 9P to set up HTTP streams. see /sys/src/doc/il/il.ps - reliable, - in order. (the other three are not hard requirements) i would think that Tstream has greater requirements. it would seem to require flow control. - erik
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 7:43 AM, erik quanstrom quans...@quanstro.netwrote: What does 9P require to function? If TStream has the same or lesser requirements, then there's no problem right? This comes back to my wondering why we don't just use 9P to set up HTTP streams. see /sys/src/doc/il/il.ps - reliable, - in order. (the other three are not hard requirements) i would think that Tstream has greater requirements. it would seem to require flow control. - erik 9P doesn't require any flow control? That doesn't seem right :-) But then again it doesn't stream, at least in the traditional way I think of streaming. To stream you typically need flow control, so 9P isn't good for streaming in the sense I think of streaming. (yet?) Fix 9P or don't is the decision to be made.
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 7:35 AM, erik quanstrom quans...@quanstro.netwrote: But then I start to wonder why we feel we want to compete with HTTP when it already works, and is still fairly simple. Nothing wrong with improving 9P I suppose, but what's so wrong with HTTP transfers that it warrants changing our beloved resource sharing protocol? Maybe I'm being too practical, and not feeling adventurous or something :-) do we put a http bag on the side of every /n/remoteslowlink fileserver, since 9p can't take care of it's own business? - erik It's not quite fair to say 9P *can't* take care of it. The question is does it take care of it well enough?. I guess my point is we can change 9P to suit every purpose under the sun if we want, but then does it still have all the nice simple, properties of 9P? Wasn't IL somewhat abandoned because to make it as good as TCP you basically had to implement TCP anyway? I'm totally undecided on this, but just asking the question to make sure everyone feels good about adding streaming to 9P, which seems to indicate a need to add flow control as well. Dave
Re: [9fans] A simple experiment
Hello how well it works with firewalls, address translation, deep inspection, etc.? never tried il outside home. . . gabi - Original Message From: erik quanstrom quans...@quanstro.net To: 9fans@9fans.net Sent: Thu, April 29, 2010 4:43:48 PM Subject: Re: [9fans] A simple experiment What does 9P require to function? If TStream has the same or lesser requirements, then there's no problem right? This comes back to my wondering why we don't just use 9P to set up HTTP streams. see /sys/src/doc/il/il.ps - reliable, - in order. (the other three are not hard requirements) i would think that Tstream has greater requirements. it would seem to require flow control. - erik
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 8:03 AM, David Leimbach leim...@gmail.com wrote: 9P doesn't require any flow control? That doesn't seem right :-) But then again it doesn't stream, at least in the traditional way I think of streaming. To stream you typically need flow control, so 9P isn't good for streaming in the sense I think of streaming. (yet?) Fix 9P or don't is the decision to be made. it's going to get fixed. ron
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 5:40 AM, roger peppe rogpe...@gmail.com wrote: what happens if the consumer is slow and the Rstream writer blocks? how do you stop all the other replies on the connection waiting for the consumer to get on with it? there are not replies -- the rstream is a reply. If the consumer is slow packets are lost. That's life. in fact, how do you stop it deadlocking if the consumer is in fact waiting for one of those replies? ?? One Tstream. Lots of Rstreams. Zero interaction. i suppose this comes down to what the API looks like. isochronous might be easier, as a slow reader is a bad reader so you could just throw away some data. Yes. ron
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 8:06 AM, David Leimbach leim...@gmail.com wrote: I'm totally undecided on this, but just asking the question to make sure everyone feels good about adding streaming to 9P, which seems to indicate a need to add flow control as well. the problem of multiplexing several streams onto one stream, with priority and avoiding head of line blocking, was solved a long time ago. Tstream is unlikely to be added to 9p. ron
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 10:16 AM, ron minnich rminn...@gmail.com wrote: On Thu, Apr 29, 2010 at 8:03 AM, David Leimbach leim...@gmail.com wrote: 9P doesn't require any flow control? That doesn't seem right :-) But then again it doesn't stream, at least in the traditional way I think of streaming. To stream you typically need flow control, so 9P isn't good for streaming in the sense I think of streaming. (yet?) Fix 9P or don't is the decision to be made. it's going to get fixed. Well, its being fixed. Whether or not it gets back-ported to Plan 9 is the question :) -eric
Re: [9fans] A simple experiment
Wasn't IL somewhat abandoned because to make it as good as TCP you basically had to implement TCP anyway? due to a failure of vision, the internet only does well with certain types of ip packets. il is still an excellent protocol for local networks. 9P doesn't require any flow control? That doesn't seem right :-) But then 9p, like aoe, is a ping-pong protocol. each message requires an ack. therefore, the transport layer doesn't need flow control. - erik
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 8:34 AM, erik quanstrom quans...@quanstro.netwrote: Wasn't IL somewhat abandoned because to make it as good as TCP you basically had to implement TCP anyway? due to a failure of vision, the internet only does well with certain types of ip packets. il is still an excellent protocol for local networks. 9P doesn't require any flow control? That doesn't seem right :-) But then 9p, like aoe, is a ping-pong protocol. each message requires an ack. therefore, the transport layer doesn't need flow control. Right, but when you have one T message for many R messages, doesn't that change things a bit? - erik
Re: [9fans] A simple experiment
9p, like aoe, is a ping-pong protocol. each message requires an ack. therefore, the transport layer doesn't need flow control. Right, but when you have one T message for many R messages, doesn't that change things a bit? yes. if you were to add that, then a flow-controlled transport would be required. - erik
Re: [9fans] A simple experiment
On Thu, 29 Apr 2010 11:34:44 EDT erik quanstrom quans...@quanstro.net wrote: Wasn't IL somewhat abandoned because to make it as good as TCP you basically had to implement TCP anyway? due to a failure of vision, the internet only does well with certain types of ip packets. :-) il is still an excellent protocol for local networks. 9P doesn't require any flow control? That doesn't seem right :-) But then 9p, like aoe, is a ping-pong protocol. each message requires an ack. therefore, the transport layer doesn't need flow control. Therefore, it is also not able to utilise bandwidth effectively over longhaul links. As an example, US coast to coast round trip time latency is about 100ms. Now consider fcp. Each worker thread of fcp does 8K read/writes. Due to pingponging, the *most* a thread can xfer coast to coast is 80KBps (for 16 threads, 1.28MBps). It is actually much worse since each thread doesn't even overlap reads with writes. Short of a sliding window that is as large as the capacity of the pipe, you can't expect to keep it full. As usual one has to trade off simplicity vs performance. I do hope 9p evolves.
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 10:08 AM, Bakul Shah bakul+pl...@bitblocks.com wrote: Short of a sliding window that is as large as the capacity of the pipe, you can't expect to keep it full. As usual one has to trade off simplicity vs performance. I do hope 9p evolves. The problem revealed by fcp is worse than that. You're rewriting a simple program (cp) to cover for a lack of capability in the kernel. There's no reason that I can't do a read(1048576) and have the kernel turn that into as many reads as possible -- in this case, for 8k, 2^7 reads. Oh, and, it is likely that when we issue that many or more reads, we want a flow controlled network. Just guessing :-) ron
Re: [9fans] A simple experiment
9p, like aoe, is a ping-pong protocol. each message requires an ack. therefore, the transport layer doesn't need flow control. Therefore, it is also not able to utilise bandwidth effectively over longhaul links. As an example, US coast to coast round trip time latency is about 100ms. Now consider fcp. Each worker thread of fcp does 8K read/writes. Due to pingponging, the *most* a thread can xfer coast to coast is 80KBps (for 16 threads, 1.28MBps). It is actually much worse since each thread doesn't even overlap reads with writes. i think you are conflating a ping-pong protocol with the limitation of a single outstanding message. there is no reason that one needs a 1:1 correspondence between threads and outstanding messages. nor does the protocol inherently prohibit 1 write from becoming multiple messages. in the case of aoe, you can have up to 2^32 - 1 outstanding packets at the same time to each target. thus the theoretical maximum would be 80 * (2^31 - 1) kbps per target. the aoe driver (in contrib) has a limit of 128 8k jumbo frames outstanding per target. the aoe driver has a fixed number of thread per target. Short of a sliding window that is as large as the capacity of the pipe, you can't expect to keep it full. As usual one has to trade off simplicity vs performance. i don't think so. the only thing a sliding window gives you is 1 ack per message sent. -erik
Re: [9fans] A simple experiment
Oh, and, it is likely that when we issue that many or more reads, we want a flow controlled network. Just guessing :-) the downside of course is that the set of suitable protocols is then greatly reduced. tftp, rudp, il, cec to name a few simple protocols (yes i've done 9p over cec) would then be unusable for 9p. and the minimum requirements for participating in a 9p converstation would be increased. maybe that's okay, but it's hard to give that up. one thing that 9p doesn't address (that aoe does) is the maximum number of outstanding messages. that can be a fairly effective means of flow control. - erik
Re: [9fans] A simple experiment
But then I start to wonder why we feel we want to compete with HTTP when it already works, and is still fairly simple. Nothing wrong with improving 9P I suppose, but what's so wrong with HTTP transfers that it warrants changing our beloved resource sharing protocol? Maybe I'm being too practical, and not feeling adventurous or something :-) See the op papers for their justification. Dave Tim Newsham | www.thenewsh.com/~newsham | thenewsh.blogspot.com
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 10:48 AM, Tim Newsham news...@lava.net wrote: But then I start to wonder why we feel we want to compete with HTTP when it already works, and is still fairly simple. Nothing wrong with improving 9P I suppose, but what's so wrong with HTTP transfers that it warrants changing our beloved resource sharing protocol? Maybe I'm being too practical, and not feeling adventurous or something :-) See the op papers for their justification. Will do! Dave Tim Newsham | www.thenewsh.com/~newsham | thenewsh.blogspot.com
Re: [9fans] A simple experiment
9P doesn't require any flow control? That doesn't seem right :-) But then again it doesn't stream, at least in the traditional way I think of streaming. To stream you typically need flow control, so 9P isn't good for streaming in the sense I think of streaming. (yet?) Fix 9P or don't is the decision to be made. it's going to get fixed. Well, its being fixed. Whether or not it gets back-ported to Plan 9 is the question :) Where is the fix happening? Maybe there are people around that are willing to help back-port :-) EBo --
Re: [9fans] A simple experiment
i think in almost all cases putting Op between 9P endpoints will solve the slowness problem of high rtt networks. But then I start to wonder why we feel we want to compete with HTTP when it already works, and is still fairly simple. Nothing wrong with improving 9P I suppose, but what's so wrong with HTTP transfers that it warrants changing our beloved resource sharing protocol? Maybe I'm being too practical, and not feeling adventurous or something :-) See the op papers for their justification. Will do! Dave Tim Newsham | www.thenewsh.com/~newsham | thenewsh.blogspot.com
Re: [9fans] A simple experiment
Hmmm is Op the same Op from Octopus? Just making sure. Dave On Thu, Apr 29, 2010 at 11:41 AM, Skip Tavakkolian 9...@9netics.com wrote: i think in almost all cases putting Op between 9P endpoints will solve the slowness problem of high rtt networks. But then I start to wonder why we feel we want to compete with HTTP when it already works, and is still fairly simple. Nothing wrong with improving 9P I suppose, but what's so wrong with HTTP transfers that it warrants changing our beloved resource sharing protocol? Maybe I'm being too practical, and not feeling adventurous or something :-) See the op papers for their justification. Will do! Dave Tim Newsham | www.thenewsh.com/~newsham | thenewsh.blogspot.com
Re: [9fans] A simple experiment
due to a failure of vision, the internet only does well with certain types of ip packets. Well now *there* is a sweeping statement about the state of the universe circa 1980. Care to elaborate a teensy bit?
Re: [9fans] A simple experiment
On Thu Apr 29 14:53:56 EDT 2010, lyn...@orthanc.ca wrote: due to a failure of vision, the internet only does well with certain types of ip packets. Well now *there* is a sweeping statement about the state of the universe circa 1980. Care to elaborate a teensy bit? if you pick a random protocol type that's not tcp or udp and the chances your little packets will make it across the intertubes will be lowered quite a bit. this isn't the ip designer's fault. although for giggles i just set this up: minooka; aux/listen1 -tv /net.alt/il!*!4000 /386/bin/cat listen started incoming call for /net.alt/il!*!4000 from 74.166.29.253 in /net.alt/il/1 and surprise, surprise, i can call out through a bridge and a telco router and touch it ; telnet il!minooka.coraid.com!4000 connected to il!minooka.coraid.com!4000 on /net/il/3 hello, world hello, worldcr i'll leave that up for a while. have fun. i'll be interested what the success rate will be. - erik
Re: [9fans] A simple experiment
But then I start to wonder why we feel we want to compete with HTTP when it already works, and is still fairly simple. http is by no means simple, although yes, it could be still more complicated.
Re: [9fans] A simple experiment
due to a failure of vision, the internet only does well with certain types of ip packets. Well now *there* is a sweeping statement about the state of the universe circa 1980. Care to elaborate a teensy bit? i think the point is IL v. TCP; this is dejavu all over again: http://groups.google.com/group/comp.os.plan9/browse_thread/thread/23484eb901f3d8e7/d261b5a523bf53d1?lnk=gstq=why+IL#d261b5a523bf53d1
Re: [9fans] A simple experiment
Hmmm is Op the same Op from Octopus? yes. nemo has put the op specific stuff here: /n/sources/contrib/nemo/octopus/op.src.tgz
Re: [9fans] A simple experiment
fyi: from rangboom.com network (colo at a ISP): cpu% telnet il!minooka.coraid.com!4000 connected to il!minooka.coraid.com!4000 on /net/il/3 from 9netics.com (centurytel DSL): still waiting :) On Thu Apr 29 14:53:56 EDT 2010, lyn...@orthanc.ca wrote: due to a failure of vision, the internet only does well with certain types of ip packets. Well now *there* is a sweeping statement about the state of the universe circa 1980. Care to elaborate a teensy bit? if you pick a random protocol type that's not tcp or udp and the chances your little packets will make it across the intertubes will be lowered quite a bit. this isn't the ip designer's fault. although for giggles i just set this up: minooka; aux/listen1 -tv /net.alt/il!*!4000 /386/bin/cat listen started incoming call for /net.alt/il!*!4000 from 74.166.29.253 in /net.alt/il/1 and surprise, surprise, i can call out through a bridge and a telco router and touch it ; telnet il!minooka.coraid.com!4000 connected to il!minooka.coraid.com!4000 on /net/il/3 hello, world hello, worldcr i'll leave that up for a while. have fun. i'll be interested what the success rate will be. - erik
Re: [9fans] A simple experiment
On Thu Apr 29 15:55:44 EDT 2010, 9...@9netics.com wrote: fyi: from rangboom.com network (colo at a ISP): cpu% telnet il!minooka.coraid.com!4000 connected to il!minooka.coraid.com!4000 on /net/il/3 from 9netics.com (centurytel DSL): still waiting :) interesting. i was able to get there on both my residential dsl lines. (att negia.net) thanks for the il thread. does anyone know what happened to the nat code that eric grosse mentioned? - erik
Re: [9fans] A simple experiment
On Thu Apr 29 15:55:44 EDT 2010, 9...@9netics.com wrote: fyi: from rangboom.com network (colo at a ISP): cpu% telnet il!minooka.coraid.com!4000 connected to il!minooka.coraid.com!4000 on /net/il/3 from 9netics.com (centurytel DSL): still waiting :) interesting. i was able to get there on both my residential dsl lines. (att negia.net) thanks for the il thread. does anyone know what happened to the nat code that eric grosse mentioned? rather than fiddle with web server configuration, telneting to that port will now return a list of all connections. - erik
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 08:06, Gabriel Díaz gd...@rejaa.com wrote: Hello how well it works with firewalls, address translation, deep inspection, etc.? never tried il outside home. . . It doesn't play well with firewalls, NAT, or deep inspection because none of the vendors have added support for it. I tried to get Cisco to add IL support back in 2001, but they politely refused. -- Christopher Nielsen They who can give up essential liberty for temporary safety, deserve neither liberty nor safety. --Benjamin Franklin The tree of liberty must be refreshed from time to time with the blood of patriots tyrants. --Thomas Jefferson
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 12:44 PM, C H Forsyth fors...@vitanuova.com wrote: But then I start to wonder why we feel we want to compete with HTTP when it already works, and is still fairly simple. http is by no means simple, although yes, it could be still more complicated. I guess I think it's simple because of the crap I work with all day... everything's relative?
Re: [9fans] A simple experiment
There are a few dsl's which require you to use their routers with NAT and other stuff, but I've never tried these out, I've always had real IP connections here... As long as we have enough ipv4 adresses I would say: Build your own firewall, router or NAT... On 4/29/10, Christopher Nielsen cniel...@pobox.com wrote: On Thu, Apr 29, 2010 at 08:06, Gabriel Díaz gd...@rejaa.com wrote: Hello how well it works with firewalls, address translation, deep inspection, etc.? never tried il outside home. . . It doesn't play well with firewalls, NAT, or deep inspection because none of the vendors have added support for it. I tried to get Cisco to add IL support back in 2001, but they politely refused. -- Christopher Nielsen They who can give up essential liberty for temporary safety, deserve neither liberty nor safety. --Benjamin Franklin The tree of liberty must be refreshed from time to time with the blood of patriots tyrants. --Thomas Jefferson
Re: [9fans] A simple experiment
It doesn't play well with firewalls, NAT, or deep inspection because none of the vendors have added support for it. I tried to get Cisco to add IL support back in 2001, but they politely refused. oddly, i'm not having trouble going through 5 different nats over 2 residential and 2 business dsl lines. one of the connections traverses 2 nats before reaching the intertubes. - erik
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 13:40, erik quanstrom quans...@quanstro.net wrote: It doesn't play well with firewalls, NAT, or deep inspection because none of the vendors have added support for it. I tried to get Cisco to add IL support back in 2001, but they politely refused. oddly, i'm not having trouble going through 5 different nats over 2 residential and 2 business dsl lines. one of the connections traverses 2 nats before reaching the intertubes. Interesting. I've seen quite a few problems in the past that were all traced back to NAT or a firewall not handling IL properly. Admittedly, I haven't used IL recently, so maybe something has changed. It's good to hear that you're not having trouble. -- Christopher Nielsen They who can give up essential liberty for temporary safety, deserve neither liberty nor safety. --Benjamin Franklin The tree of liberty must be refreshed from time to time with the blood of patriots tyrants. --Thomas Jefferson
Re: [9fans] A simple experiment
On Thu, Apr 29, 2010 at 01:32:23PM -0700, Christopher Nielsen wrote: It doesn't play well with firewalls, NAT, or deep inspection because none of the vendors have added support for it. I tried to get Cisco to add IL support back in 2001, but they politely refused. Add support to what? Also what level of 'support'? IOS should already support IL in access lists simply by virtue of the fact that one can specify a numeric IP protocol. I agree that NAT and stateful firewalls (e.g. 'ip inspect' in IOS) would need explicit support to understand the packet layout. But one can always add exceptions to the firewall rules to allow IL through uninspected. Thats what I do on my IOS routers for oddball protocols. NAT - it should simply die, until then run IL over IPv6 and avoid NAT?
Re: [9fans] A simple experiment
I agree that NAT and stateful firewalls (e.g. 'ip inspect' in IOS) would need explicit support to understand the packet layout. what il services would you apply spi to? one doesn't ftp or http over il. NAT - it should simply die, until then run IL over IPv6 and avoid NAT? il isn't defined over ip6. why should nat die? translating network addresses from one network to another seems natural enough to me—and quite similar to what various storage systems do to present logical volumes. why should renumber a formerly private network because i'd like to hook it up to the internet? why should i renumber my network because i change service providers? why is using nat to make many hosts look like one a bad thing? - erik
Re: [9fans] A simple experiment
ron said: Oh, and, it is likely that when we issue that many or more reads, we want a flow controlled network. Just guessing :-) You'd want one, sure. But (unless I'm missing something) it doesn't seem that increasing the number of outstanding messages would *require* it (your performance would just suck otherwise). If you treat the number of outstanding requests as a tunable (per mount, per network type) you'd be able to use the same protocol over minimal links as on tcp. dave said: Wasn't IL somewhat abandoned because to make it as good as TCP you basically had to implement TCP anyway? My impression is that it was pulled from mainline mostly because it was seen as being too much work to maintain for a small group. It's also not really sensible to talk about as good as without defining your context. Over fast, local networks, IL is faster than TCP. It's not as generic, but that's not the same as worse. erik said: why is using nat to make many hosts look like one a bad thing? I suspect the reaction is based on being forced to use it when you'd rather not, like many residential ISPs require. It's particularly upsetting when the CPE doesn't even have a globally routable address. aside: i made the il connection. i think you've got either good luck or a good config on your end. it didn't want to ask dns for some reason. PGP.sig Description: This is a digitally signed message part
Re: [9fans] A simple experiment
On Thu, 29 Apr 2010 13:23:00 EDT erik quanstrom quans...@quanstro.net wrote: 9p, like aoe, is a ping-pong protocol. each message requires an ack. therefore, the transport layer doesn't need flow control. Therefore, it is also not able to utilise bandwidth effectively over longhaul links. As an example, US coast to coast round trip time latency is about 100ms. Now consider fcp. Each worker thread of fcp does 8K read/writes. Due to pingponging, the *most* a thread can xfer coast to coast is 80KBps (for 16 threads, 1.28MBps). It is actually much worse since each thread doesn't even overlap reads with writes. i think you are conflating a ping-pong protocol with the limitation of a single outstanding message. there is no reason that one needs a 1:1 correspondence between threads and outstanding messages. nor does the protocol inherently prohibit 1 write from becoming multiple messages. What I am saying is that RPC (which is how 9p is mostly used) has this inherent performance limitation due to latency. To get around that if you use multiple outstanding messages (regardless of one per thread or many per thread), you *will* require flow control or you run into problems -- imagine having thousands of outstanding messages. Or imagine having a low bandwidth link for fcp. Fcp will hog the link, to the detriment of other programs. So on the high end fcp throughput is limited 1.28MBps, on the low end it will hog the link completey (unless your kernel implements some sort of fair queuing)! If you want to make optimum use of the available bandwidth while adjusting to changing conditions and not competely clog up the pipe, you need a feedback mechanism and window opening algorithm ALA TCP or something. And anyway why do all this in fcp or every other program that needs streaming? It should be factored out. Short of a sliding window that is as large as the capacity of the pipe, you can't expect to keep it full. As usual one has to trade off simplicity vs performance. i don't think so. the only thing a sliding window gives you is 1 ack per message sent. If the sender-receiver pipe can hold N bytes and the sender is streaming (that is, keeping the pipe full), the sender *will* be ahead of the receiver by N bytes. So a *streaming* protcol has to allow it to be N bytes ahead. Even if you have one ack per message, in a sliding window of N messages (or bytes), the sender is allowed to get ahead of the receiver by upto N more messages (or bytes). Here I am not concerned about in-order delivery (though typically people assume this when they talk about sliding windows). If you assume in-order delivery you can coalesce multiple acks but that can be seen as an ack optimization (but then either you throw away out of order deliveries or have to add selective ack or something).
Re: [9fans] A simple experiment
My impression is that it was pulled from mainline mostly because it was seen as being too much work to maintain for a small group. It's also not really sensible to talk about as good as without defining your context. Over fast, local networks, IL is faster than TCP. It's not as generic, but that's not the same as worse. history -fD /n/sources/plan9/sys/src/9/ip/il.c reveals just 4 interesting diffs since 2002. why is using nat to make many hosts look like one a bad thing? I suspect the reaction is based on being forced to use it when you'd rather not, like many residential ISPs require. It's particularly upsetting when the CPE doesn't even have a globally routable address. that's like saying ferraris are bad because they can be driven into trees. sometimes you really want many hosts to look like one. for example, you may want a farm of http servers may hide behind a single ip address. this is a good thing for scalability. aside: i made the il connection. i think you've got either good luck or a good config on your end. it didn't want to ask dns for some reason. cs? il was removed from the protocols cs knows about. try cs from /n/sourcesdump/2006/0101/plan9/386/bin/cs. if that fixes it, i'll fix 9atom. - erik
Re: [9fans] A simple experiment
If the sender-receiver pipe can hold N bytes and the sender is streaming (that is, keeping the pipe full), the sender *will* be ahead of the receiver by N bytes. So a *streaming* protcol has to allow it to be N bytes ahead. i don't think this is the normal definition. Even if you have one ack per message, in a sliding window of N messages (or bytes), the sender is allowed to get ahead of the receiver by upto N more messages (or bytes). Here I am not concerned about in-order delivery (though typically people assume this when they talk about sliding windows). a sliding window is just a particular implementation what's required is many messages in flight. aoe for example, has a maximum outstanding (set by the target). with good client-side congestion control, tracking a window is not helpful. the client (sender) knows best. aoe has no sequence number; just a tag. the target (receiver) just echoes tags and is allowed no interpretation. they work like fids. i don't think this qualifies as a sliding window. yet the tcp congestion algorithm works well with this setup. multiple gbe links can be saturated and yet congestion is still handled. i don't see any reason why 9p couldn't use some of the same congestion control ideas. the trick would be to feed back packet loss detection and retransmission info to the point where file io gets turned into rpcs—the mount driver - erik
Re: [9fans] A simple experiment
Ok, I admit its a trivial experiment but: fcp is still a 9p conversation. http get is a tcp stream. Fcp is better than cp but not that much better. If you're yanking one file, a TCP stream is pretty ideal. Dropping 9p on top of it, even when the 9p involves multiple TREADs in flight, is just making things slower. larch% time cp /n/sources/extra/plan9.tar.bz2 /dev/null 0.02u 0.52s 647.90r cp /n/sources/extra/plan9.tar.bz2 /dev/null larch% time fcp /n/sources/extra/plan9.tar.bz2 /dev/null 0.01u 0.85s 49.69r fcp /n/sources/extra/plan9.tar.bz2 /dev/null larch% time hget http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 /dev/null 0.37u 0.54s 32.84r hget http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 Mmm, HTTP does give better performance, but its not that extreme, and for a nightly cron script I would not worry about it. I admit I am surprised by how much a difference there is, it should be just Tread and Rread headers shouldn't it? -Steve
Re: [9fans] A simple experiment
On 28 Apr 2010, at 12:51, Steve Simon wrote: Ok, I admit its a trivial experiment but: fcp is still a 9p conversation. http get is a tcp stream. Fcp is better than cp but not that much better. If you're yanking one file, a TCP stream is pretty ideal. Dropping 9p on top of it, even when the 9p involves multiple TREADs in flight, is just making things slower. larch% time cp /n/sources/extra/plan9.tar.bz2 /dev/null 0.02u 0.52s 647.90r cp /n/sources/extra/plan9.tar.bz2 /dev/null larch% time fcp /n/sources/extra/plan9.tar.bz2 /dev/null 0.01u 0.85s 49.69r fcp /n/sources/extra/plan9.tar.bz2 /dev/null larch% time hget http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 /dev/null 0.37u 0.54s 32.84r hget http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 Mmm, HTTP does give better performance, but its not that extreme, and for a nightly cron script I would not worry about it. I admit I am surprised by how much a difference there is, it should be just Tread and Rread headers shouldn't it? Could round-trip times be adding up? Does 9p do one file at once strictly? -Steve -- Simplicity does not precede complexity, but follows it. -- Alan Perlis
Re: [9fans] A simple experiment
larch% time cp /n/sources/extra/plan9.tar.bz2 /dev/null 0.02u 0.52s 647.90r cp /n/sources/extra/plan9.tar.bz2 /dev/null larch% time fcp /n/sources/extra/plan9.tar.bz2 /dev/null 0.01u 0.85s 49.69r fcp /n/sources/extra/plan9.tar.bz2 /dev/null larch% time hget http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 /dev/null 0.37u 0.54s 32.84r hget http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 Mmm, HTTP does give better performance, but its not that extreme, and for a nightly cron script I would not worry about it. I admit I am surprised by how much a difference there is, it should be just Tread and Rread headers shouldn't it? local weather conditions? i get a much tighter grouping and fcp is faster than http: minooka; time cp /n/sources/extra/plan9.tar.bz2 /dev/null 0.00u 0.18s 343.83r cp /n/sources/extra/plan9.tar.bz2 /dev/null minooka; time fcp /n/sources/extra/plan9.tar.bz2 /dev/null 0.00u 0.20s 54.64r fcp /n/sources/extra/plan9.tar.bz2 /dev/null minooka; time hget http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 /dev/null 0.20u 0.20s 55.05r hget http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 and for good measure: minooka; time hget http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 /n/other/quanstro/plan9.tar.bz2 0.17u 0.89s 55.37r hget http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 if i do a different test where the remote as 1mb of bandwidth (43ms ping): ladd; time cp /n/t1/n/other/quanstro/plan9.tar.bz2 /dev/null 0.00u 0.19s 302.93r cp /n/t1/n/other/quanstro/plan9.tar.bz2 /dev/null ladd; time fcp /n/t1/n/other/quanstro/plan9.tar.bz2 /dev/null 0.00u 0.25s 179.87r fcp /n/t1/n/other/quanstro/plan9.tar.bz2 /dev/null ladd; time hget http://t1.example.com/files/extra/plan9.tar.bz2 /dev/null 0.20u 0.24s 175.94r hget http://t1.example.com/files/extra/plan9.tar.bz2 the original test i did was looked funny because i'd used ssl for the import but regular http. so the system time was skewed i wonder if some of the really bad 9p performance could be related to aan. - erik
Re: [9fans] A simple experiment
Could round-trip times be adding up? Does 9p do one file at once strictly? i think you mean, does 9p have 1 outstanding message. it does not. 9p supports having as many outstanding messages as one wishes. if you do a pread(2), the kernel will only maintain a single outstanding message for you. fcp is a quick hack around this. by keeping n threads each doing a read, you can keep n messages outstanding. there is nothing (save complexity) preventing the kernel from keeping multiple outstanding 9p messages. this is the technique aoe uses to overcome drive latency, which — even for enterprise slc ssds — can be considerable in the worst case (~500ms). - erik
Re: [9fans] A simple experiment
On 28 Apr 2010, at 14:26, erik quanstrom wrote: Could round-trip times be adding up? Does 9p do one file at once strictly? i think you mean, does 9p have 1 outstanding message. it does not. 9p supports having as many outstanding messages as one wishes. if you do a pread(2), the kernel will only maintain a single outstanding message for you. fcp is a quick hack around this. by keeping n threads each doing a read, you can keep n messages outstanding. there is nothing (save complexity) preventing the kernel from keeping multiple outstanding 9p messages. this is the technique aoe uses to overcome drive latency, which — even for enterprise slc ssds — can be considerable in the worst case (~500ms). - erik Yeah, if I understand right that is what I meant. Good to know, but I'm back to the drawing board on why 9p is sometimes so very much slower than transferring the equivalent files in a tar file over http. Maybe fcp should be used with a large number of threads? :s -- Simplicity does not precede complexity, but follows it. -- Alan Perlis
Re: [9fans] A simple experiment
Yeah, if I understand right that is what I meant. Good to know, but I'm back to the drawing board on why 9p is sometimes so very much slower than transferring the equivalent files in a tar file over http. rtt. Maybe fcp should be used with a large number of threads? :s the only limitation of the fcp approach is that it is built on a reliable protocol, so it's impossible to control the number of inflight requests based on missed packets. perhaps the real key to a 9p2015 that solves this problem would be building it on unreliable transport. by the way, i was trying to generate aan numbers. but fcp just breaks aan. aan is currently transferring about 10bytes/sec. - erik
Re: [9fans] A simple experiment
when timing http it's also a good idea to check that your isp hasn't got the item cached. admittedly, that in itself is a further potential advantage to http, because http caches abound, whereas 9p-based caches are few and far between.
Re: [9fans] A simple experiment
I admit I am surprised by how much a difference there is, it should be just Tread and Rread headers shouldn't it? If you have high latency or high bandwidth, the maximum message size for the 9p messages will be too small to keep the pipe full if you're using read serially. Did you take a look at how much bandwith was actually in use during your tests? -Steve Tim Newsham | www.thenewsh.com/~newsham | thenewsh.blogspot.com
Re: [9fans] A simple experiment
My apologies, I didn't notice you had used fcp as well. Oops. On Wed, 28 Apr 2010, Tim Newsham wrote: I admit I am surprised by how much a difference there is, it should be just Tread and Rread headers shouldn't it? If you have high latency or high bandwidth, the maximum message size for the 9p messages will be too small to keep the pipe full if you're using read serially. Did you take a look at how much bandwith was actually in use during your tests? -Steve Tim Newsham | www.thenewsh.com/~newsham | thenewsh.blogspot.com Tim Newsham | www.thenewsh.com/~newsham | thenewsh.blogspot.com
Re: [9fans] A simple experiment
On Wed Apr 28 13:58:40 EDT 2010, news...@lava.net wrote: I admit I am surprised by how much a difference there is, it should be just Tread and Rread headers shouldn't it? If you have high latency or high bandwidth, the maximum message size for the 9p messages will be too small to keep the pipe full if you're using read serially. Did you take a look at how much bandwith was actually in use during your tests? unless you've got some sort of interrupt or cpu bottleneck, i don't see what message size has to do with this issue. the real issue is the # of messages outstanding. - erik
Re: [9fans] A simple experiment
On Wed, Apr 28, 2010 at 11:00 AM, erik quanstrom quans...@labs.coraid.com wrote: unless you've got some sort of interrupt or cpu bottleneck, i don't see what message size has to do with this issue. the real issue is the # of messages outstanding. Trivially, it always matters: what if msize is 1? There's always a balance between packet size and parallelism in the # ops you send (i.e. outstanding packets in flight) and other factors I forget right now :-) Too big messages cause heartbreak and bad behavior in some networks. Too many messages in flight cause issues at each end. And, in the end, if what you want is a stream, then discrete request/response transactions can be made to work, but they don't always work as well. We did a simple experiment recently: added a new 9p type called Tstream, because this issue of streams vs. transactions has been bugging me for years. The semantics are simple: it's a lot like Tread (almost same packet) but a single Tstream results in any number of Rstreams, until you hit no more data (/dev/mouse) or maybe EOF (/usr/rminnich/movie). Andrey tossed a sample implementation into newsham's 9p library. We saw a 27x improvement in performance from calgary to sandia for a big file. Fcp did not come close. There's work to be done; 9p is great, but it's not the end of protocols. ron
Re: [9fans] A simple experiment
And, in the end, if what you want is a stream, then discrete request/response transactions can be made to work, but they don't always work as well. streams are an illusion of the transport layer. - erik
Re: [9fans] A simple experiment
On Wed, Apr 28, 2010 at 12:06 PM, erik quanstrom quans...@labs.coraid.com wrote: And, in the end, if what you want is a stream, then discrete request/response transactions can be made to work, but they don't always work as well. streams are an illusion of the transport layer. packets, bytes, and bits are illusions too. russ
Re: [9fans] A simple experiment
We did a simple experiment recently: added a new 9p type called Tstream, because this issue of streams vs. transactions has been bugging me for years. The semantics are simple: it's a lot like Tread (almost same packet) but a single Tstream results in any number of Rstreams, until you hit no more data (/dev/mouse) or maybe EOF (/usr/rminnich/movie). Andrey tossed a sample implementation into newsham's 9p library. We saw a 27x improvement in performance from calgary to sandia for a big file. Fcp did not come close. That's similar to a Tget in op with unlimited replies. The difference adds on quickly.
Re: [9fans] A simple experiment
On Wed, Apr 28, 2010 at 1:36 PM, Francisco J Ballesteros n...@lsub.org wrote: That's similar to a Tget in op with unlimited replies. The difference adds on quickly. neat, I need to study op more than I did :-) ron
Re: [9fans] A simple experiment
Francisco J Ballesteros n...@lsub.org said: We did a simple experiment recently: added a new 9p type called Tstream, because this issue of streams vs. transactions has been bugging me for years. The semantics are simple: it's a lot like Tread (almost same packet) but a single Tstream results in any number of Rstreams, until you hit no more data (/dev/mouse) or maybe EOF (/usr/rminnich/movie). Andrey tossed a sample implementation into newsham's 9p library. We  saw a 27x improvement in performance from calgary to sandia for a big file. Fcp did not come close. That's similar to a Tget in op with unlimited replies. The difference adds on quickly. cool. Is Tstream integrated back into 9p and publicly available? Also, where can i find out more info about op and Tget? Thanks and best regards, EBo --
Re: [9fans] A simple experiment
On Wed, Apr 28, 2010 at 2:05 PM, EBo e...@sandien.com wrote: cool. Is Tstream integrated back into 9p and publicly available? no, it's an experiment at this point. there are other ideas out there, and I should have paid more attention to op as nemo points out :-) the main point was to see what was possible, given the well known poor performance of 9p on high latency links. This poor performance has an analog in HPC, where microseconds are large units of time. ron
Re: [9fans] A simple experiment
We did a simple experiment recently: added a new 9p type called Tstream, because this issue of streams vs. transactions has been bugging me for years. The semantics are simple: it's a lot like Tread (almost same packet) but a single Tstream results in any number of Rstreams, until you hit no more data (/dev/mouse) or maybe EOF (/usr/rminnich/movie). Andrey tossed a sample implementation into newsham's 9p library. We saw a 27x improvement in performance from calgary to sandia for a big file. Fcp did not come close. That's similar to a Tget in op with unlimited replies. The difference adds on quickly. i was going to ask if anyone has tried this with op. is there an implementation of oxport/ofs for plan 9?
Re: [9fans] A simple experiment
no, it's an experiment at this point. there are other ideas out there, and I should have paid more attention to op as nemo points out :-) understand. Don't you just HATE it when you come up with something cool and then find a different alternative stashed in the literature ;-) I look forward to learning more about both. the main point was to see what was possible, given the well known poor performance of 9p on high latency links. This poor performance has an analog in HPC, where microseconds are large units of time. That is why I got so interested. EBo --
Re: [9fans] A simple experiment
yes and no. Yes in that you can use inferno + op to bridge plan 9 systems. No in that there's no native implementation yet. On Wed, Apr 28, 2010 at 11:18 PM, Skip Tavakkolian 9...@9netics.com wrote: We did a simple experiment recently: added a new 9p type called Tstream, because this issue of streams vs. transactions has been bugging me for years. The semantics are simple: it's a lot like Tread (almost same packet) but a single Tstream results in any number of Rstreams, until you hit no more data (/dev/mouse) or maybe EOF (/usr/rminnich/movie). Andrey tossed a sample implementation into newsham's 9p library. We saw a 27x improvement in performance from calgary to sandia for a big file. Fcp did not come close. That's similar to a Tget in op with unlimited replies. The difference adds on quickly. i was going to ask if anyone has tried this with op. is there an implementation of oxport/ofs for plan 9?
Re: [9fans] A simple experiment
Nice work, but couldn't you just bind /n/sources/plan9/sys/src to the hg repo and push from there?
Re: [9fans] A simple experiment
More importantly, it's going to be easier for me to bisect and find problems when I build from kernel source, which is very handy in my case. The web interface of bitbucket gives me a pretty reasonable way to compare different revs. I'm offering this note in the event others want to use this interface and repo. is plan 9 really that complicated that bisect is a useful tool? perhaps i'm still in the dark ages, but i've done fine with diff(1) and history(1). perhaps the key difference is that i sync with sources by diffing and merging by hand. /n/sources/lsr is a great guide to Things That Have Changed. pull is a little too scary if you're responsible for keeping things running. - erik
Re: [9fans] A simple experiment
On Tue, Apr 27, 2010 at 1:54 PM, j...@9srv.net wrote: Nice work, but couldn't you just bind /n/sources/plan9/sys/src to the hg repo and push from there? That would almost certainly be slower than grabbing the ISO via HTTP and getting the file tree locally. John
Re: [9fans] A simple experiment
FWIW, I pull -n first, then see what´s going to change, then merge what I care about, then pull without -n, adding -s´s and -c´s as a result of which file I merged, and I´m done. Regarding pulls from sources I´ve found no problem so far (other than speed). Indeed, speed is not a problem here because we pull from a local mirror of replica (also updated via replica). Internally, I´ve switched from replica to repl (similar to replica, but a custom tool) only because it´s easier to sync trees that I know are in sync (e.g., because I copied files by hand). hth On Tue, Apr 27, 2010 at 7:49 PM, erik quanstrom quans...@quanstro.net wrote: pull is a little too scary if you're responsible for keeping things running. - erik
Re: [9fans] A simple experiment
On Tue Apr 27 13:58:39 EDT 2010, slawmas...@gmail.com wrote: On Tue, Apr 27, 2010 at 1:54 PM, j...@9srv.net wrote: Nice work, but couldn't you just bind /n/sources/plan9/sys/src to the hg repo and push from there? That would almost certainly be slower than grabbing the ISO via HTTP and getting the file tree locally. it would be interesting to try. if hg can push in parallel, it could be competitive. fetching the iso, decompressing the iso, etc are not free. and you can't push anything until after step 2. talk about killer latency. - erik
Re: [9fans] A simple experiment
On Tue, Apr 27, 2010 at 1:59 PM, erik quanstrom quans...@quanstro.net wrote: On Tue Apr 27 13:58:39 EDT 2010, slawmas...@gmail.com wrote: On Tue, Apr 27, 2010 at 1:54 PM, j...@9srv.net wrote: Nice work, but couldn't you just bind /n/sources/plan9/sys/src to the hg repo and push from there? That would almost certainly be slower than grabbing the ISO via HTTP and getting the file tree locally. it would be interesting to try. if hg can push in parallel, it could be competitive. fetching the iso, decompressing the iso, etc are not free. and you can't push anything until after step 2. talk about killer latency. - erik My experiments have shown that copying a large file via HTTP is significantly faster than copying the same file via 9P. I haven't tested it, but I would wager that opening, reading, and closing hundreds of small files via 9P would also be much slower than grabbing the same files in a compressed archive via HTTP and uncompressing. Also, I'm not sure of the exact mechanics of hg, but I'm guessing that a commit + push would involve at least two traversals of the tree, which is not fun when you're hitting sources for every file op. John
Re: [9fans] A simple experiment
On Tue, Apr 27, 2010 at 2:35 PM, John Floren slawmas...@gmail.com wrote: On Tue, Apr 27, 2010 at 1:59 PM, erik quanstrom quans...@quanstro.net wrote: On Tue Apr 27 13:58:39 EDT 2010, slawmas...@gmail.com wrote: On Tue, Apr 27, 2010 at 1:54 PM, j...@9srv.net wrote: Nice work, but couldn't you just bind /n/sources/plan9/sys/src to the hg repo and push from there? That would almost certainly be slower than grabbing the ISO via HTTP and getting the file tree locally. it would be interesting to try. if hg can push in parallel, it could be competitive. fetching the iso, decompressing the iso, etc are not free. and you can't push anything until after step 2. talk about killer latency. - erik My experiments have shown that copying a large file via HTTP is significantly faster than copying the same file via 9P. I haven't tested it, but I would wager that opening, reading, and closing hundreds of small files via 9P would also be much slower than grabbing the same files in a compressed archive via HTTP and uncompressing. Also, I'm not sure of the exact mechanics of hg, but I'm guessing that a commit + push would involve at least two traversals of the tree, which is not fun when you're hitting sources for every file op. John Assuming hg acts like most other DVCS: The commit would happen locally to the `database' in the .hg directory. The sources tree would be read once to do the commit into blobs in .hg. Push would then only send stuff from that .hg directory. Most DVCS don't touch the working directory for push/pull operations, they simply sync up the .hg/.git/.foo directories. Merging and such are all done after that sync, locally.
Re: [9fans] A simple experiment
My experiments have shown that copying a large file via HTTP is significantly faster than copying the same file via 9P. were you using fcp? i'm curious as to where the differences could come from, since the usual suspects that can make the difference (establishing a connection, sequential reads, directory walks/stats) wont come into play for one file.
Re: [9fans] A simple experiment
looks like you got it going fetching http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 would improve your dowload speed as it only contains the source On Tue, Apr 27, 2010 at 2:38 PM, ron minnich rminn...@gmail.com wrote: I had interest in being able to see plan 9 source at bitbucket.org. Part of the driver was my continuing inability to get replica to work well at home, and part just a need to tinker :-) So, I created an empty repo at bitbucket.org, http://bitbucket.org/rminnich/sysfromiso/overview and then did the usual hg clone -e '/bin/openssh/ssh -2' ssh://h...@bitbucket.org/rminnich/sysfromiso At this point on Plan 9 I have a directory, sysfromiso, that is empty save for a .hg Now on linux or other systems, you copy a bunch of directories in there, hg add them, and away you go. Plan 9 is more interesting: hget http://plan9.bell-labs.com/plan9/download/plan9.iso.bz2/tmp/iso.bz2 rc -c 'cd /tmp; bunzip2 iso.bz2' 9660srv -f /tmp/iso iso mount /srv/iso /n/iso now I've got the sources over there in /n/iso. What's next? Simple: cd sysfromiso bind -a /n/iso . And then add some trees: hg add sys/src then hg commit hg push -e '/bin/openssh/ssh -2' And I've got a starting point. What's interesting is that the directory always looks empty until I do the bind: term% ls sysfromiso sysfromiso/.hg term% So the script to continue updating the repo is pretty simple: #!/bin/rc hget http://plan9.bell-labs.com/plan9/download/plan9.iso.bz2/tmp/iso.bz2 rc -c 'cd /tmp; bunzip2 iso.bz2' 9660srv -f /tmp/iso iso mount /srv/iso /n/iso ape/psh cd sysfromiso bind -b /n/iso . x=`date` hg commit -m $x hg push -e '/bin/openssh/ssh -2' (note I need ape/psh when I use ssh for pushes -- quoting rules issue) This can be run from cron -- once you get through the ssh issues I mentioned in the earlier note. Result is an hg repo on bitbucket.org that I can get to from anywhere, and I can watch as Geoff continues to beat on the kw port :-) More importantly, it's going to be easier for me to bisect and find problems when I build from kernel source, which is very handy in my case. The web interface of bitbucket gives me a pretty reasonable way to compare different revs. I'm offering this note in the event others want to use this interface and repo. ron -- Federico G. Benavento
Re: [9fans] A simple experiment
Federico G. Benavento benave...@gmail.com said: looks like you got it going fetching http://plan9.bell-labs.com/sources/extra/plan9.tar.bz2 would improve your dowload speed as it only contains the source On Tue, Apr 27, 2010 at 2:38 PM, ron minnich rminn...@gmail.com wrote: I had interest in being able to see plan 9 source at bitbucket.org. Part of the driver was my continuing inability to get replica to work well at home, and part just a need to tinker :-) So, I created an empty repo at bitbucket.org, http://bitbucket.org/rminnich/sysfromiso/overview and then did the usual hg clone -e '/bin/openssh/ssh -2' ssh://h...@bitbucket.org/rminnich/sysfromiso At this point on Plan 9 I have a directory, sysfromiso, that is empty save for a .hg Now on linux or other systems, you copy a bunch of directories in there, hg add them, and away you go. Plan 9 is more interesting: hget http://plan9.bell-labs.com/plan9/download/plan9.iso.bz2/tmp/iso.bz2  rc -c 'cd /tmp; bunzip2 iso.bz2'  9660srv -f /tmp/iso iso mount /srv/iso /n/iso now I've got the sources over there in /n/iso. What's next? Simple: cd sysfromiso bind -a /n/iso . And then add some trees: hg add sys/src then hg commit hg push -e '/bin/openssh/ssh -2' And I've got a starting point. What's interesting is that the directory always looks empty until I do the bind: term% ls sysfromiso sysfromiso/.hg term% So the script to continue updating the repo is pretty simple:  #!/bin/rc hget http://plan9.bell-labs.com/plan9/download/plan9.iso.bz2/tmp/iso.bz2  rc -c 'cd /tmp; bunzip2 iso.bz2'  9660srv -f /tmp/iso iso mount /srv/iso /n/iso ape/psh cd sysfromiso bind -b /n/iso . x=`date` hg commit -m $x hg push -e  '/bin/openssh/ssh -2' (note I need ape/psh when I use ssh for pushes -- quoting rules issue) This can be run from cron -- once you get through the ssh issues I mentioned in the earlier note. Result is an hg repo on bitbucket.org that I can get to from anywhere, and I can watch as Geoff continues to beat on the kw port :-) More importantly, it's going to be easier for me to bisect and find problems when I build from kernel source, which is very handy in my case. The web interface of bitbucket gives me a pretty reasonable way to compare different revs. I'm offering this note in the event others want to use this interface and repo. ron -- Federico G. Benavento --
Re: [9fans] A simple experiment
On Tue, Apr 27, 2010 at 10:49 AM, erik quanstrom quans...@quanstro.net wrote: is plan 9 really that complicated that bisect is a useful tool? perhaps i'm still in the dark ages, but i've done fine with diff(1) and history(1). Not good enough for me. perhaps the key difference is that i sync with sources by diffing and merging by hand. /n/sources/lsr is a great guide to Things That Have Changed. pull is a little too scary if you're responsible for keeping things running. I find that something like hg is just way easier to deal with than the standard plan 9 tools. ron
Re: [9fans] A simple experiment
On Tue, Apr 27, 2010 at 10:59 AM, erik quanstrom quans...@quanstro.net wrote: it would be interesting to try. if hg can push in parallel, it could be competitive. fetching the iso, decompressing the iso, etc are not free. and you can't push anything until after step 2. talk about killer latency. pulling and decompressing the iso are far, far faster for me than replica. There's no comparison. ron
Re: [9fans] A simple experiment
On Tue, Apr 27, 2010 at 12:03 PM, Skip Tavakkolian 9...@9netics.com wrote: My experiments have shown that copying a large file via HTTP is significantly faster than copying the same file via 9P. were you using fcp? i'm curious as to where the differences could come from, since the usual suspects that can make the difference (establishing a connection, sequential reads, directory walks/stats) wont come into play for one file. fcp is still a 9p conversation. http get is a tcp stream. Fcp is better than cp but not that much better. If you're yanking one file, a TCP stream is pretty ideal. Dropping 9p on top of it, even when the 9p involves multiple TREADs in flight, is just making things slower. ron
Re: [9fans] A simple experiment
is plan 9 really that complicated that bisect is a useful tool? perhaps i'm still in the dark ages, but i've done fine with diff(1) and history(1). Not good enough for me. why? - erik
