Hi!
On 21:16 Sun 28 Nov , Ian G wrote:
> Hi Michael,
>
> On 28/11/10 7:12 PM, Michael Blizek wrote:
>
> >>Only that it isn't used enough :)
> >>
> >>http://iang.org/ssl/reliable_connections_are_not.html
> >
> >I do not really get the point of this page.
>
>
> The dogma in most programming is that /if/ you need to reliably send
> some data in an application, /then/ you should use a reliable
> connection like TCP/IP.
>
> What this page is about is saying that the dogma is wrong. Using a
> reliable connection is approximately reliable, but if you really
> need real reliability, it just isn't up to the job.
>
> So you have to (and most apps do this) provide a reliability layer
> over the top anyway.
>
> Now, if you have to provide that reliability layer over the top ...
> anyway ... many times you're better off doing the job properly. And
> switching to UDP / datagrams from the very start.
TCP is only partly about reliability. It is also about connection establishing,
segmentation, flow control and other stuff. UDP is good if one side should
process requests stateless *and* requests/responses are so small that
segmentation and flow control are not needed. Creating sessions in the
application layer which are not bound to layer 4 connections is something I
try to avoid. You need a way to recreate a session either way, but by doing
this "binding" there is no need for "session cookies", as in HTTP. As soon as
the data you want to transfer is so big that it needs to be segmented, using
TCP will also spare you the pain of segmentation+reassembly and
retransmissions of chunks. You will hardly find any webpage which fits into a
single IP packet - I am afraid that sometimes not even the request does. IP
fragmentation performs very badly when there is some packet loss. Note that
there are other protocols on layer 4, like SCTP, which does connection
setup, congestion avoidance and segmentation, but you can decide to handle
lost packets for yourself, if you want. It is nice for VoIP and some other
real time applications, but it can be problematic with firewalls and NATs.
This is why IMHO the port number should have been on layer 3, not 4...
> It's basically about saying -- you know all those engineering
> problems you have in the upper layers, trying to make your app slick
> and reliable? A lot of them are to do with an original flawed
> assumption you made a long time ago -- to use TCP.
IMHO, reliable programs should reuse (good) existing code and try to be as
simple as possible.
> >1) You *can* send+receive raw IP packets and do your own TCP/IP
> >implementation
> >in userspace if you want. This is what sniffers and many "security" tools do.
> >I see no problem in modifying at least linux to do this even with non
> >privileged programs, if the operating system still does manages the ports.
> >But
> >seriously doubt that this will make TCP faster and less bug-prune. It will
> >very
> >likely do the exact opposite. After all, people who have no clue would start
> >reinventing segmentation, reordering, retransmit including acks and their
> >timing, windowing including SWS avoidance, congestion avoidance, path MTU
> >discovery, connection setup including syncookies and a lot of other stuff.
>
>
> Yes. The question is, what do you need? Many applications don't
> need all of what TCP offers.
This is why there are other transports, like SCTP and DCCP. If these are
unavailable in a given environment it might still be better to use TCP, if
the application needs anything it provides rather than build from scratch on
top of UDP.
> Think of it this way: how many applications are stream oriented?
> Almost none. HTTP is not; it's datagram oriented, a
> request-response model, GETs going out, HTML coming back.
As already said, HTTP documents are way to big to transfer over UDP.
> Even stream oriented apps like VoIP use lossy streams :)
This is due to real time optimisations and because people did not want to
build it on top of DCCP/SCTP at that time.
> >2) It is *impossible* for both the sender knowing the receiver has received
> >and processed the data *and* the receiver not starting to process the
> >received data before it knows the sender knows it has processed them. If you
> >have this problem, you provide the sender with a way to query the status any
> >time, independant of any session.
>
>
> Yes, factor in the coordination problem, a.k.a. two generals problem.
>
> The point being that your app also has these problems, anyway.
> Figure out what you need, and chances are you need it anyway,
> regardless of what the connection offers.
>
> Simple example: older websites for payments used to have a label on
> the PAY-NOW button saying "don't click twice!" The problem was that
> there were many bugs in the various stages that triggered resends.
>
> Simplistically, developers thought the users were pushing the
> buttons twice. And they were, sometimes. But there were other
> things too. So the solution was to have every purchase form include
> a unique cookie, and then strip out any duplicates.
>
> So, now we've added a datagram integrity feature, layered over the
> top of HTTP over the top of SSL over the top of TCP. Hence, the
> integrity features in the lower layers are now redundant, in some
> sense or other.
They are only partially redundant, because they still do retransmissions of
lost packets without requireing the whole transaction to be retried. This
increases the performance and resilience without causing any additional pain.
The whole point here is that the user error-retry case happens less often and
the developer of the website might not think about the flaw.
> >3) If your firewall is so closed that only HTTP gets through so be it. People
> >building websites often do not know what they do. No transport can provide
> >you
> >with a guarantee that there will be no network errors. NATs have to be dealed
> >with at application level in some cases and some people try to create insane
> >workarounds. A man in the middle may intercept all data. Seriously, what does
> >this say *against* TCP? That people using it have no clue?
> >
> >OK, TCP could be build to survive IP address change of one side. Do we want
> >this? Doesn't this negate many of the privacy benefits we have with dynamic
> >IPs?
> >
> >Maybe we could protect better against MITM and other attacks. But this is not
> >is easy as it sounds, because key setup has to be done right.
>
>
> Right, there are so many problems at so many layers that the job has
> to be done properly. By the time you've done the job properly, you
> don't need TCP, anything it does is likely not that helpful.
>
> (Note that there is one caveat to all of this: If you need
> reliability. If you don't need reliability, ignore that page.
> E.g., HTTP doesn't really need reliability, pages get dropped all
> the time, and there is a reliability feature in the user, "click
> reload".)
IMHO TCP is not less helpful if you do the job correctly. At least, because
unreliable can mean 50% packet loss in a very highly loaded network - and
because of features other then reliability.
-Michi
--
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
