> David Schwartz wrote: > > That is not only not implemented by any known implementation but quite > > literally impossible. Please tell me what implementation > > guarantees that a > > TCP 'write' after a 'select' hit for writability will not block.
> This is no use, your asking me for references and I'm asking you for > references. I don't really have the time to entertain this issue as > much as would be necessary; so I've given over to you the job to point > out the code path within a published kernel that does what you describe. No, that's precisely what got us into this mess in the first place. It may be possible that no published kernel around today violates this assumption. But that doesn't mean that a kernel won't come out tomorrow that does. This is precisely the type of thinking that caused the problems that I'm talking about. You code to assumptions because they just happen to be true (or at least, you hope they are true) *today* and then tomorrow, when they're not, your code breaks. We have *standards* that specify what behavior is guaranteed. This is a case where those standards, thankfully, provide sufficient guarantees for us to write working code. Yet some people insist on using assumptions instead of guarantees, and their code breaks. > > So all thos implementations that give a 'readability' hit on a listening > > socket and then block in 'accept' if there's a subsequent error on the > > connection don't exist? > Okay granted there is an accept() concern on some platforms, that if an > "embryonic connection" is reset some operating systems might kill the > socket between the select() and the accept() call. While others will > persists with providing the socket to application, only to then return 0 > from read() (End-of-stream) soon after. > I've not been so concerned with server specific API calls, its really > socket(), bind(), connect(), read(), write(), close() that most people > care about, in particular just the read() and write() parts. You are totally missing the point. The people who caused code to break only cared about 'accept'. They couldn't think of any way it could break. Maybe no implementation at the time had this behavior. Maybe they did, and they just missed it. The point is, they made the exact same assumption you are endorsing for the same reasons, and their code break and people suffered as a result. Some people have learned from this mistake and others have militantly insisted on repeating it. Which camp are you in? > > Umm, Linux does this. That's what caused the inetd > > denial-of-service attack. > > It's precisely people following the bad advice you are giving > > that created > > this problem. > Please cite your references, either discussion on the issue or direct > references to the code within the kernel this causes this behavior. Do > the affected kernels still work this way in their current versions today? You don't get it. It makes no difference what any kernel that exists today does. It only matters what *guarantees* you have in the standards. What you are asking me to do is document specific instances to you. What will happen is that you will come back in each instance and say, "oh, that's not like what I was talking about" or "well, the kernel was unreasonable in that case". But the point is, those instances happened and the code broke and people suffered as a result. The fact that I can't think of a way your code could break today doesn't mean anything. You have to code to the guaranteed behavior. The next Linux kernel could, if it wanted to, randomly drop 50% of UDP packets sitting in the receive queue and no standard would be violated. > Well it does, as the disappearing readability condition only occurs when > you: > * Drop all data for SOCK_DGRAM (so there is nothing left) > * Drop the head data for SOCK_STREAM (so the next byte in the stream > is no longer there) > Tail dropping is a normal occurrence it happens when the kernel is > overloaded with data and is permitted. Tail dropping does not affect > the select() event mechanism and can be a result of the network/kernel > queuing up data faster than the application is pulling it from the socket. All it takes is a lot of tail drops and the whole queue is gone. No rule prohibits kernels from dropping as many UDP packets as they want under memory pressure. But again, that is totally beside the issue. It doesn't matter how or why this might happen. It just matters that you are not guaranteed that it won't happen. > For TCP is it not application data that is tail dropped, but TCP > segments, i.e. once data has been acked to the kernel _MUST_ retain it > because the sending is never going to resend it again, but by dropping > TCP segments you make the sending end take responsibility to retransmit > that data you dropped. The kernel is free is discard entire TCP > segments when the read memory quota is exceeded for a given socket, > bearing in mind that TCP is only designed for < 3% dropped packets so > this should not be habbit forming. Why can't the kernel discard a TCP segment that has not been acknowledged but has triggered an select hit for readability? And, anyway, it doesn't matter since 'select' semantics are protocol neutral. You cannot synthesize a 'select' guarantee from TCP semantics. > For UDP the datagrams are queued in the socket, when there is too much > data queued it may start tail dropping the new segments that arrive. > The kernel may also prune the queue but it will never prune all packets > such that there is no queue what so ever. Because there is always a > reservation of at least one packet then select() event notification > won't be broken. What standard guarantees that there will always be a reservation of at least one packet? You are just making things up. > The same is true for writability, once the low water mark triggers > clearance then it will persist as the low water mark guarantees space > for at least 1 byte (which is enough for write not to block). Unless the other end shrinks the window and the implementation decides to reflect that in the send queue. What standard says it can't do this? Again, you are making things up. > >> The challenge here is for someone to point out by way of pointing to > >> published source of a BSD socket implementation that will remove all > >> packets from the socket queue after having already advised a > readability > >> indicator to the application. > > > > Huh? You can't get a guarantee out of the lack of documentation. > No but you can out of proven implementation. This is what I'm asking > for a citation on. No, you cannot. This is your fundamental basic error. You simply *cannot* get guarantees out of implementations. This is why code breaks on a newer CPU or newer OS or newer system library. I, and all other sane programmers, utterly and totally reject the notion that one can get a guarantee out of an implementation. Now, sometimes one has no choice. All you have are implementations and you have to get your code to work. So you see what the implementation does and hope for the best. But it *sucks* when you have to do that, and you know that you may cause future pain, but what choice do you have? That case is not this case. The 'select' function is standardized. > Lack of a specifics point in specification's does not mean than > implementations don't do things. It can just mean that the broad scope > the specification was to encompass needed to be vague to allow for > things like socket(SOCK_MY_BESPOKE_PSEUDO_STREAM, 0, 0) which was also > an experimental option later for vendors. Exactly. > The BSD socket mechanism is generic supporting many protocols, but for > the purpose of this discussion it is TCP that is of interest to most > uses of SSL. Right, but 'select' has to work with any number of protocols. If there was some TCP-specific version of 'select', it might be able to provide semantics that better align with TCP itself. But that is just not the case. We are left with a protocol-neutral 'select' function and the advantages and disadvantages that it has. > > Unfortunately, on Windows, you have to be able to talk to > multiple socket > > implementations. There is no guarantee that a Windows > application only has > > to deal with the Windows native socket service provider. Firewalls, for > > example, often interpose their own LSP. These LSPs have been > known to have > > slightly different semantics from the Windows native ones. > There are known > > firewalls that decide which UDP packets to drop at receive time, for > > example. > Please cite your references. Surely given that Microsoft wrote their > own specification on their own socket layer and a citation has already > been provided in this thread making it crystal clear then anything > different might be considered a vendor specific bug. This would depend > is the cited statement was lifted from the Microsoft specific socket > documentation or the device driver / SDK documentation. I guess you are militantly resistant to education. It doesn't matter what existing implementations do. You have to work with implementations that don't exist yet. Microsoft does not say that a subsequent operation is guaranteed not to block because that is quite literally *impossible* to implement. DS ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
