* David Schwartz wrote on Wed, Aug 29, 2007 at 08:07 -0700: > > > > and a blocking write should return as soon as at > > > > least one byte has been written. > > > > > > No. A blocking write should block until all the requested data cen be > > > written. > > > > ahh, interesting. Why should it? > > Because this is what most people want it to do.
This is acceptable for Perl, but not for C :-) Even if most people would want a write contradicting its man page, I'd still consider it wrong :) > If you tried to write two bytes, why would you want to wait > until the first one could be written but not wait until the > second one could be written? It just doesn't make much sense. If the first byte (or any part of the buffer) could be written instantly or (e.g. if no select returned ready before :)) after some amount of time waited, write should return to give the calling application the control. I looked up write(2) on opengroup.org and found a page that surprised me :) The information on opengroup.org tell `The write() function shall attempt to write nbyte bytes from the buffer...'. My man page tell `write writes up to count bytes to the file...'. My man page claims to be conforming to `SVr4, SVID, POSIX, X/OPEN, 4.3BSD'. The opengroup.org page distinguishes write semantics based on what the fd is kind of (file, FIFO, STREAM, ...) which IMHO cannot be correct because it destroys the abstraction. It would be a pitty if some implementations would follow the opengroup.org description and change the write semantics, woulnd't it? on this page, there are more `violations of file abstractions', someone could get the impression that the `regular files on a ext2 file system' or alike would be most true of the world, for instance `The read() function reads data previously written to a file.'. This is simply not true: [EMAIL PROTECTED]:~ # md5sum tmp fcbb60277dee9a96ec9c2fbfa47a478d tmp [EMAIL PROTECTED]:~ # dd of=/dev/urandom if=tmp count=100 bs=1 100+0 records in 100+0 records out [EMAIL PROTECTED]:~ # dd if=/dev/urandom of=tmp count=100 bs=1 100+0 records in 100+0 records out [EMAIL PROTECTED]:~ # md5sum tmp 5cdceb17716e1dd4441f9bd4027fd75e tmp :-) When the file is /dev/urandom (a random number generator device at least on linux), it shall NOT return the data previously written. For devices (and sockets :)), I think this is obvious. anyway. > There is some code out there that assumes that a blocking write > will fully complete unless there is an error. This code is, as > you point out, broken. > > However, a 'read' that blocked after some data was received > would be broken. ahh, ok, yes. I think, this is somewhat `symetric' for read and write. > > correct implementation needs to guarantee that. If queue > > discarding is possible, a flag must be stored (or so) to make > > read return EAGAIN or whatever (probably causing most code to > > break anyway, because noone expects EAGAIN after select > 0 :-)). > > That's simply impossible to do. The problem is that there is no > unambiguous way to figure out whether an operation is the one > that's not supposed to block. Consider: > > 1) A thread calls 'select'. > > 2) That thread later calls 'read'. > > If the 'select' changes the semantics of the 'read', then if the thread > didn't know that some other code called 'select' earlier, the later > read-calling code can break. If some other thread called read (or another function), of course before the next read a select must be called again. Only for the next read guarantees may be made, not for the 103th read called after the second reboot :) I think, the API must behave in the same way independently whether used by multiple threads or a single one, if possible. Of course, care must be taken when e.g. two threads call select on the same file descriptor and so on. Many ways to get it very complex, hum... > > Of course, it probably isn't the best idea to rely on that, > > maybe some embedded highly size optimised lib makes the one > > or other compromise or so... :) > > There are known implementations that perform some data > integrity checks at 'recv' time, so a 'select' hit that results > in the data being dropped later can lead to 'recvmsg' blocking. I think, select simply cannot work on the low layer that may have data that may not be valid because of pending integrity checks. select must work on the same buffer as read (which gets integrity checked data only). > You can argue that these implementations are deficient, but I > think that argument would be inconsistent. This behavior is > accepted with 'accept', and it's precisely the same issue. mmm... my manpage talks about select and read (and select and write), but not about accept. So I think this means for accept no guarantees are made. My accept man page states clearly `To ensure that accept never blocks, the passed socket s needs to have the O_NONBLOCK flag set'. I think, read and accept in conjunction with select simply have different semantics. Is that right? > > The statement `to see if a read will not block' does not > > sound very concrete or formally. For instance, only the next > > read can be in scope and probably only if no other call > > (recv, write, don't know) is performed on this fd - I guess. > > The problem is that it becomes very hard to figure out what an > "other call" is. What about 'setsockopt'? If you don't even > know what you're asking for, you wouldn't even know if you had > it. ;) setsockopt gets a filedescriptor (socket) as parameter, so it counts as other call. Maybe there are problematic calls, right, or sockets/files shared through fork(), where processes may influence each other, so that it might look for one process that behavior would be strange, but maybe in fact it is as expected just confusing because of the other processes actions. I think best is to avoid such constructions, seems to be very difficult to handle... > > Yes, and additionally, there may be implementations supporting a > > select function but at the same time not even conforming the > > standard, I think such `TCP stacks' exist. > > BTW, which standard would it be, `4.4BSD'? > > I'm talking about The Single Unix Specification or The Open Group Base > Specification. > http://www.opengroup.org/onlinepubs/009695399/functions/FD_SET.html > This is reasonably clear that 'select' reports current status, just as > functions like 'stat' do. They provide no more of a future guarantee than > functions like 'stat' do. Thanks for the link. My man page and my linux boxes here do not comply to it (fortunality :)). But what a pitty that there are multiple selects. I though, for sockets or select some agreement (as 4.4BSD) would be commonly accepted. Now there is some Single Unix doing it differently and POSIX got a pselect... What a pitty. (If `Portable Operating System Interface for Unix' and `Single Unix' contradict, maybe it had better been called `Yet another Unix' or so grpmf...) So a BSD/Linux/POSIX compliant program working on a BSD/Linux/POSIX compliant select won't work on a Single Unix compliant OS. That would mean as I understand it, that a program needs to know whether it runs on e.g. Solaris (Single Unix) or Linux (POSIX) to know how to use select? Is this really true? Or did I just misunderstood? This just shows that I have no clue about standards (I though SUS would be `POSIX compliant' and BSD and linux would comply in large parts). > > mmm... If the man page tells that `a read will not block' (which, > > BTW, is also told for write, but not for accept), I think the > > possible subsequent operations are defined: read or write. > > Why include "write" in the claim that a read will not block? Sorry, I meant, the `a read will not block' when the fd is listed in readfds - similarily this is told for write: `write will not block' when the fd is listed in writefds. > What about "setsockopt"? I don't know if it could block, I think theoretically such option may exist. Usually, I would expect it not to block (regradless whether select was called or not). I don't know when someone would call select before setsockopt (as you surely noted I'm not so experienced in all those details). I think, after setsockopt usually someone could simply reinvoke select to check if read should be called (or whatever). I would not even call select before setsockopt, only before read. Actually, my code calls setsockopt only `at the begining' after accept without any select. Is this wrong? > In any event, only Windows has documentation that uses that > kind of language. I think it was just sloppiness, as the select > function on Windows was a hack job just to support > compatability with existing Berkeley sockets implementations. :-) Similarily for other TCP/IP stacks, maybe for embedded devices. Or for linux vs. Single Unix as it turned out... > > > A file is always ready. There is never anything to wait > > > for. > > > I disagree here. Files may not be ready, because NFS Server > > may not be responsing, a USB stick may be slow, a FTP file > > system may need to dialup an ISDN line, the FIFO or STDIN > > could be empty, the harddisk may be busy and need some > > milliseconds to read in the requested blocks --- or the file > > could be a device or a socket :) > > Nevertheless, it's ready and cannot be waited for. With slow > file systems, they generally try to perfectly mimic the > semantics of fast file systems. An empty file is still ready to > be read right now, to report correctly that it is empty. But select on STDIN usually works (on linux) - is this linux specific? I would consider it almost useless, if select couldn't be called on STDIN because STDIN could be a `fast file system'! Does this also mean that select wouldn't work as I expect it when reading a e.g. ext2 filesystem from a slow media, let's say NFS loop back, USB Stick or floppy disk? I assumed it would work, but now as you pointed to it I find no statement about in the man pages... :( > > mmm... beside I'd consider a program expecting write to block > > until all data has been written already broken, I see your point, > > ok... So you say that theoretically it is guaranteed that read > > won't block after select but practically there will always be a > > risk to have an implementation not fulfilling this guarantee, > > right? > > No, it's not guranteed even theoretically. The 'select' > function is simply a status-reporting function that does not > and cannot guarantee what will happen in the future. sorry, I don't get this. I'm afraid that this become a kind of circle :) For me, select technically reports the state of a buffer. Of course it could be implemented with a one byte buffer or maybe even without any buffer but a flag (or set of flags). Anyway. So lets say it is a buffer. Because the access to the buffer is limited by the same API, I think it should be able to guarantee. If the buffer is empties, by a file truncation or so, either this could be unnoticed (meaning, that the read returns the data, as it happens when using fread buffers) or EOF would be returned - nonblocking in any case. The status of the buffers encapsultated and hidden in the implementation won't change in the future except through the implementation - and this could be catched to make read at least return an error instead of block. Actually, what happens with the `source' of the data for this buffer (file, tcp, udp) does not matter at all. Without any buffers (or flags), select cannot be implemented of course, because it is required to limit/control the access to it (or at least to get reliable information/notification about to set a flag). So I would conclude that it is a status-reporting function but also could guarantee. What do I miss? > Assuming it will is as serious a bug as checking permissions > with something like 'access' and then assuming the information > must still be valid in the future. but access makes no statement at all about blocking/nonblocking future calls? Also, I would assume that the information of access will be valid in future as long as no other call will be made to this resource (e.g. a chown via NFS or simply that the file would be removed by another process - which require calls to this resource, e.g. a remote unlink or so). Maybe access is a slightly different topic? About the other mail from Yves: > > > Is that not to say that if select() says it's ready to > > > read, I'm guaranteed my next read() won't block? > > > > Nope. To say that, it would have to say "will not block". > > Instead it says "would not block". This refers to a > > hypothetical concurrent call not blocking at some instant > > in-between when you called 'select' and when it returned. I think `will not block' wouldn't be said correctly because it is not required to even call read at all, and if it would not be called it couldn't be said whether it blocks or does not block, but may english is far away from being good enough to understand such specifics correctly. However, the `hypothetical concurrent call' IMHO is hypothetical :) - maybe some `hypothetical sequential call' was meant, could this be the case? I'm not sure if the word sequential is right, maybe proximate, successive or even `in direct succession'? At least the discussion IMHO shows that specs are not clear and using APIs correctly is a challenge, because of those doubts the best practice is to explicitely use non-blocking fds and that the best documentation is no replacement for deep testing :-) oki, Steffen About Ingenico Throughout the world businesses rely on Ingenico for secure and expedient electronic transaction acceptance. Ingenico products leverage proven technology, established standards and unparalleled ergonomics to provide optimal reliability, versatility and usability. This comprehensive range of products is complemented by a global array of services and partnerships, enabling businesses in a number of vertical sectors to accept transactions anywhere their business takes them. www.ingenico.com This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. About Ingenico Throughout the world businesses rely on Ingenico for secure and expedient electronic transaction acceptance. Ingenico products leverage proven technology, established standards and unparalleled ergonomics to provide optimal reliability, versatility and usability. This comprehensive range of products is complemented by a global array of services and partnerships, enabling businesses in a number of vertical sectors to accept transactions anywhere their business takes them. www.ingenico.com This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
