On Tue, 7 Jul 2009, Mohun Biswas wrote:
I'd say the meta-problem is precisely due to the deliberately vague phrasing. I understand the value of leaving certain things unspecified, but there's such a thing as being too imprecise.
Oh indeed. The problem is to find the right ballance. To give enough info to let the user understand exactly what to expect and how to use it, but still not give info than isn't completely backed up or is subject to change.
Unfortunately, the vague phrasing means we have to know what sockets are open along with their current state and semantics.
I disagree with that, but I realize I'm so biased here so it's not really relevant how I interpret the user docs.
For instance, even a non-expert like me knows that in order to transfer a file from here to there we need two file descriptors: one from which to read the file and one to write to. The libcurl-multi document makes no distinction between the two, which I guess is because you assume we understand that one is a socket which can block and the other is likely to be a file which will not.
The libcurl docs makes no distinction because it has no knowledge about one of them - the file reading one - and you never tell libcurl anything about it. Thus libcurl doesn't care about it and the docs doesn't mention it.
How do you suggest we would?
The libcurl-multi doc says things like "libcurl will then transfer data if there is anything available to transfer". Since I'm doing uploads I read this, as I think any native speaker of English would, to mean "if there's anything ready *on the input side*". I'm guessing you really mean "libcurl will then transfer data if any socket is ready". Or maybe even "... if doing so would not cause the program to sleep". Or something - since I don't actually _know_ what it does, it's hard to suggest language.
Upload is callback based (too). libcurl calls your callback function to get data when it already knows that the server socket is ready to receive. It doesn't know any file descriptor or similar for reading that data when uploading. Your app provides the data when libcurl asks for it.
Next, wrt CURLM_CALL_MULTI_PERFORM you might want to be more clear in saying "you probably don't want to do anything which would cause your program to block while curl_multi_perform still has data ready to transfer". You state it in terms of select() but there are more ways to block and that - you could wait for a child, for instance. Again, this assumes I now understand it.
Perhaps, but we can't cover it all in the docs. And other blocking ways will just delay the progress, while using libcurl's own API to wait for further actions before the already reported actions haven't yet been dealt with feels much more like an actual error to me.
Here's another thing I don't know about sockets: is there any correlation between a socket being "ready" and the speed of the physical connection?
Yes, but hard to figure out. In general it'll become ready more often if the network speed is high so that it empties the internal send buffers faster.
What I'm afraid of is that a socket on a slow connection will continually signal ready.
It won't. libcurl will send() data when the socket is ready to receive data and then the TCP stack will send that buffer to the peer and the socket will not be "ready" again until the stack is again ready to get a send().
If I need to send a large file to it, it seems to me I could still be tied up for a very long time sending data there because the socket might remain ready even though the connection is slow.
No. There's also a limited buffer in the TCP stack that will prevent libcurl from passing on very large amounts and it'll instead get EWOUDBLOCK (or similar) returned and then go back to waiting for the socket to be "ready" to receive more data.
But maybe I misunderstand and the socket will become unready until its buffers are drained? This is the sort of low-level detail that I think should either be handled internally by an API or spelled out in its documentation.
To me that's the sort of lowlevel socket behaviors that are just universally known! ;-)
But more seriously, it is hard to document this as then we're stepping into documenting how sockets work and it is hard to know where to draw the line then. I'm not saying I know where it would be drawn best.
-- / daniel.haxx.se
