IP address connection fail-over is broken for non-blocking sockets

2013-10-09 Thread David Strauss
there be a flag to force blocking behavior to allow fail-over to occur? -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http

Re: Clarifications on using libcurl

2013-09-13 Thread David Strauss
. The way we stream-parse the XML is also a bundled libcurl example [2]. [1] https://github.com/pantheon-systems/fusedav [2] http://curl.haxx.se/libcurl/c/xmlstream.html -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Re: A Question On Libcurl Performance

2013-09-11 Thread David Strauss
On Sat, Aug 31, 2013 at 11:57 AM, Thomas Dineen tdin...@ix.netcom.com wrote: For both Solaris 10 and Fedora 14 Fedora 14 hasn't been supported since 2011, and many of its libraries are very old now. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Re: New SSL backend GSKit, certinfo for everyone, etc.

2013-07-12 Thread David Strauss
commit it in a few days. Good week-end to all of you, Patrick --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html -- David Strauss | da...@davidstrauss.net

Re: Regarding Usage of Custom Application Layer in Internet Protocol Suite

2013-05-31 Thread David Strauss
You could also just use an SSL sockets library directly or look at ZeroMQ. libcurl isn't suited to using HTTP without the HTTP. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette:

Re: Patch: Ability to set options for hashing, HTTP Content-MD5 support, Supporting Digest qop=auth-int part 1

2013-05-28 Thread David Strauss
I know Daniel has his reservations, but I would love to see optional Content-MD5 support in libcurl. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html

Re: header functions and error response headers

2013-05-27 Thread David Strauss
/libcurl/c/curl_easy_getinfo.html#CURLINFORESPONSECODE -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail

Re:

2013-05-27 Thread David Strauss
the server expected it, like when you've said you will upload N bytes and you upload less than N bytes), you may experience that the server hangs waiting for the rest of the data that won't come. [1] http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTREADFUNCTION -- David Strauss | da

Re: Interest in an Expat XML example?

2013-05-19 Thread David Strauss
I've submitted an XML stream parsing example as a pull request: https://github.com/bagder/curl/pull/66 I dropped the PROPFIND stuff for clarity. It's probably best to keep it to basics for this first example. --- List admin:

Re: Interest in an Expat XML example?

2013-05-19 Thread David Strauss
I wanted to note that this is far more efficient than the existing XML/HTML parsing examples. The existing examples pull the entire response into memory before starting to parse. This new example only allocates buffers to fit the character data between tags. This is pretty important for our use

Re: Interest in an Expat XML example?

2013-05-19 Thread David Strauss
On Sun, May 19, 2013 at 2:22 AM, Daniel Stenberg dan...@haxx.se wrote: ... and I've merged and pushed it already. Thanks! Awesome! I verified that everything is clean using cppcheck, Clang Analyzer, and Valgrind, so it should be solid. -- David Strauss | da...@davidstrauss.net | +1 512

Re: Read Certificates from curl

2013-05-17 Thread David Strauss
manage the SSL context, but it is not portable to non-OpenSSL backends for libcurl. [1] http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTSSLCTXFUNCTION -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Re: axtls memory leak

2013-05-17 Thread David Strauss
On Thu, May 16, 2013 at 2:10 PM, Hu, Eric e...@directv.com wrote: Regarding the patch itself, it still blocks in the connect phase. This is true for the NSS SSL back-end, too. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Interest in an Expat XML example?

2013-05-17 Thread David Strauss
. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html

Re: curl-library Digest, Vol 93, Issue 27

2013-05-15 Thread David Strauss
On Tue, May 14, 2013 at 9:46 PM, Aldrich fenghelongn...@gmail.com wrote: OK,I will try it. That doesn't have any context as a reply to a digest. Required reading: http://curl.haxx.se/mail/etiquette.html -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Re: how curl get access_token in OAuth?

2013-05-14 Thread David Strauss
think you want something like curl_easy_getinfo() with CURLINFO_EFFECTIVE_URL [2] [1] http://liboauth.sourceforge.net/ [2] http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Re: IMAP and POP3 with libcurl 7.30

2013-05-12 Thread David Strauss
that the command-line curl uses libcurl. If something is possible with the curl CLI, it's possible with libcurl. There are a couple of cases where the curl CLI provides additional logic, like HTTP retries, on top of libcurl. But, it's never redundant with what libcurl provides itself. -- David Strauss

Re:

2013-05-10 Thread David Strauss
trivial mistake, if this is the case please let me know, I would greatly appreciate it. source: http://pastie.org/pastes/7827550/text valgrind output: http://pastie.org/pastes/7827548/text curl version: 7.30.0 Your trace seems to be missing a lot of symbols. -- David Strauss | da

Re: SSL with NSS not properly timing out

2013-05-08 Thread David Strauss
On Wed, May 8, 2013 at 11:29 AM, David Strauss da...@davidstrauss.net wrote: which I think it is now I'm referring to very recent releases here, not now as in current Fedora packages. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Re: SSL with NSS not properly timing out

2013-05-08 Thread David Strauss
On Wed, May 8, 2013 at 5:01 AM, Kamil Dudka kdu...@redhat.com wrote: Only the handshake still blocks with NSS. Is there a way to fix this, too? Does it at least time out based on the configured libcurl connection or request timeout? -- David Strauss | da...@davidstrauss.net | +1 512 577

Re: SSL with NSS not properly timing out

2013-05-08 Thread David Strauss
On Wed, May 8, 2013 at 11:37 AM, David Strauss da...@davidstrauss.net wrote: Does it at least time out based on the configured libcurl connection or request timeout? Read some source and found the answer: yes. snip timeout = PR_MillisecondsToInterval((PRUint32) time_left); /* Force

SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
); curl_easy_setopt(session, CURLOPT_TIMEOUT, 60 * 3); Is this an NSS bug, or is it an issue with how libcurl uses NSS? I'm on Fedora 17 with libcurl 7.24.0 and NSS 3.14.3. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
On Tue, May 7, 2013 at 1:46 PM, David Strauss da...@davidstrauss.net wrote: NSS seems stuck in poll loop, which has been going on for hours Actually, I'm not sure it's NSS stuck there. The loop could be higher up. I just see an unending series of polls from strace. -- David Strauss | da

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
This -1 timeout is also in the current master: https://github.com/bagder/curl/blob/master/lib/nss.c#L1518 On Tue, May 7, 2013 at 2:11 PM, David Strauss da...@davidstrauss.net wrote: It looks like PR_Recv(conn-ssl[num].handle, buf, (int)buffersize, 0, -1) in nss_recv() (nss.c) may be the problem

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
It looks like PR_Recv(conn-ssl[num].handle, buf, (int)buffersize, 0, -1) in nss_recv() (nss.c) may be the problem. That sets the timeout for NSS to 4294967295. On Tue, May 7, 2013 at 1:57 PM, David Strauss da...@davidstrauss.net wrote: On Tue, May 7, 2013 at 1:46 PM, David Strauss da

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
://developer.mozilla.org/en-US/docs/PRIntervalTime On Tue, May 7, 2013 at 2:14 PM, David Strauss da...@davidstrauss.net wrote: This -1 timeout is also in the current master: https://github.com/bagder/curl/blob/master/lib/nss.c#L1518 On Tue, May 7, 2013 at 2:11 PM, David Strauss da...@davidstrauss.net wrote

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
the remaining timeout allowable for the request (considering how DNS, etc, have already contributed to request time) and sending that in. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
On Tue, May 7, 2013 at 2:44 PM, Daniel Stenberg dan...@haxx.se wrote: That's already done before the function is called in the first place. The the GnuTLS and OpenSSL versions of that function for example are completely non-blocking. Well, then that sounds perfect! -- David Strauss | da

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
a backport of the fix once it's in: https://bugzilla.redhat.com/show_bug.cgi?id=960765 -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette

Re: [bagder/curl] 8ec2cb5544 WIN32 MemoryTracking

2013-05-07 Thread David Strauss
On Tue, May 7, 2013 at 3:10 PM, Mel Smith syn...@cox.net wrote: I don't know *how* to revert to an earlier commit :(( git revert 8ec2cb5544 That will do a sort of reverse cherry-pick of that single change. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
at 2:59 PM, David Strauss da...@davidstrauss.net wrote: On Tue, May 7, 2013 at 2:49 PM, Daniel Stenberg dan...@haxx.se wrote: Assuming it actually makes any difference for your case at least! ;-) If it means that it respects the timeouts we give, it's absolutely a fix for the problem we see

Re: Proposed changes to SSL comparison documentation

2013-05-04 Thread David Strauss
/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette

Lockup in SSL-based communication

2013-05-01 Thread David Strauss
in curl_easy_perform (curl=curl@entry=0x7fb8c387c000) at easy.c:541 snip -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http

Re: Lockup in SSL-based communication

2013-05-01 Thread David Strauss
On Wed, May 1, 2013 at 12:28 PM, David Strauss da...@davidstrauss.net wrote: What timeouts should we be configuring to give up faster when it's in this state? Based on empirical data, it looks like CURLOPT_TIMEOUT is taking effect, but I'm curious is there's a more precise way to time out here

Verification of Content-Length

2013-04-29 Thread David Strauss
home in the documentation for what the split in responsibilities is between libcurl and a user of the library? Is there interest in adding optional Content-MD5 support? -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Re: Verification of Content-Length

2013-04-29 Thread David Strauss
/client communication and our communication with S3, so the set of implementations needing to interoperate is pretty limited. The goal is to checksum, not to avoid attacks. MD5 is quite adequate for that. I would like to use a standard rather than rolling our own method, though. -- David Strauss

Re: sharedhandle Curl_resolv_unlock() error!

2013-04-26 Thread David Strauss
-- End of curl-library Digest, Vol 92, Issue 59 --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html -- David Strauss

Re: sharedhandle Curl_resolv_unlock() error!

2013-04-26 Thread David Strauss
Along the mailing list etiquette lines, sorry for my top-post. That's not cool, either. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html

Re: [PATCH] SFTP file listing suggestion

2013-04-20 Thread David Strauss
/mail/etiquette.html -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html

Re: Adding PROPFIND support

2013-04-15 Thread David Strauss
cURL protocols, too. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html

Re: Adding PROPFIND support

2013-04-14 Thread David Strauss
coherent unit of data rather than a buffer of incoming bytes. From a layering perspective, though, this could all live in a new library that provides libcurl-compatible write callbacks for directory listings that abstract the differences between protocols. -- David Strauss | da...@davidstrauss.net

Re: URL parsing

2013-04-13 Thread David Strauss
On Sat, Apr 13, 2013 at 3:12 AM, Steve Holme steve_ho...@hotmail.com wrote: Whilst I have 20 odd years' experience as a C/C++ developer would someone be so kind to check the four uses of sscanf() in url.c between lines 4381 and 4402 to see if this is the best / most optimal way of extracting

Re: Inject a PEM certificate using CURLOPT_SSL_CTX_FUNCTION

2013-04-13 Thread David Strauss
sure it's also possible programmatically using the OpenSSL API. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se

Re: URL parsing

2013-04-13 Thread David Strauss
to prove correctness of a full fledged lexer/parser or separate library? I can't see how that can be... A quality lexer/parser guarantees that a specified grammar lacks ambiguity and that crazy/malicious input patterns get handled safely. -- David Strauss | da...@davidstrauss.net | +1 512 577

Adding PROPFIND support

2013-04-13 Thread David Strauss
: https://github.com/pantheon-systems/fusedav/blob/curl/src/props.c Is there interest in ls-style output for WebDAV, provided the path ends in a slash and an option gets set? -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Re: DNS-based cluster awareness for connection pools and pipelines

2013-04-12 Thread David Strauss
also use it for our PHP and Python API clients, which also connect through load balancers but don't run into as many saturation issues. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http

Re: curl_share and persistent connections

2013-04-11 Thread David Strauss
I've sent in a patch to the docs. On Wed, Apr 10, 2013 at 10:31 PM, Nick Zitzmann n...@chronosnet.com wrote: On Apr 10, 2013, at 6:32 PM, David Strauss da...@davidstrauss.net wrote: The share interface documentation [1] specifies that DNS lookups and cookie data get shared

Re: certificate verification against system cert (?) when custom CAINFO is set

2013-04-11 Thread David Strauss
What is the output of curl -V? The SSL/TLS library cURL is linked to has a major impact on how it performs system-level validation. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette:

Re: certificate verification against system cert (?) when custom CAINFO is set

2013-04-11 Thread David Strauss
namespace to remove access to any system-level trusted certificates. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se

Re: certificate verification against system cert (?) when custom CAINFO is set

2013-04-11 Thread David Strauss
On Thu, Apr 11, 2013 at 1:39 PM, Daniel Stenberg dan...@haxx.se wrote: Apple has added some magic for certificate verification in their OpenSSL version. Apple OS X has a certificate management system that might even be accessible within a chroot.

Re: Only retrieve the headers of a GET reply and return?

2013-04-11 Thread David Strauss
that simply returns the number of bytes sent in, and (3) CURLOPT_PROGRESSFUNCTION (with CURLOPT_NOPROGRESS set to zero) to cancel after the body starts. Part #3 is optional. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile

Re: Only retrieve the headers of a GET reply and return?

2013-04-11 Thread David Strauss
On Thu, Apr 11, 2013 at 1:40 PM, David Strauss da...@davidstrauss.net wrote: (3) CURLOPT_PROGRESSFUNCTION (with CURLOPT_NOPROGRESS set to zero) to cancel after the body starts. Here's a good write-up on how to do that: http://curl.haxx.se/mail/lib-2009-04/0296.html -- David Strauss | da

Re: Only retrieve the headers of a GET reply and return?

2013-04-11 Thread David Strauss
Oh, actually it looks like you can make the transfer fail right from the CURLOPT_WRITEFUNCTION, which means you could just have it return zero. You would have to expect libcurl to consider the request failed, though. On Thu, Apr 11, 2013 at 1:46 PM, David Strauss da...@davidstrauss.net wrote

DNS-based cluster awareness for connection pools and pipelines

2013-04-10 Thread David Strauss
I see some exciting pipeline-management features landing in new cURL releases that balance connections to the same hostname. Is there any interest in extending such support to balancing/fail-over between multiple A/ records returned for a domain? Since cURL seems to prefer its own DNS client,

curl_share and persistent connections

2013-04-10 Thread David Strauss
The share interface documentation [1] specifies that DNS lookups and cookie data get shared, but is there an exhaustive list of what gets shared? Specifically, do persistent connections get shared? [1] http://curl.haxx.se/libcurl/c/libcurl-share.html -- David Strauss | da...@davidstrauss.net