Re: curl_easy_perform() failed: Couldn't connect to server
On Fri, Mar 15, 2013 at 12:07:44AM -0400, Anil J wrote: Just to test a new scenario, I changed the URL to a HTTP web server running on the same host, but still same error persist. I can fetch the web server from the firefox browser, but not from the curl program. Looks like something obvious is missing. Can somebody help? [...] --2013-03-14 18:23:58-- http://gmail.com/ Resolving localhost (localhost)... 127.0.0.1 Connecting to localhost (localhost)|127.0.0.1|:8080... failed: Connection refused. It looks like you have a proxy configured. If you're not doing it explicitly in your app, then you probably have an environment variable set. See if env | grep -i proxy shows such a variable, clear it and try again! Or, see if Firefox has a different proxy server configured and use that one instead. Dan --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() fails with Problem with the SSL CA cert (path? access rights?) after first time calling this routine
On Thu, 14 Mar 2013, cnm marketing wrote: * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small Please stop top-posting and full-quoting. My 3.2 seconds of googling on this topic lead to this: http://comments.gmane.org/gmane.comp.encryption.openssl.user/43777 -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RE: CURLE_OPERATION_TIMEDOUT using curl_multi_socket_action
On Wed, 13 Mar 2013, Paras S wrote: Find the sample program attached which illustrates the problem I am facing. I am using Mozilla's NSPR: http://www.mozilla.org/projects/nspr/ I don't know nor use NSPR so I can't comment on all subtle details here. But really, you're using epoll in the example so that is what you should rather focus more on using correctly here. Let's try to straighten out everything from the ground up, as this source code is very VERY far from a proper use of libcurl. It is kind of amazing that it works to any degree at all! 1 - your code doesn't use CURLMOPT_TIMERFUNCTION[1] which in the case of the socket_action API is just wrong unless you have a very specfic idea on how to survive without it and knowledge about why. 2 - why use a thread pool at all in this case? the multi API handles parallell connections in a single thread. You only make things harder for yourself by mixing threads into the game like this. 3 - your socket_callback doesn't _do_ anything??? You are supposed to catch the socket in there and subscribe to notifications from your event library on that sockets (and that specific action). 4 - you're using an event-based API so call libcurl when events trigger! NSPR is not an event library/API, epoll is though. Perhaps things would be easier if you'd explain more what exactly you want to accomplish and we can possibly help you get there... [1] = http://curl.haxx.se/libcurl/c/curl_multi_setopt.html#CURLMOPTTIMERFUNCTION -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] Addition of trailer headers in HTTP requests generated by libcurl
Hi, I attach a patch with some changes and a test case.
In the patch:
-- I copy my 'trailer_headers_buf' into the original buffer and I free
mine at the end.
(no leaks from 'trailer_headers_buf' anymore)
-- I added a bool variable 'done' so as to update 'data-req.upload_done' at its
original place.
0003-CURLOPT_TRAILERFUNCTION-support-chunked-encoding-req.patch
Description: Binary data
/***
* _ _ _
* Project ___| | | | _ \| |
* / __| | | | |_) | |
*| (__| |_| | _ | |___
* \___|\___/|_| \_\_|
*
* Copyright (C) 1998 - 2013, Daniel Stenberg, dan...@haxx.se, et al.
*
* This software is licensed as described in the file COPYING, which
* you should have received as part of this distribution. The terms
* are also available at http://curl.haxx.se/docs/copyright.html.
*
* You may opt to use, copy, modify, merge, publish, distribute and/or sell
* copies of the Software, and permit persons to whom the Software is
* furnished to do so, under the terms of the COPYING file.
*
* This software is distributed on an AS IS basis, WITHOUT WARRANTY OF ANY
* KIND, either express or implied.
*
***/
#include test.h
#include memdebug.h
/*
*
* This example shows an HTTP PUT (chuncked), with a trailer header sent at
* the end of the last chunk(zero-chunk).
*
*/
size_t filecode;
/* Trailer header callback function */
struct curl_slist * trailerheader_callback(CURL *handle, void *userdata) {
struct curl_slist *trailer_headers = NULL;
int *code = (int *)userdata;
const char *buf_eof = mytrailer: EOF;
const char *buf_error = mytrailer: error;
if(!(*code)) {
if((trailer_headers = curl_slist_append(trailer_headers,
buf_eof)) == NULL) {
fprintf(stderr, curl_slist_append() failed\n);
curl_easy_cleanup(handle);
curl_global_cleanup();
return TEST_ERR_MAJOR_BAD;
}
}
else {
if((trailer_headers = curl_slist_append(trailer_headers,
buf_error)) == NULL) {
fprintf(stderr, curl_slist_append() failed\n);
curl_easy_cleanup(handle);
curl_global_cleanup();
return TEST_ERR_MAJOR_BAD;
}
}
return trailer_headers;
}
/* Read callback function */
size_t read_callback(void *ptr, size_t size, size_t nmemb, void *stream) {
size_t retcode;
retcode = fread(ptr, size, nmemb, stream);
if(!retcode) {
if(ferror(stream))
filecode = 1;
if(feof(stream))
filecode = 0;
}
return retcode;
}
int test(char *URL)
{
CURL *curl;
CURLcode res = CURLE_OK;
FILE *hd_src;
int error;
struct curl_slist *hl = NULL;
struct curl_slist *headerlist=NULL;
const char *buf_1 = Transfer-Encoding: chunked;
const char *buf_2 = Trailer: mytrailer;
if (!libtest_arg2) {
fprintf(stderr, Usage: url file-to-upload\n);
return -1;
}
hd_src = fopen(libtest_arg2, rb);
if(NULL == hd_src) {
error = ERRNO;
fprintf(stderr, fopen() failed with error: %d %s\n,
error, strerror(error));
fprintf(stderr, Error opening file: %s\n, libtest_arg2);
return -2; /* if this happens things are major weird */
}
if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK) {
fprintf(stderr, curl_global_init() failed\n);
fclose(hd_src);
return TEST_ERR_MAJOR_BAD;
}
/* get a curl handle */
if ((curl = curl_easy_init()) == NULL) {
fprintf(stderr, curl_easy_init() failed\n);
curl_global_cleanup();
fclose(hd_src);
return TEST_ERR_MAJOR_BAD;
}
/* build a list of commands to pass to libcurl */
if ((hl = curl_slist_append(headerlist, buf_1)) == NULL) {
fprintf(stderr, curl_slist_append() failed\n);
curl_easy_cleanup(curl);
curl_global_cleanup();
fclose(hd_src);
return TEST_ERR_MAJOR_BAD;
}
if ((headerlist = curl_slist_append(hl, buf_2)) == NULL) {
fprintf(stderr, curl_slist_append() failed\n);
curl_slist_free_all(hl);
curl_easy_cleanup(curl);
curl_global_cleanup();
fclose(hd_src);
return TEST_ERR_MAJOR_BAD;
}
headerlist = hl;
/* enable uploading */
test_setopt(curl, CURLOPT_UPLOAD, 1L);
/* enable verbose */
test_setopt(curl, CURLOPT_VERBOSE, 1L);
/* specify target */
test_setopt(curl,CURLOPT_URL, URL);
/* pointer to pass to the file read function */
test_setopt(curl, CURLOPT_READDATA, hd_src);
/* read callback function */
test_setopt(curl, CURLOPT_READFUNCTION, read_callback);
/* trailer header callback function */
curl_easy_setopt(curl, CURLOPT_TRAILERFUNCTION, trailerheader_callback);
/* pointer to pass to the trailer header function */
curl_easy_setopt(curl, CURLOPT_TRAILERDATA, filecode);
/* Now
Re: Strange performance discrepancy with proxy
On Wed, 13 Mar 2013, Matt Kunze wrote: We have recently integrated libcurl into our application to provide the underlying HTTP transport (replacing WinINET, which is a huge improvement, thanks!). Nice. If you have anything additional to add to http://curl.haxx.se/libcurl/wininet.html about this transition and possibly some reasons for it, then do let us know! I'm seeing strange behavior where requests are _much_ faster when routed through a proxy (in this case the Fiddler diagnostic tool). Note that this is not a caching proxy, and all the requests are POSTs anyway, so I don't think that is the explanation. If I run the app with an HTTP_PROXY environment variable defined it is much faster than without - the code is the same in both cases. I've instrumented the code as well to display the progression of requests, and it appears to be the same in both cases except the proxied requests return much faster. That is very curious indeed. There's in fact no particular different code path within libcurl either! Using a proxy or a direct server will still use the same functions and code, leading me to suspect that you see this difference because of something in your network/setup that actually makes the proxy use the network better... It isn't very easy for me to start guessing any more details though. I assume you're talking about plain HTTP here? What libcurl version are you using on what Windows version? -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: how to use poll with libcurl
On Thu, 14 Mar 2013, jianqiu lv wrote: Recently I meet a problem the when using select after muti_curl_fdset, the maxfd is over 1024 in my process, and it is necessary to have such a value. maxfd over FD_SETSIZE is indeed a problem in the select()-using parts of the world... I search in libcurl's example( http://curl.haxx.se/libcurl/c/hiperfifo.html) , they have an example for how to use libevent with curl_multi_socket_action. I am not familiar with libevent nor have the plan to use libevent, since the socket fd in my process is not too much (less than 50,using poll which is O(n) is almost the same with using epoll O(1)), most all the fds are local files. In the curl project we decided early on to provide a select() oriented API since select() was then (and is still) the most widely available API for this kind of operation. Later on we added the socket_action API which allows the application to wait for sockets using any API at all that knows about sockets/file descriptors. Just a few versions ago we added curl_multi_wait() as a way to avoid the 1024 limit for the normal API by using poll() underneath but without exposing this to the outside. This made it even simpler for typical applications to wait for libcurl's and the application's own transfers. To me, it sounds like perhaps curl_multi_wait() would be the easiest way to solve your immediate problem. This project is a maintenance project ,so I don't want to change a lot of code to cause any side effect. So I think can I just use curl_multi_setopt set the callback function before curl_multi_add_handle and when the call back happen , I add the fd into pullfd array. Then I poll on the pullfd array to replace the select I used to call.But when my code execute it will stuck when multi curl happen. Does anyone have any idea why my code not working? Those little snippets weren't enough for me to figure out how everything works or not. Can you provide a complete example? -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] Addition of trailer headers in HTTP requests generated by libcurl
Chrysovaladis Datsios cdats...@gmail.com wrote: In the patch: ... (line 155 in patch): + trailer_headers_buf = malloc(headers_buf_size); + if(trailer_headers_buf == NULL) +return CURLE_BAD_FUNCTION_ARGUMENT; W/o looking at your code in details, why isn't this returning 'CURLE_OUT_OF_MEMORY'? --gv --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] Addition of trailer headers in HTTP requests generated by libcurl
W/o looking at your code in details, why isn't this returning 'CURLE_OUT_OF_MEMORY'? --gv According to Dan Fandrich: --- And I'm not sure CURLE_OUT_OF_MEMORY is the best error code here--the system is technically not out of memory. Maybe CURLE_BAD_FUNCTION_ARGUMENT? --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
calling multi-single.c example in an endless loop?
Hello,
starting from the multi-single.c example I try to convert this to something
like this:
curl_easy_setopt(http_handle, CURLOPT_URL, my-url);
while (1) {
..
rest of example code
..
}
Unfortunately I did not get this to work without also calling
curl_multi_init/curl_multi_cleanup inside the loop. This leeds to a
permanent reonnection of the tcp socket.
However, it works fine using the simple-interface (without socket
reconnection).
The reason I need to use the multi-interface is that I also want to add my
own file descriptors to the select loop to occasionaly do something else
while URL fetching (one url at a time, but always the same one) is still in
progress.
Sven
P.S.: This might look strange, but the url I'm about to fetch will delay on
the server side doing some kind of pseudo push mechanism, so this will not
kill the server.
--
The source code is not comprehensible
(found in bug section of man 8 telnetd on Redhat Linux)
/me is giggls@ircnet, http://sven.gegg.us/ on the Web
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() fails with Problem with the SSL CA cert (path? access rights?) after first time calling this routine
How that can be? Does libcurl also use openssl? From my code, I only invoke libcurl routines. Again the following output are the libcurl output by using CURLOPT_VERBOSE and CURLOPT_STDERR. When setting CURLOPT_SSL_VERIFYHOST to 1, we got the debug1.txt output, when setting CURLOPT_SSL_VERIFYHOST to 0, we get the debug.txt output [root@l2se0132 bin]# more /debug1.txt * About to connect() to l2se0060.lss.emc.com port 8443 (#0) * Trying 10.247.73.60... * Connected to l2se0060.lss.emc.com (10.247.73.60) port 8443 (#0) * successfully set certificate verify locations: * CAfile: /usr/yhuang/cert.pem CApath: none * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * Closing connection 0 [root@l2se0132 bin]# more /debug.txt * About to connect() to l2se0060.lss.emc.com port 8443 (#0) * Trying 10.247.73.60... * Connected to l2se0060.lss.emc.com (10.247.73.60) port 8443 (#0) * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * Closing connection 0 On Fri, Mar 15, 2013 at 3:45 AM, Daniel Stenberg dan...@haxx.se wrote: On Thu, 14 Mar 2013, cnm marketing wrote: * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small Please stop top-posting and full-quoting. My 3.2 seconds of googling on this topic lead to this: http://comments.gmane.org/**gmane.comp.encryption.openssl.**user/43777http://comments.gmane.org/gmane.comp.encryption.openssl.user/43777 -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: Hardwiring the CA cert bundle
I wanted to thank Guenter for answering my questions. After reading his first reply - I am now using the environment variable approach I added #define CURL_WANTS_CA_BUNDLE_ENV To curl_setup.h and then set the environment variable (via Control Panel - System - Advanced - blah blah blah) I my case CURL_CA_BUNDLE is set to C:\Projects\curl_certs\curl-ca-bundle.crt. I will also try his next solution and that is to use the API but with forward slashes like usual. I am a newbie to Windows programming and the documentation on haxx site has been superb. Thanks so much! Progress is sweet, Katie On Sun, Mar 3, 2013 at 5:54 PM, Guenter li...@gknw.net wrote: Hi Katie, Am 03.03.2013 22:10, schrieb Katie Scott: I can now compile the program so that it posts and xml file and receives the pdf as a response from the server - using https. To get it to work I built libcurl with static support for SSL (and SSH) why? If you *only* need https you also only need SSL support and not SSH. and added a #define that points to the cert bundle in the compilation of libcurl with: #define CURL_CA_BUNDLE C:\\Projects\\curl_certs That last bit is to work around a 'bug' in the Windows implementation of curl_easy_setopt(CURLOPT_**CAPATH, ... That is - when I defined the fully qualified filename to the ca-bundle in the static library - it worked. I am relieved but still puzzled as to why that should be a Windows issue. Thanks again for the clues. have you tried to use forward slashes with the ca-path? F.e.: curl_easy_setopt(CURLOPT_**CAPATH, c:/Projects/curl_certs) AFAICT the Windows file open APIs work also with forward slashes, and that saves you from escaping the backslashes ... Gün. --**--**--- List admin: http://cool.haxx.se/list/**listinfo/curl-libraryhttp://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/**etiquette.htmlhttp://curl.haxx.se/mail/etiquette.html --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() fails with Problem with the SSL CA cert (path? access rights?) after first time calling this routine
On 15-03-13 15:44, cnm marketing wrote: How that can be? Does libcurl also use openssl? libcurl is able to use 9 different SSL implementation as its SSL library for SSL connections. And yes, OpenSSL is one of them. From my code, I only invoke libcurl routines. Again the following output are the libcurl output by using CURLOPT_VERBOSE and CURLOPT_STDERR. When setting CURLOPT_SSL_VERIFYHOST to 1, we got the debug1.txt output, when setting CURLOPT_SSL_VERIFYHOST to 0, we get the debug.txt output I've never seen the error message * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small What SSL implementation is your libcurl using? How is the SSL stack build and how did you build libcurl? Oscar smime.p7s Description: S/MIME Cryptographic Signature --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() fails with Problem with the SSL CA cert (path? access rights?) after first time calling this routine
libcurl is able to use 9 different SSL implementation as its SSL library for SSL connections. And yes, OpenSSL is one of them. [cnm] Does libcurl statically link ssl or dynamically load those ssl? I've never seen the error message * * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * [cnm] If you believe this error message is NOT from libcurl itself, then I believe this error message is from the openssl routines that the libcurl invokes. The question is why openssl throw this error, and in what situation (from libcurl code) this error will be throwed from libcurl? What SSL implementation is your libcurl using? How is the SSL stack build and how did you build libcurl? [cnm] 1. I am not sure if I understand your first question, we use libcurl, if libcurl uses ssl, then we use whatever is on the system, in our case, we use openssl. 2. We use dlopen()/dlsym()/dlclose() to load libcurl library. This is the ONLY way that we can fit libcurl into our software layers. I need to check with other groups and see how many they use ssl in their layers. I'll get back to you on this. On Fri, Mar 15, 2013 at 10:44 AM, cnm marketing cnn.market...@gmail.comwrote: How that can be? Does libcurl also use openssl? From my code, I only invoke libcurl routines. Again the following output are the libcurl output by using CURLOPT_VERBOSE and CURLOPT_STDERR. When setting CURLOPT_SSL_VERIFYHOST to 1, we got the debug1.txt output, when setting CURLOPT_SSL_VERIFYHOST to 0, we get the debug.txt output [root@l2se0132 bin]# more /debug1.txt * About to connect() to l2se0060.lss.emc.com port 8443 (#0) * Trying 10.247.73.60... * Connected to l2se0060.lss.emc.com (10.247.73.60) port 8443 (#0) * successfully set certificate verify locations: * CAfile: /usr/yhuang/cert.pem CApath: none * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * Closing connection 0 [root@l2se0132 bin]# more /debug.txt * About to connect() to l2se0060.lss.emc.com port 8443 (#0) * Trying 10.247.73.60... * Connected to l2se0060.lss.emc.com (10.247.73.60) port 8443 (#0) * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * Closing connection 0 On Fri, Mar 15, 2013 at 3:45 AM, Daniel Stenberg dan...@haxx.se wrote: On Thu, 14 Mar 2013, cnm marketing wrote: * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small Please stop top-posting and full-quoting. My 3.2 seconds of googling on this topic lead to this: http://comments.gmane.org/**gmane.comp.encryption.openssl.**user/43777http://comments.gmane.org/gmane.comp.encryption.openssl.user/43777 -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() fails with Problem with the SSL CA cert (path? access rights?) after first time calling this routine
On Fri, 15 Mar 2013, cnm marketing wrote: [cnm] If you believe this error message is NOT from libcurl itself, then I believe this error message is from the openssl routines that the libcurl invokes. The question is why openssl throw this error, and in what situation (from libcurl code) this error will be throwed from libcurl? We _know_ it is from OpenSSL and I already pointed you to a very long discussion about when this error message appears and why it can happen and what to do about it. Here's the link again: http://comments.gmane.org/gmane.comp.encryption.openssl.user/43777 -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: Avoid write callbacks
* Dan Fandrich: It sounds like the API you want doesn't quite match the API libcurl provides, in which case some amount of code is going to be necessary to handle the mismatch. It doesn't have to be a lot. Take a look also at the existing C++ binding (or bindings?) to see if that might not better suit you. What binding options are there? curlpp appears to be mostly dead. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() fails with Problem with the SSL CA cert (path? access rights?) after first time calling this routine
What SSL implementation is your libcurl using? How is the SSL stack build and how did you build libcurl? [cnm] 1. I am not sure if I understand your first question, we use libcurl, if libcurl uses ssl, then we use whatever is on the system, in our case, we use openssl. 2. We use dlopen()/dlsym()/dlclose() to load libcurl library. This is the ONLY way that we can fit libcurl into our software layers. There are at least 3 different layers in our layer that uses openssl, we are not the first one to be loaded. On Fri, Mar 15, 2013 at 12:59 PM, cnm marketing cnn.market...@gmail.comwrote: libcurl is able to use 9 different SSL implementation as its SSL library for SSL connections. And yes, OpenSSL is one of them. [cnm] Does libcurl statically link ssl or dynamically load those ssl? I've never seen the error message * * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * [cnm] If you believe this error message is NOT from libcurl itself, then I believe this error message is from the openssl routines that the libcurl invokes. The question is why openssl throw this error, and in what situation (from libcurl code) this error will be throwed from libcurl? What SSL implementation is your libcurl using? How is the SSL stack build and how did you build libcurl? [cnm] 1. I am not sure if I understand your first question, we use libcurl, if libcurl uses ssl, then we use whatever is on the system, in our case, we use openssl. 2. We use dlopen()/dlsym()/dlclose() to load libcurl library. This is the ONLY way that we can fit libcurl into our software layers. I need to check with other groups and see how many they use ssl in their layers. I'll get back to you on this. On Fri, Mar 15, 2013 at 10:44 AM, cnm marketing cnn.market...@gmail.comwrote: How that can be? Does libcurl also use openssl? From my code, I only invoke libcurl routines. Again the following output are the libcurl output by using CURLOPT_VERBOSE and CURLOPT_STDERR. When setting CURLOPT_SSL_VERIFYHOST to 1, we got the debug1.txt output, when setting CURLOPT_SSL_VERIFYHOST to 0, we get the debug.txt output [root@l2se0132 bin]# more /debug1.txt * About to connect() to l2se0060.lss.emc.com port 8443 (#0) * Trying 10.247.73.60... * Connected to l2se0060.lss.emc.com (10.247.73.60) port 8443 (#0) * successfully set certificate verify locations: * CAfile: /usr/yhuang/cert.pem CApath: none * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * Closing connection 0 [root@l2se0132 bin]# more /debug.txt * About to connect() to l2se0060.lss.emc.com port 8443 (#0) * Trying 10.247.73.60... * Connected to l2se0060.lss.emc.com (10.247.73.60) port 8443 (#0) * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * Closing connection 0 On Fri, Mar 15, 2013 at 3:45 AM, Daniel Stenberg dan...@haxx.se wrote: On Thu, 14 Mar 2013, cnm marketing wrote: * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small Please stop top-posting and full-quoting. My 3.2 seconds of googling on this topic lead to this: http://comments.gmane.org/**gmane.comp.encryption.openssl.**user/43777http://comments.gmane.org/gmane.comp.encryption.openssl.user/43777 -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() fails with Problem with the SSL CA cert (path? access rights?) after first time calling this routine
error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small 1. When libcurl uses Diffie-Hellman, does libcurl hardcode the Diffie-Hellman key and the length? 2. Does libcurl have an option that allow libcurl user to re-set the Diffie-Hellman key length? We are still wondering (90% convinced) whether the error message has something to do with the openssl that is getting loaded from another layer. When the openssl is being loaded by another layer, libcurl is trying to use that openssl context and find the key size (set by libcurl) is too small compare to the key set by another layer. On Fri, Mar 15, 2013 at 2:00 PM, cnm marketing cnn.market...@gmail.comwrote: What SSL implementation is your libcurl using? How is the SSL stack build and how did you build libcurl? [cnm] 1. I am not sure if I understand your first question, we use libcurl, if libcurl uses ssl, then we use whatever is on the system, in our case, we use openssl. 2. We use dlopen()/dlsym()/dlclose() to load libcurl library. This is the ONLY way that we can fit libcurl into our software layers. There are at least 3 different layers in our layer that uses openssl, we are not the first one to be loaded. On Fri, Mar 15, 2013 at 12:59 PM, cnm marketing cnn.market...@gmail.comwrote: libcurl is able to use 9 different SSL implementation as its SSL library for SSL connections. And yes, OpenSSL is one of them. [cnm] Does libcurl statically link ssl or dynamically load those ssl? I've never seen the error message * * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * [cnm] If you believe this error message is NOT from libcurl itself, then I believe this error message is from the openssl routines that the libcurl invokes. The question is why openssl throw this error, and in what situation (from libcurl code) this error will be throwed from libcurl? What SSL implementation is your libcurl using? How is the SSL stack build and how did you build libcurl? [cnm] 1. I am not sure if I understand your first question, we use libcurl, if libcurl uses ssl, then we use whatever is on the system, in our case, we use openssl. 2. We use dlopen()/dlsym()/dlclose() to load libcurl library. This is the ONLY way that we can fit libcurl into our software layers. I need to check with other groups and see how many they use ssl in their layers. I'll get back to you on this. On Fri, Mar 15, 2013 at 10:44 AM, cnm marketing cnn.market...@gmail.comwrote: How that can be? Does libcurl also use openssl? From my code, I only invoke libcurl routines. Again the following output are the libcurl output by using CURLOPT_VERBOSE and CURLOPT_STDERR. When setting CURLOPT_SSL_VERIFYHOST to 1, we got the debug1.txt output, when setting CURLOPT_SSL_VERIFYHOST to 0, we get the debug.txt output [root@l2se0132 bin]# more /debug1.txt * About to connect() to l2se0060.lss.emc.com port 8443 (#0) * Trying 10.247.73.60... * Connected to l2se0060.lss.emc.com (10.247.73.60) port 8443 (#0) * successfully set certificate verify locations: * CAfile: /usr/yhuang/cert.pem CApath: none * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * Closing connection 0 [root@l2se0132 bin]# more /debug.txt * About to connect() to l2se0060.lss.emc.com port 8443 (#0) * Trying 10.247.73.60... * Connected to l2se0060.lss.emc.com (10.247.73.60) port 8443 (#0) * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * Closing connection 0 On Fri, Mar 15, 2013 at 3:45 AM, Daniel Stenberg dan...@haxx.se wrote: On Thu, 14 Mar 2013, cnm marketing wrote: * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small Please stop top-posting and full-quoting. My 3.2 seconds of googling on this topic lead to this: http://comments.gmane.org/**gmane.comp.encryption.openssl.** user/43777http://comments.gmane.org/gmane.comp.encryption.openssl.user/43777 -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() fails with Problem with the SSL CA cert (path? access rights?) after first time calling this routine
On Fri, Mar 15, 2013 at 7:59 PM, cnm marketing cnn.market...@gmail.com wrote: error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small 1. When libcurl uses Diffie-Hellman, does libcurl hardcode the Diffie-Hellman key and the length? 2. Does libcurl have an option that allow libcurl user to re-set the Diffie-Hellman key length? We are still wondering (90% convinced) whether the error message has something to do with the openssl that is getting loaded from another layer. When the openssl is being loaded by another layer, libcurl is trying to use that openssl context and find the key size (set by libcurl) is too small compare to the key set by another layer. libcurl does not fool around with certificate contents nor keys. The problem is in the certificate you are using which does not have a long enough Diffie-Hellman key. -- -=[Yang]=- --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] Addition of trailer headers in HTTP requests generated by libcurl
On Fri, Mar 15, 2013 at 02:07:47PM +0200, Chrysovaladis Datsios wrote: W/o looking at your code in details, why isn't this returning 'CURLE_OUT_OF_MEMORY'? --gv According to Dan Fandrich: --- And I'm not sure CURLE_OUT_OF_MEMORY is the best error code here--the system is technically not out of memory. Maybe CURLE_BAD_FUNCTION_ARGUMENT? That was in reference to the check if the given headers were too large to fit into the fixed-size buffer. If a malloc() fails, clearly the system is out of memory. Dan --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() fails with Problem with the SSL CA cert (path? access rights?) after first time calling this routine
*error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * libcurl does not fool around with certificate contents nor keys. [cnm] libcurl uses openssl, that error message comes from openssl. The problem is in the certificate you are using which does not have a long enough Diffie-Hellman key. [cnm] I don't understand what you are refering to, please give a bit more details. When we use libcurl, we don't set Diffie-Hellman key. We are getting the same Diffie-Hellman error message for both CURLOPT_SSL_VERIFYPEER to 1 and CURLOPT_SSL_VERIFYPEER to 0. Please refer to my previous email thread!! On Fri, Mar 15, 2013 at 2:59 PM, cnm marketing cnn.market...@gmail.comwrote: error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small 1. When libcurl uses Diffie-Hellman, does libcurl hardcode the Diffie-Hellman key and the length? 2. Does libcurl have an option that allow libcurl user to re-set the Diffie-Hellman key length? We are still wondering (90% convinced) whether the error message has something to do with the openssl that is getting loaded from another layer. When the openssl is being loaded by another layer, libcurl is trying to use that openssl context and find the key size (set by libcurl) is too small compare to the key set by another layer. On Fri, Mar 15, 2013 at 2:00 PM, cnm marketing cnn.market...@gmail.comwrote: What SSL implementation is your libcurl using? How is the SSL stack build and how did you build libcurl? [cnm] 1. I am not sure if I understand your first question, we use libcurl, if libcurl uses ssl, then we use whatever is on the system, in our case, we use openssl. 2. We use dlopen()/dlsym()/dlclose() to load libcurl library. This is the ONLY way that we can fit libcurl into our software layers. There are at least 3 different layers in our layer that uses openssl, we are not the first one to be loaded. On Fri, Mar 15, 2013 at 12:59 PM, cnm marketing cnn.market...@gmail.com wrote: libcurl is able to use 9 different SSL implementation as its SSL library for SSL connections. And yes, OpenSSL is one of them. [cnm] Does libcurl statically link ssl or dynamically load those ssl? I've never seen the error message * * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * [cnm] If you believe this error message is NOT from libcurl itself, then I believe this error message is from the openssl routines that the libcurl invokes. The question is why openssl throw this error, and in what situation (from libcurl code) this error will be throwed from libcurl? What SSL implementation is your libcurl using? How is the SSL stack build and how did you build libcurl? [cnm] 1. I am not sure if I understand your first question, we use libcurl, if libcurl uses ssl, then we use whatever is on the system, in our case, we use openssl. 2. We use dlopen()/dlsym()/dlclose() to load libcurl library. This is the ONLY way that we can fit libcurl into our software layers. I need to check with other groups and see how many they use ssl in their layers. I'll get back to you on this. On Fri, Mar 15, 2013 at 10:44 AM, cnm marketing cnn.market...@gmail.com wrote: How that can be? Does libcurl also use openssl? From my code, I only invoke libcurl routines. Again the following output are the libcurl output by using CURLOPT_VERBOSE and CURLOPT_STDERR. When setting CURLOPT_SSL_VERIFYHOST to 1, we got the debug1.txt output, when setting CURLOPT_SSL_VERIFYHOST to 0, we get the debug.txt output [root@l2se0132 bin]# more /debug1.txt * About to connect() to l2se0060.lss.emc.com port 8443 (#0) * Trying 10.247.73.60... * Connected to l2se0060.lss.emc.com (10.247.73.60) port 8443 (#0) * successfully set certificate verify locations: * CAfile: /usr/yhuang/cert.pem CApath: none * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * Closing connection 0 [root@l2se0132 bin]# more /debug.txt * About to connect() to l2se0060.lss.emc.com port 8443 (#0) * Trying 10.247.73.60... * Connected to l2se0060.lss.emc.com (10.247.73.60) port 8443 (#0) * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small * Closing connection 0 On Fri, Mar 15, 2013 at 3:45 AM, Daniel Stenberg dan...@haxx.sewrote: On Thu, 14 Mar 2013, cnm marketing wrote: * error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small Please stop top-posting and full-quoting. My 3.2 seconds of googling on this topic lead to this: http://comments.gmane.org/**gmane.comp.encryption.openssl.** user/43777http://comments.gmane.org/gmane.comp.encryption.openssl.user/43777 -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() fails with Problem with the SSL CA cert (path? access rights?) after first time calling this routine
On 15-03-13 21:26, cnm marketing wrote: /error:0506706E:Diffie-Hellman routines:GENERATE_KEY:key size too small / libcurl does not fool around with certificate contents nor keys. [cnm] libcurl uses openssl, that error message comes from openssl. The problem is in the certificate you are using which does not have a long enough Diffie-Hellman key. [cnm] I don't understand what you are refering to, please give a bit more details. When we use libcurl, we don't set Diffie-Hellman key. We are getting the same Diffie-Hellman error message for both CURLOPT_SSL_VERIFYPEER to 1 and CURLOPT_SSL_VERIFYPEER to 0. Please refer to my previous email thread!! If you desire assistance in this matter, it would be helpful to follow the email thread composition guide lines (no top posting) and have a look at the link presented by Daniel as a first reply in the thread. May I also point out that there are no other options than CURLOPT_SSL_VERIFYPEER 0 and 1. We can safely rule out this to be of any factor in this matter. The source of the problem is described on the page earlier, at the bottom of the page there are the first hints about possible solutions even of which I give it a very high likely hood that this is exactly what you are facing here. With respect to the dlopen()/dlsym()/dlclose() being the only way to load libcurl, I think this is sheer a design choice. smime.p7s Description: S/MIME Cryptographic Signature --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() fails with Problem with the SSL CA cert (path? access rights?) after first time calling this routine
On Fri, Mar 15, 2013 at 9:26 PM, cnm marketing cnn.market...@gmail.com wrote: The problem is in the certificate you are using which does not have a long enough Diffie-Hellman key. [cnm] I don't understand what you are refering to, please give a bit more details. Assuming that the certificate file you are using is '/usr/temp/cert.pem' and that you have openssl installed you can verify DH parameters running following command: openssl dhparam -in /usr/temp/cert.pem -noout -text If it has DH parameters you should see output similar to: PKCS#3 DH Parameters: (1024 bit) prime: [...] generator: 2 (0x2) If yours does not have DH Parameters or these are less than (1024 bit), ask whoever provided you with cert.pem to give you another PEM file which includes 1024 bit DH parameters. HTH -- -=[Yang]=- --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: vms_show hack explained
On 3/14/2013 1:23 PM, Yang Tse wrote: On Thu, Mar 14, 2013, John E. Malmberg wb8...@qsl.net wrote: It is more palatable for me the inclusion of source code functions into lib/setup-vms.h as long as the following are all met: 1) curlbuild.h is just a copy of curlbuild.h.dist or it is a true equivalent of a properly generated one based on curlbuild.h.in. The resulting one shall be auto contained and can not depend on HAVE_* macros. Notice that this file is used when building the library and also when using it already installed with no config.h around. The daily tarball that I used earlier did not have a curlbuild.h file in it, so I had to generate it. In reading the curlbuild.h.in file several of the values in the file depended on what had previously been generated in the curl_config.h file and I did not see any cases where curlbuild.h was used with out curl_config.h being include prior to it. For implementing the build_curlbuild_h.com procedure, I then had the choice of: 1. Having the curlbuild.h use definitions from curl_config.h 2. Duplicate some complex code from the config_h.com procedure. 3. Read or parse the curl_config.h file As it is, the values for VMS in curlbuild.h in the March 14 daily tarball are incorrect for some !__VAX VMS releases. For !__VAX the size of off_t is only 32 bits for older versions of VMS and is only 64 bits on more recent versions and requires that support to be requested at compile time. I had changed the VMS build procedure to default to asking for that support on non-VAX, but just because it asked for it, does not guarantee that it will get it. The procedure to generate the curl_config.h does set the off_t size based on what the platform it is run on actually supports. 2) Generated or hand-crafted config.h does not include a single header file. The config_h.com is a generic procedure shared by multiple projects. While it knows how to generate most of what is in the config.h file, it does not know how to get everything correct. So it includes a config_vms.h file that is built by a project specific command file that knows what config_h.com was not able to calculate for it. For example, the config_h.com reads the configure script to determine how to set some symbols that it did not find in the header files. It takes the first assignment it finds, which is not the default that we want for many of the settings. So a command procedure is run first based on the build options selected and it generates the config_vms.h that combined with the auto-generated config.h is correct. That procedure also renames the config.h file to curl_config.h. 3) No change is done to the placement where header files curlbuild.h and setup-vms.h are included. Some things need to be in the order that I set them or the correct wrapper will not be used. I will try to check the latest daily build this weekend. 4) Any special VMS need in other source files protected with 'ifdef __VMS' guards instead of 'ifdef HAVE_*' ones. Although with what you have in mind it seems near none will remain in some future. That is the plan. For now, we should focus on making sure that 7.30.0 will build out of the box. I would like to see that. Regards, -John wb8...@qsl.net Personal Opinion Only --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curl_easy_perform() failed: Couldn't connect to server
Thanks Dan. This worked. /anil. On Fri, Mar 15, 2013 at 3:21 AM, Dan Fandrich d...@coneharvesters.comwrote: On Fri, Mar 15, 2013 at 12:07:44AM -0400, Anil J wrote: Just to test a new scenario, I changed the URL to a HTTP web server running on the same host, but still same error persist. I can fetch the web server from the firefox browser, but not from the curl program. Looks like something obvious is missing. Can somebody help? [...] --2013-03-14 18:23:58-- http://gmail.com/ Resolving localhost (localhost)... 127.0.0.1 Connecting to localhost (localhost)|127.0.0.1|:8080... failed: Connection refused. It looks like you have a proxy configured. If you're not doing it explicitly in your app, then you probably have an environment variable set. See if env | grep -i proxy shows such a variable, clear it and try again! Or, see if Firefox has a different proxy server configured and use that one instead. Dan --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: vms_show hack explained
On Sat, Mar 16, 2013, John E. Malmberg wrote: The daily tarball that I used earlier did not have a curlbuild.h file in it, so I had to generate it. You might have been unfortunate back then fetching a badly broken daily snapshot. I've just verified that curl-7.30.0-20130314.tar certainly has curlbuild.h in it. $ find . -name curlbuild.h ./curl-7.30.0-20130314/include/curl/curlbuild.h In reading the curlbuild.h.in file several of the values in the file depended on what had previously been generated in the curl_config.h file and I did not see any cases where curlbuild.h was used with out curl_config.h being include prior to it. ./curl-7.30.0-20130314/include/curl/curl.h (line 34) #include curlbuild.h Included also by external API when no curl_config.h exists once the library is built and installed. As it is, the values for VMS in curlbuild.h in the March 14 daily tarball are incorrect for some !__VAX VMS releases. For !__VAX the size of off_t is only 32 bits for older versions of VMS and is only 64 bits on more recent versions and requires that support to be requested at compile time. Even if it sounds funny curl_off_t is not off_t. Read ./curl-7.30.0-20130314/lib/README.curl_off_t and 'Note 2' at the beginning of ./curl-7.30.0-20130314/include/curl/curlbuild.h I had changed the VMS build procedure to default to asking for that support on non-VAX, but just because it asked for it, does not guarantee that it will get it. The procedure to generate the curl_config.h does set the off_t size based on what the platform it is run on actually supports. If you base or set curl_off_t on off_t you will get a wrongly built libcurl. 2) Generated or hand-crafted config.h does not include a single header file. The config_h.com is a generic procedure shared by multiple projects. While it knows how to generate most of what is in the config.h file, it does not know how to get everything correct. Notice that even when all projects might be using autotools and all might define certain preprocessor macro such as HAVE_* each project is free to impose the conditions upon when it is ok to define that macro. Some might define it doing only some naive checks and others might do very thorough checks in order to define it. So, the same macro definition isn't actually shareable among projects. And not even whether it gets defined or not. So it includes a config_vms.h file that is built by a project specific command file that knows what config_h.com was not able to calculate for it. For example, the config_h.com reads the configure script to determine how to set some symbols that it did not find in the header files. It takes the first assignment it finds, which is not the default that we want for many of the settings. So a command procedure is run first based on the build options selected and it generates the config_vms.h that combined with the auto-generated config.h is correct. That procedure also renames the config.h file to curl_config.h. You are going to have lots of fun. But If your priority is being able to build a proper libcurl it is not going to be a short term target. 3) No change is done to the placement where header files curlbuild.h and setup-vms.h are included. Some things need to be in the order that I set them or the correct wrapper will not be used. I will try to check the latest daily build this weekend. Trust me, concentrate first in achieving an auto contained curlbuild.h and the rest will fit nicely. -- -=[Yang]=- --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
