Serverside disconnection vs. simple Closing connection #n
Hi All,
I¹m trying to use curl easy to http_get a web camera stream. Everything
works perfectly except one thing.
If the server in the camera closes the connection curl only logs ³Closing
connection #n² and finishes the transfer from curl_easy_perform with error
code CURLE_OK :-O
This makes impossible to detect server side interruption.
(technically now we could treat every return from curl_easy_perform as a
server side disconnection, but in other cases the problem is a real thing)
I compiled a debug version of libcurl (7.19.3) and found in readwrite_data
function in transfer.c this code
At line 422:
if(0 = nread) {
/* if we receive 0 or less here, the server closed the connection
and we bail out from this! */
DEBUGF(infof(data, nread = 0, server closed connection,
bailing\n));
k-keepon = ~KEEP_READ;
break;
}
This shows that curl breaks the read loop without setting returning result
value and returns simply CURLE_OK at the end of the function if server side
disconnect happened.
I'm really wondering is there any special reason why that function not set
result error to something like CURLE_GOT_NOTHING or CURL_PEER_DISCONNECTED
(which is totally not existing in the error code defined list, I know) to
let inform the caller the real reason of closing the connection.
I'm also interesting it if I would modify the code mentioned above to return
with an error result, would it case any problem in any cases later.
Thank you in advance.
Bests
Hofi
[no subject]
Hi,
I'm using libcurl 7.19.4 with Visual C++ 2008 express edition
(9.0.30729.1 SP) on Windows Vista Business. I'm trying to get the html
of an online dictionary page, but:
- the page has some sort of
redirect (I'm not able to block it from browsers)
- libcurl gets only a
partial and broken html
In my code, I define a webpage() method that,
given an url, retrieves the html code.
I directly took the code from an
example on the curl website.
Here's it:
static void *myrealloc(void
*ptr, size_t size);
static size_t WriteMemoryCallback(void *ptr,
size_t size, size_t nmemb, void *data);
struct MemoryStruct {
char*
memory;
size_t size;
};
static void *myrealloc(void *ptr, size_t
size)
{
if(ptr) return realloc(ptr, size);
else return malloc
(size);
}
static size_t WriteMemoryCallback(void *ptr, size_t size,
size_t nmemb, void *data)
{
size_t realsize = size * nmemb;
struct
MemoryStruct *mem = (struct MemoryStruct*)data;
mem-memory =
(char*)myrealloc(mem-memory, mem-size + realsize + 1);
if(mem-
memory) {
memcpy((mem-memory[mem-size]), ptr, realsize);
mem-
size += realsize;
mem-memory[mem-size] = 0;
}
return
realsize;
}
string webpage(const string url)
{
CURL
*curl_handle;
struct MemoryStruct chunk;
chunk.memory = NULL;
chunk.size = 0;
curl_global_init(CURL_GLOBAL_ALL);
curl_handle = curl_easy_init();
curl_easy_setopt(curl_handle,
CURLOPT_URL, url.c_str());
curl_easy_setopt(curl_handle,
CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
curl_easy_setopt
(curl_handle, CURLOPT_WRITEDATA, (void*)chunk);
curl_easy_setopt
(curl_handle, CURLOPT_USERAGENT, libcurl-agent/1.0);
curl_easy_perform(curl_handle);
curl_easy_cleanup(curl_handle);
string s;
if(chunk.memory)
{
s = string(chunk.memory);
free(chunk.memory);
}
curl_global_cleanup();
return s;
}
Then I simply call it with the url http://old.demauroparavia.
it//@rozzo
string html = webpage(http://old.demauroparavia.
it//@rozzo);
but here's what it retrieves(I include it in an html
comment, hope this is enough to block html rendering):
!--
/div
!-- fine contenuto --
!-- inizio menu di navigazione --
div id=
menu
h2Per saperne di piugrave;/h2
ol
lia href=
avvertenze title=consulta questa sezione per meglio comprendere le
voci del dizionarioAvvertenze per la consultazione /a/li
lia
href=avanzata title=consulta le indicazioni per effettuare ricerche
complesseRicerca avanzata /a/lilia href=abbreviazioni title=
consulta l'elenco delle abbreviazioni presenti nel dizionario
Abbreviazioni /a/lilia href=quadri/ title=consulta la tavola
delle coniugazioniTavole delle coniugazioni dei verbi /a/lilia
href=lemmario title=consulta il lemmario completoLemmario completo
/a/lilia href=consultati title=consulta la statistica dei 500
lemmi piugrave; consultatiI lemmi piugrave; consultati /a/li
/ol
h2Collegamenti/h2
ol
lia href=http://www.paravia.
it/vetrina_diz.php title=Vai alla pagina dei dizionari cartacei sul
sito ParaviaI dizionari Paravia /a/li
lia href=http:
//oxfordparavia.it title=Vai al Dizionario italiano-inglese Oxford
Paravia ConciseDizionario italiano-inglesebr /
Oxford Paravia
Concise /a/li
lia href=http://dizionariodai.it; title=Vai al
Dizionario DAI ParaviaDizionario di apprendimentobr /della lingua
inglese /a/li
lia href=http://www.ldoceonline.com/; title=Vai
al Dizionario Inglese LongmanDizionario inglese Longman /a/li
lia href=http://www.mozillaitalia.org/searchplugins/; title=Vai al
sito ufficiale Mozilla per scaricare il Plugin De Mauro per FireFoxDe
Mauro plugin per FireFox /a/li
!--lia href=http://www.apple.
com/downloads/dashboard/search/demauro.html title=Vai al sito Apple e
scarica il Widget per Dashboard su Mac OS XDashboard Widget per Mac
/a/li--
/ol
h2Scrivi alla redazione/h2
ol
lia href=
email title=vai alla pagina dalla quale puoi spedire una email alla
redazioneEgrave; possibile contattare la redazione per chiarimenti o
per segnalare eventuali errori /a/li
/ol
!-- Include the
Google Friend Connect javascript library. --
script type=
text/javascript src=http://www.google.
Re: Serverside disconnection vs. simple Closing connection #n
On Wed, 22 Apr 2009, Istvan Hoffmann wrote: I'm really wondering is there any special reason why that function not set result error to something like CURLE_GOT_NOTHING or CURL_PEER_DISCONNECTED (which is totally not existing in the error code defined list, I know) to let inform the caller the real reason of closing the connection. I'm also interesting it if I would modify the code mentioned above to return with an error result, would it case any problem in any cases later. The reason is that in most cases the server closure is not an error, and for the cases where the close is an error we are able to detect it otherwise. How is your case actually an error HTTP-wise? What response headers does libcurl get before the stream starts? -- / daniel.haxx.se
RPATH incorrect with custom SSL builds and 7.19.4
After building Curl 7.19.4 and fixing the configure issue around using a custom openssl build, I found that the resulting binary was not executable. This is because the compiled in rpath is missing the path to the custom build of OpenSSL: [bu...@build01 .libs]$ readelf -d curl 0x000f (RPATH) Library rpath: [/opt/zimbra/curl-7.19.4/lib:/opt/zimbra/heimdal-1.2.1/lib] The RPATH should also have included /opt/zimbra/openssl-0.9.8k/lib as it did in previous releases. The end result is: [zim...@freelancer bin]$ ./curl ./curl: error while loading shared libraries: libssl.so.0.9.8: cannot open shared object file: No such file or directory Even though the Heimdal libraries (since they are correctly built) find their OpenSSL lib dependencies just fine. [zim...@freelancer bin]$ ldd curl libcurl.so.4 = /opt/zimbra/curl-7.19.4/lib/libcurl.so.4 (0x2b6d55c1a000) libidn.so.11 = /usr/lib64/libidn.so.11 (0x0032b3e0) libssl.so.0.9.8 = not found librt.so.1 = /lib64/librt.so.1 (0x0032b520) libgssapi.so.2 = /opt/zimbra/heimdal-1.2.1/lib/libgssapi.so.2 (0x2b6d55e7) libheimntlm.so.0 = /opt/zimbra/heimdal-1.2.1/lib/libheimntlm.so.0 (0x2b6d5609f000) libkrb5.so.25 = /opt/zimbra/heimdal-1.2.1/lib/libkrb5.so.25 (0x2b6d562a5000) libhx509.so.4 = /opt/zimbra/heimdal-1.2.1/lib/libhx509.so.4 (0x2b6d56571000) libwind.so.0 = /opt/zimbra/heimdal-1.2.1/lib/libwind.so.0 (0x2b6d567b5000) libdl.so.2 = /lib64/libdl.so.2 (0x0032b3a0) libasn1.so.8 = /opt/zimbra/heimdal-1.2.1/lib/libasn1.so.8 (0x2b6d569dd000) libcom_err.so.1 = /opt/zimbra/heimdal-1.2.1/lib/libcom_err.so.1 (0x2b6d56c61000) libcrypto.so.0.9.8 = not found libroken.so.18 = /opt/zimbra/heimdal-1.2.1/lib/libroken.so.18 (0x2b6d56e65000) libcrypt.so.1 = /lib64/libcrypt.so.1 (0x0032b5e0) libresolv.so.2 = /lib64/libresolv.so.2 (0x0032b720) libz.so.1 = /usr/lib64/libz.so.1 (0x0032b4a0) libpthread.so.0 = /lib64/libpthread.so.0 (0x0032b420) libc.so.6 = /lib64/libc.so.6 (0x0032b360) libssl.so.0.9.8 = /opt/zimbra/openssl-0.9.8k/lib/libssl.so.0.9.8 (0x2b6d5707a000) libcrypto.so.0.9.8 = /opt/zimbra/openssl-0.9.8k/lib/libcrypto.so.0.9.8 (0x2b6d572c6000) /lib64/ld-linux-x86-64.so.2 (0x0032b320) --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: RPATH incorrect with custom SSL builds and 7.19.4
--On Wednesday, April 22, 2009 2:28 PM -0700 Quanah Gibson-Mount
qua...@zimbra.com wrote:
After building Curl 7.19.4 and fixing the configure issue around using a
custom openssl build, I found that the resulting binary was not
executable. This is because the compiled in rpath is missing the path to
the custom build of OpenSSL:
In particular, the relink command for curl seems to be wrong:
curl:relink_command=(cd
/home/build/p4/main/ThirdParty/curl/curl-7.19.4/src; { test -z
\\${LIBRARY_PATH+set}\ || unset LIBRARY_PATH || { LIBRARY_PATH=; export
LIBRARY_PATH; }; }; { test -z \\${COMPILER_PATH+set}\ || unset
COMPILER_PATH || { COMPILER_PATH=; export COMPILER_PATH; }; }; { test -z
\\${GCC_EXEC_PREFIX+set}\ || unset GCC_EXEC_PREFIX || { GCC_EXEC_PREFIX=;
export GCC_EXEC_PREFIX; }; }; { test -z \\${LD_RUN_PATH+set}\ || unset
LD_RUN_PATH || { LD_RUN_PATH=; export LD_RUN_PATH; }; }; { test -z
\\${LD_LIBRARY_PATH+set}\ || unset LD_LIBRARY_PATH || { LD_LIBRARY_PATH=;
export LD_LIBRARY_PATH; }; };
PATH=/usr/kerberos/sbin:/usr/local/mysql/bin:/usr/local/ant/bin:/usr/local/java/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/build/bin:/usr/local/p4/bin;
export PATH; gcc -O2 -g0 -Wno-system-headers -o \$progdir/\$file main.o
hugehelp.o urlglob.o writeout.o writeenv.o getpass.o homedir.o curlutil.o
strtoofft.o strdup.o rawstr.o -L/opt/zimbra/heimdal-1.2.1/lib
-L/opt/zimbra/openssl-0.9.8k/lib -L/usr/lib64/lib ../lib/.libs/libcurl.so
/usr/lib/libidn.so -lssl -lrt /opt/zimbra/heimdal-1.2.1/lib/libgssapi.so
/opt/zimbra/heimdal-1.2.1/lib/libheimntlm.so
/opt/zimbra/heimdal-1.2.1/lib/libkrb5.so
/opt/zimbra/heimdal-1.2.1/lib/libhx509.so
/opt/zimbra/heimdal-1.2.1/lib/libwind.so -ldl
/opt/zimbra/heimdal-1.2.1/lib/libasn1.so
/opt/zimbra/heimdal-1.2.1/lib/libcom_err.so -lcrypto
/opt/zimbra/heimdal-1.2.1/lib/libroken.so -lcrypt -lresolv -lz -pthread
-Wl,-rpath -Wl,/home/build/p4/main/ThirdParty/curl/curl-7.19.4/lib/.libs
-Wl,-rpath -Wl,/opt/zimbra/heimdal-1.2.1/lib -Wl,-rpath
-Wl,/opt/zimbra/curl-7.19.4/lib -Wl,-rpath
-Wl,/opt/zimbra/heimdal-1.2.1/lib)
The last bit there should have also had -Wl,-rpath
-Wl,/opt/zimbra/openssl-0.9.8k/lib
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
Re: RPATH incorrect with custom SSL builds and 7.19.4
On Wed, 22 Apr 2009, Quanah Gibson-Mount wrote: In particular, the relink command for curl seems to be wrong: curl:relink_command=(cd /home/build/p4/main/ThirdParty/curl/curl-7.19.4/src; So 7.19.3 did this right and 7.19.4 does not? It feels like this problem is related to my switch to libtool 2.2.6 which was done between those two versions, and libtool is responsible for that line above... -- / daniel.haxx.se
Visual Studio 2008 Solution for Curl
I wanted to make the solution files available for those who use Microsoft
Visual Studio 2008. I spent the morning figuring out why VS9 wouldn't compile
the solution after it auto-upgraded the VS6 workspace. libcurl project wasn't
outputing to the correct directory (curlsrc was expecting
Lib-Release/Lib-Debug, not Release/Debug). libcurl project wasn't naming output
properly in the debug builds (always would compile as libcurl.lib regardless of
build; curlsrc was expecting libcurld.lib on debug builds). Lastly, I didn't
find any documentation on where to define USE_WINDOWS_SSPI. In VS9, you go to
the project properties of libcurl and add USE_WINDOWS_SSPI to the preprocessor
definitions.
I'm new to open source and not exactly sure how stuff gets shared and updated.
I would expect that these files would get dropped onto CVS by if given the
thumbs up. Hopefully this email will go through with attachments.
vc9curl.sln in /curl
curlsrc.vcproj in /curl/src
libcurl.vcproj in /curl/lib
_
Rediscover Hotmail®: Get e-mail storage that grows with you.
http://windowslive.com/RediscoverHotmail?ocid=TXT_TAGLM_WL_HM_Rediscover_Storage2_042009
Microsoft Visual Studio Solution File, Format Version 10.00
# Visual Studio 2008
Project({8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}) = libcurl,
lib\libcurl.vcproj, {87EE9DA4-DE1E-4448-8324-183C98DCA588}
EndProject
Project({8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}) = curlsrc,
src\curlsrc.vcproj, {9B008335-54D1-46A6-80F7-0F32BC785421}
ProjectSection(ProjectDependencies) = postProject
{87EE9DA4-DE1E-4448-8324-183C98DCA588} =
{87EE9DA4-DE1E-4448-8324-183C98DCA588}
EndProjectSection
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Win32 = Debug|Win32
Release|Win32 = Release|Win32
using libcurl DLL Debug|Win32 = using libcurl DLL Debug|Win32
using libcurl DLL Release|Win32 = using libcurl DLL
Release|Win32
using libcurl LIB Debug|Win32 = using libcurl LIB Debug|Win32
using libcurl LIB Release|Win32 = using libcurl LIB
Release|Win32
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.Debug|Win32.ActiveCfg =
Debug|Win32
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.Debug|Win32.Build.0 =
Debug|Win32
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.Release|Win32.ActiveCfg
= Release|Win32
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.Release|Win32.Build.0 =
Release|Win32
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.using libcurl DLL
Debug|Win32.ActiveCfg = Debug|Win32
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.using libcurl DLL
Debug|Win32.Build.0 = Debug|Win32
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.using libcurl DLL
Release|Win32.ActiveCfg = Release|Win32
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.using libcurl DLL
Release|Win32.Build.0 = Release|Win32
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.using libcurl LIB
Debug|Win32.ActiveCfg = Debug|Win32
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.using libcurl LIB
Debug|Win32.Build.0 = Debug|Win32
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.using libcurl LIB
Release|Win32.ActiveCfg = Release|Win32
{87EE9DA4-DE1E-4448-8324-183C98DCA588}.using libcurl LIB
Release|Win32.Build.0 = Release|Win32
{9B008335-54D1-46A6-80F7-0F32BC785421}.Debug|Win32.ActiveCfg =
using libcurl DLL Debug|Win32
{9B008335-54D1-46A6-80F7-0F32BC785421}.Debug|Win32.Build.0 =
using libcurl DLL Debug|Win32
{9B008335-54D1-46A6-80F7-0F32BC785421}.Release|Win32.ActiveCfg
= using libcurl LIB Release|Win32
{9B008335-54D1-46A6-80F7-0F32BC785421}.Release|Win32.Build.0 =
using libcurl LIB Release|Win32
{9B008335-54D1-46A6-80F7-0F32BC785421}.using libcurl DLL
Debug|Win32.ActiveCfg = using libcurl DLL Debug|Win32
{9B008335-54D1-46A6-80F7-0F32BC785421}.using libcurl DLL
Debug|Win32.Build.0 = using libcurl DLL Debug|Win32
{9B008335-54D1-46A6-80F7-0F32BC785421}.using libcurl DLL
Release|Win32.ActiveCfg = using libcurl DLL Release|Win32
{9B008335-54D1-46A6-80F7-0F32BC785421}.using libcurl DLL
Release|Win32.Build.0 = using libcurl DLL Release|Win32
{9B008335-54D1-46A6-80F7-0F32BC785421}.using libcurl LIB
Debug|Win32.ActiveCfg = using libcurl LIB Debug|Win32
{9B008335-54D1-46A6-80F7-0F32BC785421}.using libcurl LIB
Debug|Win32.Build.0 = using libcurl LIB Debug|Win32
{9B008335-54D1-46A6-80F7-0F32BC785421}.using libcurl LIB
Release|Win32.ActiveCfg = using libcurl LIB Release|Win32
[PATCH] segfault when explicitly setting Host header and using CURLOPT_FOLLOWLOCATION
Hello, We're running into a segfault when setting an explicit Host: header with CURLOPT_HTTPHEADER and CURLOPT_FOLLOWLOCATION is set to 1. The bug occurs when the server returns a redirect and curl is trying to decide whether to keep track of the original host name for cookies. The line of code that crashes is in Curl_http(). It is trying to string-compare data-state.first_host, unfortunately data-state.first_host is null at that point because this_is_a_follow is true at the time we called Curl_http_connect(). In the other place we check the first_host field (in http_output_auth()), we first check if it is null. So I think the null-check was inadvertently left off. I've attached a patch. This patch seems to fix the problem. John Whaley libcurl_nullptr_host_followlocation.diff Description: Binary data
Re: About libcurl on Solaris 10
On Wed, Apr 22, 2009 at 9:34 PM, CHEN Xiaolei A xiaolei.a.c...@alcatel-sbell.com.cn wrote: Hello everyone, Nice to see you. I am a new comer to this maillist, I hope to study how to use libcurl to implement SFTP and make friends with you! Just as the subject said, now i am working on sun solaris 10 platform, and want to download libcurl for solaris. But I can not find a corresponding version on url: *http://curl.haxx.se/dlwiz/* http://curl.haxx.se/dlwiz/. Do you know where could i download the libcurl for solaris 10? Thanks a lot! You could download the latest source and compile it. A full installation of Solaris 10 should include GCC, probably in /usr/sfw/bin. Ralph Mitchell
RE: About libcurl on Solaris 10
Get it, thanks a lot! From: curl-library-boun...@cool.haxx.se [mailto:curl-library-boun...@cool.haxx.se] On Behalf Of Ralph Mitchell Sent: 2009年4月23日 11:13 To: libcurl development Subject: Re: About libcurl on Solaris 10 On Wed, Apr 22, 2009 at 9:34 PM, CHEN Xiaolei A xiaolei.a.c...@alcatel-sbell.com.cn wrote: Hello everyone, Nice to see you. I am a new comer to this maillist, I hope to study how to use libcurl to implement SFTP and make friends with you! Just as the subject said, now i am working on sun solaris 10 platform, and want to download libcurl for solaris. But I can not find a corresponding version on url: http://curl.haxx.se/dlwiz/ http://curl.haxx.se/dlwiz/ . Do you know where could i download the libcurl for solaris 10? Thanks a lot! You could download the latest source and compile it. A full installation of Solaris 10 should include GCC, probably in /usr/sfw/bin. Ralph Mitchell
Re: About libcurl on Solaris 10
On Wed, Apr 22, 2009 at 22:34, CHEN Xiaolei A xiaolei.a.c...@alcatel-sbell.com.cn wrote: Hello everyone, Nice to see you. I am a new comer to this maillist, I hope to study how to use libcurl to implement SFTP and make friends with you! Just as the subject said, now i am working on sun solaris 10 platform, and want to download libcurl for solaris. But I can not find a corresponding version on url: http://curl.haxx.se/dlwiz/. Do you know where could i download the libcurl for solaris 10? Thanks a lot! CURL has been in Solaris/Nevada since Nevada build 72. 7.19.0 is in OpenSolaris 2008.11, and CURL 7.19.4 will be in OpenSolaris.2009.06 (to be released sometime in June). --Stefan -- Stefan Teleman KDE e.V. stefan.tele...@gmail.com
