Re: sftp download issue (Daniel Stenberg)
Hi,
As i had posted earlier,
Do you always get this same error (and byte amount) no matter how large
file
you get? If not, do you get the same error and byte amount for the same
file
if you retry the exact same transfer?
Eh, so does it work for _any_ sftp files at all? If so, which?
Even if i try to download a 2MB file with sftp protocol, sometimes it will
download completely and sometimes it will show the error : **curl: (18)
transfer closed with 574 bytes
remaining to read*. So, this problem is arising for smaller file sizes
also (sometimes). No matter how large the file is , i will get the same
error.
Libraries: libssh2-1.2.9
I am using Curl-7.21.7 package to generate the libcurl on the 64-bit solaris
machine, here in the file lib/sendf.c, there is a piece of code having:
/* If session can pipeline, check connection buffer */
if(pipelining) {
size_t bytestocopy = CURLMIN(conn-buf_len - conn-read_pos,
sizerequested);
/* Copy from our master buffer first if we have some unread data there*/
if(bytestocopy 0) {
memcpy(buf, conn-master_buffer + conn-read_pos, bytestocopy);
conn-read_pos += bytestocopy;
conn-bits.stream_was_rewound = FALSE;
*n = (ssize_t)bytestocopy;
return CURLE_OK;
}
/* If we come here, it means that there is no data to read from the
buffer,
* so we read from the socket */
bytesfromsocket = CURLMIN(sizerequested, BUFSIZE * sizeof (char));
buffertofill = conn-master_buffer;
}
else {
bytesfromsocket = CURLMIN((long)sizerequested,
conn-data-set.buffer_size ?
conn-data-set.buffer_size : BUFSIZE);
/* * bytesfromsocket = 1;* */
/* *OR* */
/* * bytesfromsocket = 16384(=BUFSIZE);* */
buffertofill = buf;
}
See the else part, where bytesfromsocket is assigned with some values,
normally it will have value like 16384 bytes each time, if the remaining
file size to be downloaded is large. Once the remaining file size to be
downloaded is between 16384 and 1, bytesfromsocket will be equal to
sizerequested (less than 16384), then i am getting the error as .*curl: (18)
transfer closed with 4456 bytes remaining to read*.
So, i changed bytesfromsocket to a constant value 1 or to
16384(BUFSIZE) which i have commented, and again rebuilded the library
libcurl.so.4, After doing this, it was perfectly downloading for all the
file sizes. Want to know what i have changed is right or not?
Thanks
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: Re: Re: Problem using static libraries libcurl with OpenSSL, minGW, Eclipse
Gisle Vanem gvanem_at_broadpark.no wrote: Oleksiy patriot_of_ua_at_ukr.net wrote: g++ -DCURL_STATICLIB -O0 -g3 -Wall -c -fmessage-length=0 -osrc\main.o ..\src\main.cpp g++ -L..\lib -oYTUploader.exe src\main.o -lcurl -lws2_32 -lwldap32 -leay32 -lssleay32 -lz The same thing Try producing a .map file then: g++ -Wl,--print-map,--sort-common,--cref -L..\lib -oYTUploader.exe \ src\main.o -lcurl -lws2_32 -lwldap32 -leay32 -lssleay32 -lz YTUploader.map Study it and figure out why things are as they are. Can you help me? I can't figure the things out. I generated map-file, here it is http://pastebin.com/JxuCDXzM --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RE: Linker errors when statically compiled linking libcurl+openssl to my project
Hi, I am trying to statically link libcurl (with SSL) into my project but I am encountering lots of linker errors: ../lib/libcrypto.a(rand_win.o):rand_win.c:(.text+0xa9c): undefined reference to `CreateDCA@16' ../lib/libcrypto.a(rand_win.o):rand_win.c:(.text+0xaa9): undefined reference to `CreateCompatibleDC@4' ../lib/libcrypto.a(rand_win.o):rand_win.c:(.text+0xaba): undefined reference to `GetDeviceCaps@8' Looking at rand_win.c it uses CreateDCA, CreateCompatiableDC(), GetDeciceCaps() etc... which are defined in gdi32.lib. ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x1b3b): undefined reference to `inet_addr@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x1ba4): undefined reference to `inet_addr@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x5d1e): undefined reference to `htonl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x5eed): undefined reference to `ntohl@4' These are defined in ws2_32.lib. What libraries is your application linking against? I'm not a MinGW / Eclipse user but it looks like you will need to add gdi32.lib and ws2_32.lib to your project. Kind Regards Steve --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RE: RE: Linker errors when statically compiled linking libcurl+openssl to my project
Looking at rand_win.c it uses CreateDCA, CreateCompatiableDC(), GetDeciceCaps() etc... which are defined in gdi32.lib. These are defined in ws2_32.lib. What libraries is your application linking against? I'm not a MinGW / Eclipse user but it looks like you will need to add gdi32.lib and ws2_32.lib to your project. ws2_32.lib was already linked, I added gdi32.lib. Errors decreased. The following remain g++ -DCURL_STATICLIB -DUSE_LIBSSH2 -DHAVE_LIBSSH2 -DCURL_DISABLE_LDAP -DHAVE_LIBSSH2_H -DLIBSSH2_WIN32 -DLIBSSH2_LIBRARY -DUSE_SSLEAY -O0 -g3 -Wall -c -fmessage-length=0 -osrc\CURLDown.o ..\src\CURLDown.cpp ..\src\CURLDown.cpp:18: warning: 'int writer(char*, size_t, size_t, std::string*)' defined but not used g++ -L../lib -oCURLDown.exe src\CURLDown.o -lcurl -lcrypto -lws2_32 -lwldap32 -leay32 -lidn -lrtmp -lssh2 -lz -lssl32 -lssl -lgdi32 ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x1b3b): undefined reference to `inet_addr@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x1ba4): undefined reference to `inet_addr@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x5d1e): undefined reference to `htonl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x5eed): undefined reference to `ntohl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x6f9a): undefined reference to `htonl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x728d): undefined reference to `htonl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x732c): undefined reference to `htonl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x757f): undefined reference to `ntohl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0xb45): undefined reference to `timeGetTime@0' ../lib/librtmp.a(hashswf.o):hashswf.c:(.text+0x164): undefined reference to `inet_addr@4' --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: Re: Re: Problem using static libraries libcurl with OpenSSL, minGW, Eclipse
As I was told on StackOverflow, maybe the openssl-libs I link against seem to be import-libraries. That means they only contain the information your code needs to call functions and then load and call the corresponding functions from the dll. So the problem is: although you link to static libs, the libs then load and use dynamic dlls. They are no real static libs. I downloaded them from http://www.shininglightpro.com/download/Win32OpenSSL-1_0_0d.exe , used .a-files in folder called MinGW. Maybe I took wrong files from wrong place? I need static OpenSSL libs --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RE: RE: Linker errors when statically compiled linking libcurl+openssl to my project
Hiya, ws2_32.lib was already linked, I added gdi32.lib. Errors decreased. The following remain ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x1b3b): undefined reference to `inet_addr@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x732c): undefined reference to `htonl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x757f): undefined reference to `ntohl@4' Interesting. These are definitely in ws2_32.lib. ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0xb45): undefined reference to `timeGetTime@0' I just had to check MSDN for this one - and it says this function is defined in winmm.lib. I'm not too familiar with librtmp, so I just downloaded the source for it and checked out it's makefile which includes the following when building with mingw: -lws2_32 -lwinmm -lgdi32 Can you add -winmm and see if that decreases the number of errors by 1? If not, then there is something else going wrong (as the unresolved externals for ws2_32 have got me baffled at the minute)! S. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RE: RE: RE: RE: Linker errors when statically compiled linking libcurl+openssl to my project
I just had to check MSDN for this one - and it says this function is defined in winmm.lib. I'm not too familiar with librtmp, so I just downloaded the source for it and checked out it's makefile which includes the following when building with mingw: -lws2_32 -lwinmm -lgdi32 Can you add -winmm and see if that decreases the number of errors by 1? If not, then there is something else going wrong (as the unresolved externals for ws2_32 have got me baffled at the minute)! Looks like this changes nothing :( g++ -DCURL_STATICLIB -DUSE_LIBSSH2 -DHAVE_LIBSSH2 -DCURL_DISABLE_LDAP -DHAVE_LIBSSH2_H -DLIBSSH2_WIN32 -DLIBSSH2_LIBRARY -DUSE_SSLEAY -O0 -g3 -Wall -c -fmessage-length=0 -osrc\CURLDown.o ..\src\CURLDown.cpp ..\src\CURLDown.cpp:18: warning: 'int writer(char*, size_t, size_t, std::string*)' defined but not used g++ -L../lib -oCURLDown.exe src\CURLDown.o -lcurl -lcrypto -lws2_32 -lwinmm -lgdi32 -lwldap32 -leay32 -lidn -lrtmp -lssh2 -lz -lssl32 -lssl ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x1b3b): undefined reference to `inet_addr@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x1ba4): undefined reference to `inet_addr@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x5d1e): undefined reference to `htonl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x5eed): undefined reference to `ntohl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x6f9a): undefined reference to `htonl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x728d): undefined reference to `htonl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x732c): undefined reference to `htonl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0x757f): undefined reference to `ntohl@4' ../lib/librtmp.a(rtmp.o):rtmp.c:(.text+0xb45): undefined reference to `timeGetTime@0' ../lib/librtmp.a(hashswf.o):hashswf.c:(.text+0x164): undefined reference to `inet_addr@4' --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] Added AUTH NTLM for SMTP
Hi Steve, 2011/8/24 Yang Tse wrote: Right now I see nothing relative to all this in already provided patches. I'll dig a bit further in case it is hidden somewhere, but you are the one who actually knows by heart where should we arrive. Yep, your initial consolidated patch already included moving the base64 encoding of type-1 and type-3 messages to Curl_ntlm_create_typeX_message functions. To stop all this 'if we should', 'if it is really desirable' on my side, given that it really seems to be the right thing to do, and to allow us to move on. I've just pushed this change into master. In a while I'll comment on the 'SMTP Modifications' thread. -- -=[Yang]=- --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] SMTP Modifications
2011/8/24 Steve Holme wrote: Please find attached my next modification for the SMTP protocol - which adds the ability to specify whether authenticated connections (AUTH PLAIN and AUTH LOGIN) send the initial response in the AUTH command or in the next command. This can be set via the CURLOPT_MAIL_SINGLE_AUTH option but defaults to TRUE to maintain compatibility with versions of curl prior to 7.22.0. It seems to me that you want to provide the capability of _not_ sending the 'client initial response' in the AUTH command in order to allow interoperability with servers that support the AUTH extension but that don't tolerate the optional client initial response argument. It seems to me that in order to achieve this you propose an option named 'CURLOPT_MAIL_SINGLE_AUTH'. I believe that literal is a bit misleading relative to its actual intended behavior. If the purpose of the option is what I understood in above paragraph, I think a more appropriate literal would be 'CURLOPT_MAIL_AUTH_DISABLE_INITIAL_RESPONSE'. The need for the CURLOPT_MAIL_AUTH_DISABLE_INITIAL_RESPONSE option as an individual CURLOPT_* option may not be fully justified. We can provide the ability to disable sending the client initial-response AUTH argument by other means. Introducing CURLOPT_MAILAUTH 'bitmasked' option would allow to specify which authentication methods are exclusively allowed to be used, along with any specific parametrization. For example bitmask argument for CURLOPT_MAILAUTH could be composed with bits such as: CURLMAIL_AUTH_ANY CURLMAIL_AUTH_CRAM_MD5 CURLMAIL_AUTH_LOGIN CURLMAIL_AUTH_PLAIN CURLMAIL_AUTH_[...] CURLMAIL_AUTH_DISABLE_INITIAL_RESPONSE Something very similar is already done for other protocols ie CURLOPT_HTTPAUTH. Daniel, should we use an independent CURLOPT_MAIL_AUTH_DISABLE_INITIAL_RESPONSE option or do we simply let it be a bit for the CURLOPT_MAILAUTH one? Steve, once that Daniel has commented on all this, you can provide a single patch that provides whole functionality, at least for the optional disabling of client-response AUTH argument. curl_easy_setopt.3 man page adjustments and test cases can be left out now, but would be nice to have in a soonish subsequent patch. -- -=[Yang]=- --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RE: [PATCH] SMTP Modifications
Hi, It seems to me that you want to provide the capability of _not_ sending the 'client initial response' in the AUTH command in order to allow interoperability with servers that support the AUTH extension but that don't tolerate the optional client initial response argument. Exactly that. It seems to me that in order to achieve this you propose an option named 'CURLOPT_MAIL_SINGLE_AUTH'. I believe that literal is a bit misleading relative to its actual intended behavior. Yep. I must admit I was struggling for sensible [short] names ;-) Introducing CURLOPT_MAILAUTH 'bitmasked' option would allow to specify which authentication methods are exclusively allowed to be used, along with any specific parametrization. For example bitmask argument for CURLOPT_MAILAUTH could be composed with bits such as: Being able to specify which authentication protocols is something that is already on my to do list - however, I didn't think it was something I would have time to complete for v7.22.0. Steve, once that Daniel has commented on all this, you can provide a single patch that provides whole functionality, at least for the optional disabling of client-response AUTH argument. Sure. Kind Regards S. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: can Curl use socket() created by application?
On Wed, Aug 24, 2011 at 6:21 PM, Daniel Stenberg dan...@haxx.se wrote: On Wed, 24 Aug 2011, amit paliwal wrote: I am using correct version of the library here. Once I connected to the server, I have done some manual send() and recv() which works fine, after that I tried sending HTTP POST to the server but I got an error Failed sending data to the peer i.e. error no 55. So what happened with the connect failures you had before? What might be the reason for this? CURLOPT_VERSBOSE might show more details, but 55 is CURLE_SEND_ERROR and that means quite what it sounds like: it failed (trying to) sending data Before sending this HTTP POST I set all the options which are there in our example. As before, if you show us a complete program that we can try in our ends we can probably help out quicker and better. Otherwise there's just debugging and researching to see what happens and why. There are no magic tricks involved. i fixed it, there was some problem at application side . Thanks For the great help. Just one query, do I need to set these socket related options once only or every-time when I call curl_easy_perform() I need to set them? -- / daniel.haxx.se --**--**--- List admin: http://cool.haxx.se/list/**listinfo/curl-libraryhttp://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/**etiquette.htmlhttp://curl.haxx.se/mail/etiquette.html -- Regards, Amit --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: can Curl use socket() created by application?
On Thu, Aug 25, 2011 at 11:58:16AM -0400, amit paliwal wrote: i fixed it, there was some problem at application side . Thanks For the great help. Just one query, do I need to set these socket related options once only or every-time when I call curl_easy_perform() I need to set them? Once they're set on an easy handle, they should stay that way until they're explicitly changed or reset. Dan --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] SMTP Modifications
Hi, In case a single CURLOPT_MAILAUTH 'bitmasked' option is favoured relevant bits in curl.h could be something like the following, in which lower bits are used for authentication mechanisms and higher bits for parametrization purposes. Bits in between free for future expansion. #define CURLMAIL_AUTH_LOGIN (10) #define CURLMAIL_AUTH_PLAIN (11) #define CURLMAIL_AUTH_CRAM_MD5(12) #define CURLMAIL_AUTH_DIGEST_MD5 (13) #define CURLMAIL_AUTH_GSSAPI (14) #define CURLMAIL_AUTH_EXTERNAL(15) #define CURLMAIL_AUTH_NTLM(16) #define CURLMAIL_AUTH_ANY((10) (11) (12) (13) \ (14) (15) (16)) #define CURLMAIL_AUTH_NO_IR (131) Obviously CURLMAIL_AUTH_NO_IR is the relevant bit to disable sending client initial response in AUTH command. -- -=[Yang]=- --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] SMTP Modifications
On Thu, 25 Aug 2011, Yang Tse wrote: Something very similar is already done for other protocols ie CURLOPT_HTTPAUTH. Daniel, should we use an independent CURLOPT_MAIL_AUTH_DISABLE_INITIAL_RESPONSE option or do we simply let it be a bit for the CURLOPT_MAILAUTH one? I don't mind having it among the auth type bits. It would be similar to the the 'CURLAUTH_ONLY' bit we have for HTTP which serves the purpose of allowing an app to have libcurl prope for authentication but still only accept/use a single specific authentication type. I think adding it among the bits is better than having it as a completely separate option. But also, as Steve brought up, we're in feature freeze right now and even if it isn't a rule we can ignore at times I think we first need to consider it carefully first if it really needs to go in now. We do have these feature freeze periods in order to settle and iron out bugs. Of course we can still work on patches and discuss how to do things for inclusion after the next release. -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] SMTP Modifications
On Thu, Aug 25, 2011 at 06:52:51PM +0200, Yang Tse wrote: #define CURLMAIL_AUTH_LOGIN (10) #define CURLMAIL_AUTH_PLAIN (11) #define CURLMAIL_AUTH_CRAM_MD5(12) #define CURLMAIL_AUTH_DIGEST_MD5 (13) #define CURLMAIL_AUTH_GSSAPI (14) #define CURLMAIL_AUTH_EXTERNAL(15) #define CURLMAIL_AUTH_NTLM(16) As I brought up in March when this first came up, since SMTP uses SASL for authentication, I suggest renaming these to something like CURL_SASL_AUTH_x so that they can be used for other SASL-using protocols in the future. Dan --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] SMTP Modifications
2011/8/25 Daniel Stenberg wrote: I think adding it among the bits is better than having it as a completely separate option. I also prefer that one. But also, as Steve brought up, we're in feature freeze right now and even if it isn't a rule we can ignore at times I think we first need to consider it carefully first if it really needs to go in now. Yes of course. That is your decision. We do have these feature freeze periods in order to settle and iron out bugs. Yes certainly. I've attached chronology that shows that feature freeze period has not been ignored. Everything pushed after Aug-16 have been fixes for stuff already in master repo. Of course we can still work on patches and discuss how to do things for inclusion after the next release. Patch requested today should show that there's little left. But functionality this patch would provide already existed in first patches provided Jul-29 and Aug-02 by Steve. It is not my decision weather this should go in before next release. It is yours. What I'm asking here is you postpone this decision until Steve provides requested patch and we can see if it needs further polishing and better evaluate risks associated with pushing it in feature freeze period. Steve, please, I'll test your patches locally here. Don't let the ball drop now ;-) -- -=[Yang]=- Chronology of NTLM SMTP fixes... Jul-23 Steve joined the mailing list Jul-29 Steve provides first patch Aug-02 Steve provides followup patch Aug-04 TODO-RELEASE item #303 added Aug-04 Help with TODO-RELEASE items requested Aug-05 I ask Steve for a consolidated patch of Jul-29 and Aug-02 Aug-05 Steve provides consolidated patch Aug-08 I ask steve to break the consolidated patch into pieces Aug-09 Feature freeze announced for Aug-16 Aug-09 Steve provides 0001-Tidied-up-http_ntlm-prior-to-splitting-the-ntlm-spec.patch Aug-09 I push patch along with TAB and lengthfixes Aug-10 Steve provides 0001-Moved-ntlm-specific-code-into-separate-curl_ntlm-mod.patch Aug-10 I require some modifications in that patch Aug-10 Steve provides 0001-Tidied-up-more-inconsistent-spacing.patch Aug-11 I pushed patch with a couple of edits Aug-11 Steve provides 0001-Moved-ntlm-specific-code-into-separate_curl_ntlm_mod_2.patch Aug-11 I require some modifications in that patch Aug-12 Steve provides 0001-Moved-ntlm-specific-code-into-separate-curl_ntlm-mod_3.patch Aug-12 I push part of Aug-12 patch Aug-13 I push minor edits Aug-14 I push some adjustments Aug-15 I push some adjustments Aug-16 FEATURE FREEZE Aug-22 I push remaining bits from Aug-12 patch Aug-22 I ask steve another patch for the SMTP authentication orderingstuff Aug-22 Steve provides 0001-Modified-the-order-of-the-preferred-authentication-m.patch Aug-24 I push Curl_base64 interface fixes Aug-24 I push Aug-22 patch Aug-24 Steve provides 0001-Added-the-ability-to-specify-whether-authenticated-c.patch Aug-25 I push NTLM refactoring followup Aug-25 I Ask Steve to change Aug-24 patch to better fit libcurl API Here we are. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RE: [PATCH] SMTP Modifications
Hi Dan, As I brought up in March when this first came up, since SMTP uses SASL for authentication, I suggest renaming these to something like CURL_SASL_AUTH_x so that they can be used for other SASL-using protocols in the future. I totally agree but can the optional initial response flag be classed as SASL though / Do other SASL protocols do the same sort of thing as SMTP with AUTH [mechanism] [optional initial response]? S. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] SMTP Modifications
On Thu, 25 Aug 2011, Yang Tse wrote: We do have these feature freeze periods in order to settle and iron out bugs. Yes certainly. I've attached chronology that shows that feature freeze period has not been ignored. Everything pushed after Aug-16 have been fixes for stuff already in master repo. Oh, I didn't mean to suggest or imply that the rules have been violated in any way. I was just thinking that this auth bits stuff we're discussing might. -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RE: [PATCH] SMTP Modifications
Hi all, Patch requested today should show that there's little left. But functionality this patch would provide already existed in first patches provided Jul-29 and Aug-02 by Steve. Given that this, potentially, is part of much larger work would it be best to leave this until v7.22.1, along with the CURLOPT_MAIL_AUTH change that I had in my original patch as well? I was going to submit that feature next, but given that it is only half Technically complete as well then it may be best to leave it and get all this working properly in the next release. Just to summaries the CURLOPT_MAIL_AUTH parameter that I added, allowed the user of libcurl to specify whether or not the optional AUTH= is part of the MAIL command. The problem with my existing patch, is that it does not allow for an empty AUTH= e.g MAIL FROM:u...@domain.com AUTH= as per the RFC. In addition to this I also wanted to work on the following: * Being able to specify which authentication mechanisms to use (pretty much as already suggested by Yang). * Potentially, being able to specify the order of authentication mechanisms (This still needs a lot of thought in my own head and might not be possible). * Being able to specify the optional SIZE=xxx parameter in the MAIL command. This allows the server to reject a message, normally one that contains an attachment, before the DATA content is sent. * Fix the fully qualified hostname issue. What I'm asking here is you postpone this decision until Steve provides requested patch and we can see if it needs further polishing and better evaluate risks associated with pushing it in feature freeze period. Rather than rush a patch out now... Would it be better to concentrate on adding just the NTLM support to SMTP and leave all of the above until after that? If my post NTLM patch(es) are polished enough we might be able to squeeze them into v7.22.0 ;-) ? Steve --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RE: RE: RE: RE: Linker errors when statically compiled linking libcurl+openssl to my project
Hiya, Can you add -winmm and see if that decreases the number of errors by 1? If not, then there is something else going wrong (as the unresolved externals for ws2_32 have got me baffled at the minute)! Looks like this changes nothing :( Mmmm very strange. I must admit I don't include SSH or RTMP support (at the moment) when I build libcurl here. I use Visual Studio (and have solution and project files for .NET 2003 (v7.1), 2005 (v8.0), 2008 (v9.0) and 2010 (v10.0)) and have them configured to build against OpenSSL (as a DLL rather than statically) and to include NTLM support. Are you building RTMP yourself, or do you have a pre-built version? S. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: sftp download issue (Daniel Stenberg)
On Thu, 25 Aug 2011, Sumukh Anantha Manohar wrote: Eh, so does it work for _any_ sftp files at all? If so, which? Even if i try to download a 2MB file with sftp protocol, sometimes it will download completely and sometimes it will show the error : **curl: (18) transfer closed with 574 bytes remaining to read*. So, this problem is arising for smaller file sizes also (sometimes). No matter how large the file is , i will get the same error. Ugha. I can repeat this. I'll see what I can do about it... -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] SMTP Modifications
2011/8/25 Steve Holme wrote: Hi Dan, As I brought up in March when this first came up, since SMTP uses SASL for authentication, I suggest renaming these to something like CURL_SASL_AUTH_x so that they can be used for other SASL-using protocols in the future. I totally agree but can the optional initial response flag be classed as SASL though / Do other SASL protocols do the same sort of thing as SMTP with AUTH [mechanism] [optional initial response]? Following sections of mentioned RFCs indicate that at least SASL for POP3, SMTP and IMAP follow same scheme relative to the 'initial-response' being optional for each respective authenticate-command. -- -=[Yang]=- --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] SMTP Modifications
2011/8/25 Daniel Stenberg wrote: Oh, I didn't mean to suggest or imply that the rules have been violated in any way. I was just thinking that this auth bits stuff we're discussing might. Yes certainly. Any new option/symbol/function in external headers would easily pinpoint such a violation. -- -=[Yang]=- --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] SMTP Modifications
Ha, ha left out ther references in previous message, Following sections of mentioned RFCs indicate that at least SASL for POP3, SMTP and IMAP follow same scheme relative to the 'initial-response' being optional for each respective authenticate-command. POP3 SASL - rfc5034 (section 5) http://tools.ietf.org/html/rfc5034 SMTP SASL - rfc4954 (section 8) http://tools.ietf.org/html/rfc4954 IMAP SASL - rfc4959 (section 3) http://tools.ietf.org/html/rfc4959 -- -=[Yang]=- --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RE: [PATCH] SMTP Modifications
Ha, ha left out ther references in previous message, I thought I was going mad for a minute as I couldn't see them in your previous message ;-) S. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] SMTP Modifications
2011/8/25 Steve Holme wrote: If my post NTLM patch(es) are polished enough we might be able to squeeze them into v7.22.0 ;-) ? In order to consider that work complete, it would be convenient to better integrate samba's winbind daemon auth helper implementation when used. At many places the fact of using samba's winbind daemon auth helper is denoted as NTLM_SSO capability. When as a matter of fact NTLM SSO (single sign on) is just a feature that may be achieved using samba's winbind daemon auth helper or by other means. I started fixing this, but given that it was interfering with Steve work I stopped, now I may restart this. Steve now that you are more familiar with git these changes/fixes might disturb your work less. -- -=[Yang]=- --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: RE: RE: RE: Linker errors when statically compiled linkinglibcurl+openssl to my project
Steve Holme steve_ho...@hotmail.com wrote: I use Visual Studio (and have solution and project files for .NET 2003 (v7.1), 2005 (v8.0), 2008 (v9.0) and 2010 (v10.0)) and have them configured to build against OpenSSL (as a DLL rather than statically) and to include Visual Studio uses a 2 pass linker. gnu-ld does not. The OP may have to reorder the libs in the link-command. -lws2_32 (missing htonl@4 etc.) should come after all other libs referring to those functions. --gv --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RE: [PATCH] SMTP Modifications
Hi Yang, In order to consider that work complete, it would be convenient to better integrate samba's winbind daemon auth helper implementation when used. Steve now that you are more familiar with git these changes/fixes might disturb your work less. No worries. I have enclosed a patch adding NTLM support to SMTP. Please note that halfway through this patch I realised that Curl_ntlm_create_type1_message() and Curl_ntlm_create_type3_message didn't support the outlen / size parameter (as I had written a couple of weeks ago). As such, this patch includes the addition of this parameter to these functions, as well as the actual addition of NTLM to SMTP. I hope this is acceptable, rather than splitting the work into two patches, so for speed and as it's almost midnight and I need to get some sleep I have attached one patch ;-) ...so on that note, my bed is calling - Goodnight all. Kind Regards Steve 0001-Added-support-for-NTLM-authentication-to-SMTP.patch Description: Binary data --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Problem with NTLM_SSO and signals
libcurl calls 'fork' and 'waitpid' in only one case: to run the helper app in the newly-added NTLM_SSO authentication method. But, forking in a library is somewhat dangerous as it can interfere with the parent application's handling of signals. Here's one failure scenario: an application can install a signal handler for SIGCHLD that just calls wait() and throws away the response code. This is sufficient to avoid zombies and a common approach in applications that fork. But, applications that do this and use NTLM_SSO can face disaster. Once the NTLM_SSO helper application exits, a SIGCHLD signal is sent and the application reaps it with wait(). libcurl then calls waitpid() on that process ID to reap it itself. Since that has already been done by the app itself, waitpid() will normally error out benignly and everyone stays happy. But on a loaded system with many processes, this opens a race condition whereby waitpid() actually reaps a *different* process that has reused the PID of the NTLM_SSO helper. There's been a discussion around this recently on the Busybox list: http://lists.busybox.net/pipermail/busybox/2011-August/076507.html Another failure scenario is due to CURLOPT_NOSIGNAL not being respected, which can cause problems in multithreaded programs, among others. There's no way to cause libcurl's fork() to avoid raising a SIGCHLD on exit without changing the global state. It would be nice to have a solution for this before the next libcurl release, since that will be the first one with NTLM_SSO support. But, if not, these issues should at least be documented so application writers can be aware. Dan --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
