Re: Curl and SSL in an IMB's OnDemand environment
On Wed, 11 Sep 2019, Michael Rellstab wrote: Do you mean, OnDemand itself has libcurl linked (statically?) into its binaries? And my code uses this binary instead of the libcurl that is installed on the Linux? That was my question too =) I'm not really sure if we simply can focus on the different TLS backends. One thing I don't understand is, that in my code, I'm reading out the CURL's versions by curl_version_info(). The result is: 2019-09-10 15:11:07 DEBUG CURL version:7.29.0 2019-09-10 15:11:07 DEBUG CURL ssl version:NSS/3.34 I can't say that I understand your setup completely as you show NSS used here but then the verbose output says gskit. So I assume, the CURL which my code uses, is the CURL that is installed on my Linux and therefore is the CURL that is compiled against NSS. Some steps later I simply call curl_easy_perform(). Thats all. Internally it is CURL that requests for an SSL connection. And here is the point, I'm getting confused. When the CURL, which my code is using, is compiled against NSS (CURL explain me this by curl_version_info()), what could be the reason that CURL wants to connect by Curl_gskit_connect_nonblocking? It can't. It will only use gskit if it was built that way and since your version isn't, the only explanation is that in this case it runs *another* libcurl that is built to use gskit. 2019-09-10 15:11:07 DEBUG == Info: Curl_gskit_connect_nonblocking in This verbose output message is not even done by modern libcurl. Maybe it once did and this is just a very old libcurl with gskit. Shouldn't this be any other ssl connect method instead of a *gskit* method? Which conditions must be met that CURL uses a gskit connection method instead of any NSS connection method? It was built to use gskit instead of NSS. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Curl and SSL in an IMB's OnDemand environment
On Wed, Sep 11, 2019 at 11:22 AM Michael Rellstab via curl-library wrote: [...] > Do you mean, OnDemand itself has libcurl linked (statically?) into its > binaries? If it were statically linked, the symbols of that curl lib would not be visible to your module when it is loaded. > And my code uses this binary instead of the libcurl that is > installed on the Linux? Your module probably loads the curl library that you have linked it against, but the symbols (functions) from both the libcurl bundled with OnDemand and your libcurl are used to resolve the references in your module. I'm not sure how the runtime linker selects a symbol if there are multiple candidates. You could try linking your module with a static libcurl, or link your libcurl with symbol versioning (see e.g. https://www.gnu.org/software/gnulib/manual/html_node/LD-Version-Scripts.html https://www.bottomupcs.com/libraries_and_the_linker.xhtml) and see to that your module requires those specific versions of the curl functions. It may also be necessary to link your libcurl to use those specific versions to ensure that internal function calls from your libcurl don't end up using the OnDemand libcurl. Rainer --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Curl and SSL in an IMB's OnDemand environment
Hi Rainer Both of your responses are correct. In the meantime, I downloaded and compiled the latest CURL and I added curl_global_sslset(). The result is still the same. I've replyed to the answer of Daniel with more details about all this. Thanks! Michael On 10.09.2019 18:02, Rainer Canavan via curl-library wrote: [...] apparently the curl you're using is compiled with support for dynamic ssl backends. Try selecting NSS with https://curl.haxx.se/libcurl/c/curl_global_sslset.html I should have checked before writing. The libcurl that ships with CentOS does _not_ have support for curl_global_sslset(). rainer --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Curl and SSL in an IMB's OnDemand environment
Hi Daniel
Thanks for your response!
On 10.09.2019 17:38, Daniel Stenberg wrote:
> On Tue, 10 Sep 2019, Michael Rellstab via curl-library wrote:
>
>> Since several days I'm trying out to get my project to work, but I
>> don't have any success. Giving a short overview: I have to implement
>> a UserExit (callback routine) for the IBM's OnDemand Software. Inside
>> this UserExit I'm using CURL (linked as shared library). This works
>> perfectly as long as I don't use an SSL secured communication. As
>> soon as I activate SSL (TLS1.2), there is no communication anymore.
>
> This seems to assume a few things that you didn't explain.
>
> This "OnDemand Software" calls the callback you write. How is that
> using libcurl? Is this software linked with libcurl already so you're
> just calling the libcurl API from within this callback?
>
This is correct, OnDemand calls my code by callback function. I just
have to implement a method fulfilling a predefined signature, so
OnDemand will find the entry point inside my binary. My code is linked by:
gcc -m64 -fPIC -pthread -Wl,-eSECURITY -shared -lcurl -L/usr/lib64 -o
arsusec ${MY_OWN_OBJECT_FILES}
Then, I simply have to copy the compiled binary 'arsusec' into a
specific folder of OnDemand and thats it.
Do you mean, OnDemand itself has libcurl linked (statically?) into its
binaries? And my code uses this binary instead of the libcurl that is
installed on the Linux?
>> I'm running on a CentOS with the NSS SSL framework compiled into
>> CURL. When I use my UserExit without OnDemand (using the same source
>> code, but executed by my main function), CURL runs together with NSS
>> without any problems. As soon as my code runs in the context of
>> OnDemand, SSL is not working anymore. I expect, this has to do with
>> IBM's OnDemand, because they are using their GsKit as SSL framework.
>
> If you're using libcurl the same way but it behaves differently
> depending on which TLS backend that runs, then I think we can focus on
> the differences in the TLS backends.
>
> The gskit code in curl is virtually unmaintained and it is likely to
> be the worst TLS choice of all the TLS backends libcurl supports.
> gskit is also not available for me to use so I can't test or improve
> it either.
>
I'm not really sure if we simply can focus on the different TLS
backends. One thing I don't understand is, that in my code, I'm reading
out the CURL's versions by curl_version_info(). The result is:
2019-09-10 15:11:07 DEBUG CURL version:7.29.0
2019-09-10 15:11:07 DEBUG CURL ssl version:NSS/3.34
So I assume, the CURL which my code uses, is the CURL that is installed
on my Linux and therefore is the CURL that is compiled against NSS.
Some steps later I simply call curl_easy_perform(). Thats all.
Internally it is CURL that requests for an SSL connection. And here is
the point, I'm getting confused.
When the CURL, which my code is using, is compiled against NSS (CURL
explain me this by curl_version_info()), what could be the reason that
CURL wants to connect by Curl_gskit_connect_nonblocking?
2019-09-10 15:11:07 DEBUG == Info: Trying 192.168.27.108...
2019-09-10 15:11:07 DEBUG == Info: Connected to 192.168.27.108
(192.168.27.108) port 8443 (#0)
2019-09-10 15:11:07 DEBUG == Info: Curl_gskit_connect_nonblocking in
Shouldn't this be any other ssl connect method instead of a *gskit* method?
Which conditions must be met that CURL uses a gskit connection method
instead of any NSS connection method?
By the way I would also be happy, if I not must use gskit. It would be
nicer to me too, if my code runs with OpenSSL or NSS.
>> 2019-09-10 15:11:07 DEBUG CURL version:7.29.0
>
> Can I also highlight that this is a *very* old curl version.
>
Yes, I am aware of this. 7.29.0 is the latest package for CentOS, but if
newer versions will solve my problem, it makes of course sense to
compile the newest CURL.
> I'd urge you to contact the OnDmeand support as they are the ones
> providing this API for you. And they provide a libcurl built with
> gskit for you. Alternatively, ask the gskit team how you can debug
> your gskit-using libcurl-omdemand application and its TLS connections.
> I don't see how we can help with that!
>
Because of my description above, I am not really sure if OnDemand is
intentionally providing libcurl for use. Perhaps that simply are
unfortunate circumstances that CURL + SSL behave that way. If OnDemand
intentionally providing libcurl, I would prefer to not use them, but
using the 'standard' libs from the OS instead.
By the way: I did some additional tests. I downloaded the lates CURL
source and compiled it together with OpenSSL. Additional, I also added a
code line curl_global_sslset(CURLSSLBACKEND_OPENSSL, NULL, NULL).
The result is the same. Although, CURL version and ssl framework have
changed now to 7.66.0-DEV and OpenSSL
2019-09-10 16:34:23 DEBUG CURL version:7.66.0-DEV
2019-09-10 16:34:23 DEBUG CURL
Re: Curl and SSL in an IMB's OnDemand environment
[...] > apparently the curl you're using is compiled with support for dynamic > ssl backends. > Try selecting NSS with https://curl.haxx.se/libcurl/c/curl_global_sslset.html I should have checked before writing. The libcurl that ships with CentOS does _not_ have support for curl_global_sslset(). rainer --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Curl and SSL in an IMB's OnDemand environment
On Tue, 10 Sep 2019, Michael Rellstab via curl-library wrote: Since several days I'm trying out to get my project to work, but I don't have any success. Giving a short overview: I have to implement a UserExit (callback routine) for the IBM's OnDemand Software. Inside this UserExit I'm using CURL (linked as shared library). This works perfectly as long as I don't use an SSL secured communication. As soon as I activate SSL (TLS1.2), there is no communication anymore. This seems to assume a few things that you didn't explain. This "OnDemand Software" calls the callback you write. How is that using libcurl? Is this software linked with libcurl already so you're just calling the libcurl API from within this callback? I'm running on a CentOS with the NSS SSL framework compiled into CURL. When I use my UserExit without OnDemand (using the same source code, but executed by my main function), CURL runs together with NSS without any problems. As soon as my code runs in the context of OnDemand, SSL is not working anymore. I expect, this has to do with IBM's OnDemand, because they are using their GsKit as SSL framework. If you're using libcurl the same way but it behaves differently depending on which TLS backend that runs, then I think we can focus on the differences in the TLS backends. The gskit code in curl is virtually unmaintained and it is likely to be the worst TLS choice of all the TLS backends libcurl supports. gskit is also not available for me to use so I can't test or improve it either. 2019-09-10 15:11:07 DEBUGCURL version:7.29.0 Can I also highlight that this is a *very* old curl version. I would be really happy, if someone has an idea how to fix this issue or getting closer to the problem. If you need more information, just ask what you need to know and I will try to give you as much detail as I can. I'd urge you to contact the OnDmeand support as they are the ones providing this API for you. And they provide a libcurl built with gskit for you. Alternatively, ask the gskit team how you can debug your gskit-using libcurl-omdemand application and its TLS connections. I don't see how we can help with that! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Curl and SSL in an IMB's OnDemand environment
On Tue, Sep 10, 2019 at 5:19 PM Michael Rellstab via curl-library wrote: > > Hi there! > > Since several days I'm trying out to get my project to work, but I don't have > any success. > Giving a short overview: > I have to implement a UserExit (callback routine) for the IBM's OnDemand > Software. Inside this UserExit I'm using CURL (linked as shared library). > This works perfectly as long as I don't use an SSL secured communication. As > soon as I activate SSL (TLS1.2), there is no communication anymore. > > I'm running on a CentOS with the NSS SSL framework compiled into CURL. When I > use my UserExit without OnDemand (using the same source code, but executed by > my main function), > CURL runs together with NSS without any problems. As soon as my code runs in > the context of OnDemand, SSL is not working anymore. I expect, this has to do > with IBM's OnDemand, because they are using their GsKit as SSL framework. > > As you can see on my log output: > > 2019-09-10 15:11:07 DEBUGCURL version:7.29.0 [...] > 2019-09-10 15:11:07 DEBUGCURL ssl version:NSS/3.34 [...] > 2019-09-10 15:11:07 DEBUG== Info: Trying 192.168.27.108... > 2019-09-10 15:11:07 DEBUG== Info: Connected to 192.168.27.108 > (192.168.27.108) port 8443 (#0) > 2019-09-10 15:11:07 DEBUG== Info: Curl_gskit_connect_nonblocking in [...] apparently the curl you're using is compiled with support for dynamic ssl backends. Try selecting NSS with https://curl.haxx.se/libcurl/c/curl_global_sslset.html rainer --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Curl and SSL in an IMB's OnDemand environment
Hi there! Since several days I'm trying out to get my project to work, but I don't have any success. Giving a short overview: I have to implement a UserExit (callback routine) for the IBM's OnDemand Software. Inside this UserExit I'm using CURL (linked as shared library). This works perfectly as long as I don't use an SSL secured communication. As soon as I activate SSL (TLS1.2), there is no communication anymore. I'm running on a CentOS with the NSS SSL framework compiled into CURL. When I use my UserExit without OnDemand (using the same source code, but executed by my main function), CURL runs together with NSS without any problems. As soon as my code runs in the context of OnDemand, SSL is not working anymore. I expect, this has to do with IBM's OnDemand, because they are using their GsKit as SSL framework. As you can see on my log output: 2019-09-10 15:11:07 DEBUGCURL version:7.29.0 2019-09-10 15:11:07 DEBUGCURL host:x86_64-redhat-linux-gnu 2019-09-10 15:11:07 DEBUGCURL features:886BD 2019-09-10 15:11:07 DEBUGCURL ssl version:NSS/3.34 2019-09-10 15:11:07 DEBUGSuccessfully initialized Environment 2019-09-10 15:11:07 DEBUGTry to authenticate user 'nonadmin' by external authentication server 2019-09-10 15:11:07 DEBUG== Info: Trying 192.168.27.108... 2019-09-10 15:11:07 DEBUG== Info: Connected to 192.168.27.108 (192.168.27.108) port 8443 (#0) 2019-09-10 15:11:07 DEBUG== Info: Curl_gskit_connect_nonblocking in 2019-09-10 15:11:07 DEBUG== Info: curl_gskit_connect infd=34 2019-09-10 15:11:07 DEBUG== Info: curl_gskit_connect after init 35 1 2019-09-10 15:11:07 DEBUG== Info: curl_gskit_connect out 35 1 2019-09-10 15:11:07 DEBUG== Info: Curl_gskit_connect_nonblocking out 35 0 2019-09-10 15:11:07 DEBUG== Info: Closing connection 0 2019-09-10 15:11:07 DEBUG== Info: Curl_gskit_close in 2019-09-10 15:11:07 DEBUG== Info: Curl_gskit_close out 2019-09-10 15:11:07 DEBUG== Info: Curl_gskit_close in 2019-09-10 15:11:07 DEBUG== Info: Curl_gskit_close out 2019-09-10 15:11:07 DEBUG== Info: Curl_gskit_close in 2019-09-10 15:11:07 DEBUG== Info: Curl_gskit_close out 2019-09-10 15:11:07 ERRORInternal CURL call failed, CURLcode=35, CURLmsg= Although my CURL is compiled against NSS (default CentOS package), it seems CURL tries to use gskit for connecting. How can I force CURL to use NSS instead of gskit? I'm really not a professional and have too less knowledge about all this shared libraries, linking and the process context. I would be really happy, if someone has an idea how to fix this issue or getting closer to the problem. If you need more information, just ask what you need to know and I will try to give you as much detail as I can. Thanks very much in advance! Regards, Michael --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
