Re: Windows users! Help us test upload performance tuning?
On Sat, Aug 4, 2018 at 2:06 AM, Daniel Jeliński wrote: > 2018-08-03 22:47 GMT+02:00 Daniel Stenberg : > > On Fri, 3 Aug 2018, Ray Satiro wrote: > > This is strange. I see Ray's mail in the archives [1] but not in my > mailbox. On the other hand, I don't see Rickard Alcock's reply in the > archives, but I have it in my mailbox. What gives? > Gmail dropped Ray's message in my Spam folder with this note: "This message has a from address in yahoo.com but has failed yahoo.com's required tests for authentication" Ralph Mitchell --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: http post request with authentication
On Mon, May 23, 2016 at 5:00 PM, Michael Pfeffer via curl-library < curl-library@cool.haxx.se> wrote: > At first i´m a newbie in using libcurl. I must login via http post > (libcurl) to http://earthexplorer.usgs.gov and collect images. I use the > POSTFIELDS data to set search criteria but it doesnt seem to work. How can > i select a option? How can i select a inputfield without name? > I even don't know if i ask the right questions. I used some other sides > with get request, it worked fine. > > curl_easy_setopt(curl,CURLOPT_URL,"http://earthexplorer.usgs.gov;); > curl_easy_setopt(curl,CURLOPT_USERPWD,"login:password"); > > curl_easy_setopt(curl,CURLOPT_POSTFIELDS,"typeAddress=WRS1=3$rowAddress=3"); > Is that really supposed to be "$rowAddress=3" instead of "=3" Ralph Mitchell --- List admin: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: SMTP with TLS error handling
It may be deliberate behaviour by Office365, to avoid giving out information that may assist an attacker. Ralph Mitchell On Thu, Jun 4, 2015 at 11:26 AM, Jamie Gordon ja...@deslock.com wrote: Hi I have a problem with the correct error handling for SMTP errors when using libcurl and I'd be pleased if anyone can help/advise me. Specifically in this case I am using libcurl 7.42.1 but the same behaviour has been seen with previous versions I have tested with. This issue is, if I use the wrong password with Office365 I get error CURLE_OPERATION_TIMEDOUT and the internal curl message (CURLOPT_ERRORBUFFER) is Operation timed out after 0 milliseconds with 0 out of 0 bytes received. Actually some other servers using TLS do this also, but they have been at customer sites and I don't have any debug from those servers. But the issue is repeatable for me using Office365. If I use the correct username/password an email is sent fine. And I've been using the same sending code for a while with standard port 25 SMTP. You can see the debug from the libcurl below (this is output from my own CURLOPT_DEBUGFUNCTION callback function). I've just used a made up a username/password of t...@example.com/moo. debug EHLO myhost 250-AM2PR09CA0025.outlook.office365.com Hello [194.75.73.194] 250-SIZE 157286400 250-PIPELINING 250-DSN 250-ENHANCEDSTATUSCODES 250-AUTH LOGIN 250-8BITMIME 250-BINARYMIME 250 CHUNKING AUTH LOGIN 334 VXNlcm5hbWU6 dGVzdEBleGFtcGxlLmNvbQ== 334 UGFzc3dvcmQ6 bW9v Operation timed out after 0 milliseconds with 0 out of 0 bytes received Closing connection 0 /debug In terms of function calls: curl_easy_perform returned 28 (CURLE_OPERATION_TIMEDOUT) curl_easy_getinfo(CURLINFO_HTTP_CONNECTCODE) returned 334 and curl_easy_getinfo(CURLINFO_RESPONSE_CODE) returned 0 Interestingly, if I use gmail with the wrong password I see more relevant info in the debug. And curl_easy_perform also gives me CURLE_LOGIN_DENIED which is much more user friendly. debug 535-5.7.8 Username and Password not accepted. Learn more at 535 5.7.8 https://support.google.com/mail/answer/14257 ju2sm6707323wid.12 - gsmtp Closing connection 0 /debug So this seems maybe specific to the server somehow. Is it a problem with the Office365 response in TLS? Is their exchange server doing something wrong? Or is there some bad handling in libcurl somewhere? Or maybe could I have initialised something wrong in my c++ calling libcurl? If this behaviour by libcurl is correct then is it possible to interpret what I have to maybe suggest to the user that they have entered invalid credentials, rather than presenting them with Operation timed out after 0 milliseconds...? Otherwise if it is a bug, then I'll try to look into the library in more details and see if I can work out what's wrong. Any advice appreciated. Thanks Jamie --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: WPAD proxy (was Re: Need Help Please)
On Tue, Dec 30, 2014 at 5:30 PM, Dan Fandrich d...@coneharvesters.com wrote: On Tue, Dec 30, 2014 at 02:58:52PM +, Jones, Rick wrote: I have packaged cURL for my enterprise developers to use when needed but am having some issues. The installation part is easy, but I need to modify my package due to a configuration issue. Please let me explain. We are in a very locked down environment; users and devs do not have access rights to their machines; so everything must be packaged for them and installed via SCCM. The problem is our proxy, we are using WPAD to get to an external proxy which lets people out. What settings do I need to modify in order to use cURL? I have tried setting http_proxy and https_proxy but I’m still unable to connect. WPAD generally provides a small Javascript program that's executed by the browser to determine which sites to send to a proxy and which proxy to use. curl doesn't contain a Javascript interpreter, so it can't use such a proxy configuration directly. It's (theoretically) possible to integrate a standalone Javascript interpreter with libcurl to handle this kind of proxy configuration, but it wouldn't be trivial. See the mailing list archives for previous questions about this. Alternately, for certain simple cases you could manually interpret the proxy configuration, extract the correct proxy server for the site(s) you're interested in, and provide that proxy server in http_proxy. That little bit of Javascript is probably a proxy-auto-config (.pac) file. I put in a few minutes way back when to see if we could automate processing those. See this post for details on handling .pac in a script: http://curl.haxx.se/mail/archive-2002-06/0104.html As long as you know where to get the .pac file, you may be able to exec a script from your program to process it and return something usable. Dunno how you'd do that in Windows, though. Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: LibCurl C/C++ Authentification
On Jun 11, 2013 1:30 AM, Spencer Elliott fm_reb...@hotmail.com wrote:
Ralph,
I hope you do not mind me emailing you, it’s a bit random I guess, but I
saw your email in a public thread related to LibCurl authentification, and
I was hoping you might be able to help me.
I have built an application (Win32 console) that joins an IRC server, and
listens to the chat, waiting for a command. This part is working fine.
I have included the libcurl library into the application and performed
some simple examples.
However, I want the application to log-into an admin panel for a game
server, and then perform some admin functions (the game server can only be
administrated from a web panel, and the ability to perform these functions
in IRC instead is something the users of this game really want).
My challenge is that while I am okay programming in C++, I have very
little experience with cookies, http header files and libcurl.
Would you be able to show me the correct method of logging in (and
assuming I want to stay authenticated while the session is open)? I know as
much that it requires POST and have tried with the following:
void CurlPostTest()
{
CURL *curl;
CURLcode res;
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if(curl)
{
curl_easy_setopt(curl, CURLOPT_URL,
http://208.115.205.106:8075/ServerAdmin/;);
curl_easy_setopt(curl,
CURLOPT_FOLLOWLOCATION, 1);
curl_easy_setopt(curl, CURLOPT_POST, 1);
curl_easy_setopt(curl,
CURLOPT_POSTFIELDS, UserName=mynamePassword=mypass);
curl_easy_setopt(curl, CURLOPT_COOKIEJAR,
cookie);
curl_easy_setopt(curl,
CURLOPT_COOKIEFILE, cookie);
res = curl_easy_perform(curl);
//if(res != CURLE_OK)
//{
fprintf(stderr,
curl_easy_perform() failed: %s\n, curl_easy_strerror(res));
//}
curl_easy_cleanup(curl);
}
curl_global_cleanup();
}
However, this was largely guess work to be honest. It returns:
Title401 Unauthorized/titleH1401 Unauthorized/H1
Curl_easy_perform() failed: No error
Based on the url http://208.115.205.106:8075/ServerAdmin/ would you know
what the correct post fields are? That string is really what I need, I
think (variablename=mynamepasswordvairable=mypass).
How do you determine what the field names are?
If you felt really generous, would you mind changing the entire function
to log-in properly to the above the url? If I had an example, I’m sure I
could then do everything else I needed to.
I appreciate this is a weird random request. However, I am hoping you
remember your own struggles as a newbie and are willing to help?
I've copied this to the libcurl mailing list, because you'll get a better
answer, faster.
I don't use c/c++ at all, so I can't even start to help you out.
Ralph Mitchell
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: Questions on wifi portal login
On Fri, Feb 22, 2013 at 11:47 AM, Ben Greear gree...@candelatech.comwrote:
I've been trying to get a slightly hacked version of curl to do
an automated login to a wifi hotspot type of thing, but to date,
I'm not having too much luck.
I suspect part of the problem might be javascript related, but maybe
it's user error. The final page I get from the AP talks about enabling
cookies..though it seems they are properly enabled in curl.
Does anyone see anything obviously wrong with this script below?
And second, are there any suggestions for doing something like scripting
a 'real' browser such as firefox or google-chrome so that any funny
javascript
stuff is automatically taken care of? I'd have to integrate c-ares and
hack the device IP bindings into a browser...so not sure how hard
that would be...
use strict;
$|=1;
my $dev = $ARGV[0];
my $ip4 = $ARGV[1];
my $ip6 = $ARGV[2];
my $dns = $ARGV[3];
my $mgt_pipe = $ARGV[4];
my $rslt = OK;
print IFUP-POST: dev: $dev ip4: $ip4 ip6: $ip6 dns: $dns mgt-pipe:
$mgt_pipe\n;
# Add whitelist for which interfaces we are interested in.
# Could read this from a file for more flexibility.
my @devs = (); # Fix this before the script will really do anything
#my @devs =(wlan0,sta1, sta2, sta3);
my @username =(username, username, username, username);
my @passwd = (passwd, passwd,passwd, passwd);
my $login_url = http://www.google.com;; # Assume we'll get a portal
redirect
my $cookie = /tmp/${dev}_cookie.txt;
unlink($cookie);
my $i;
for ($i = 0; $i@devs; $i++) {
if ($devs[$i] eq $dev) {
# Add logic here to access wifi portal web pages, etc.
# Bind to interface and IP address or it may not work as desired.
my $un = $username[$i];
my $pd = $passwd[$i];
# Default to fail..set to OK if we log in properly.
$rslt = FAIL;
my $dns_srv = ;
if ($dns ne NA) {
$dns_srv = --dns_servers $dns;
}
my $curl_args = ./local/bin/curl -c $cookie -b $cookie -L --insecure
-4 --include --interface if!$dev --localaddr $ip4 $dns_srv --dns_interface
$dev --dns_ipv4_addr $ip4;
print Command -:$curl_args $login_url:-\n;
my $page = `$curl_args $login_url`;
print Result for url: $login_url -:$page:-\n;
if ($page =~ /.*Location:\s+(\S+).*/g) {
# Re-direct
my $url = $1;
print Got URL: $url\n;
$url =~ s/gateway/Login.action/g;
$page = `$curl_args $url`;
print Result for url: $url -:$page:-\n;
if ($page =~ /.*sessionid=(\S+)\.*/g) {
my $session_id = $1;
# Now, post the login-id
my $cmd = $curl_args -d \guestUser.name=$un**
guestUser.password=$pd**redirect=switch_url=err_**
flag=byodSessionId=$session_**idbyodAction=\ $url;
print Curl command: $cmd\n;
$page = `$cmd`;
print Result for (POST) url: $url -:$page:-\n;
$rslt = OK;
}
}
last;
}
}
There's an addon for Firefox called LiveHTTPheaders that records the
headers going back and forth when you visit a web page. It shows the urls
that you GET or POST, and the variables that are posted. You might be
missing a step, or not sending all the variables.
As for handling javascript - that's not so easy. If it's pure javascript,
just doing math, text manipulation or something like that, you can pipe it
through a javascript engine (e.g. spidermonkey) and catch the results.
I've had some success doing that. If the javascript is trying to access
page elements, such as forms, those will not work.
Ralph Mitchell
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: http://curl.haxx.se/ca/cacert.pem
On Thu, Feb 7, 2013 at 4:47 PM, Kristian Fiskerstrand kristian.fiskerstr...@sumptuouscapital.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 02/07/2013 10:40 PM, Daniel Stenberg wrote: On Thu, 7 Feb 2013, Kristian Fiskerstrand wrote: Anyway, I was wondering if it might be possible to update the website to serve these over a secure connection? Alternatively (or in addition) could it make sense to digitally sign the files using the OpenPGP standard (GnuPG) ? That's an interesting idea, but this output is generated automatically in a cronjob and it would be a bit annoying for me to have to sign it everytime it happens to change. And having a mis-matching signature lingering would be terrible! Presuming that the cronjob happens on a separate system than the actual webserver even an lower-security signing key could make sense, then the signing process could be performed automatically as well (--batch mode). It wouldn't help (as much) for a system compromise (nothing would), but it would be of great help for a poisoned DNS record or other MITM attack vector. Anyone really keen to roll their own bundle could pick up CA certs from Thawte, in the Top Issues tab here: https://search.thawte.com/support/ssl-digital-certificates/index.html and Verisign: https://knowledge.verisign.com/support/ssl-certificates-support/index?page=contentid=AR657 I imagine the other Certificate Authorities have similar downloads. Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: libcurl with client ssl certificate
On Mon, Aug 20, 2012 at 2:14 AM, Chris Baylis chrisba...@gmail.com wrote: Thank you all for your input. It lead me to a little investigation and as it turns out I didn't know what I was doing with the keys. Originally the client key was signed by the client itself. I now have client keys, signed by the web server. And can run simplessl.c with my keys and curl_easy_setopt(curl,CURLOPT_CAINFO,pCACertFile) disabled. Curious though how `curl -E cert url` worked in the original scenerario when simplessl.c did not. Judging by the outputs in your original email, command-line curl was looking in the /etc/ssl/certs *directory* for a CA cert to validate the server, whereas the compiled program was looking either in the /etc/ssl/certs/ca-certificates.crt *file* (as shown in the output), or in the cert.pem *file* (as shown in the source snippet). So, maybe the CA cert for your server is sitting in the directory, but is not appended to whichever file was read? Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: libcurl with client ssl certificate
On Sun, Aug 19, 2012 at 7:26 AM, Daniel Stenberg dan...@haxx.se wrote: On Sun, 19 Aug 2012, Chris Baylis wrote: static const char *pCertFile = cert.pem; static const char *pCACertFile=cert.pem; This seems like a highly unlikely scenario. The same file, really? A self-signed cert would be its own CA cert, so it's possible. He also has: pKeyName = cert.pem; so the key would need to be in the same file too. The one thing that I'm not sure about is the compiled program reporting: * found 142 certificates in /etc/ssl/certs/ca-certificates.crt when it *should* be looking for CA certs in the file designated by pCACertFile. Does libcurl automatically look in ca-certificates as well as any file you designate? Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: libcurl with client ssl certificate
On Sun, Aug 19, 2012 at 1:57 PM, Daniel Stenberg dan...@haxx.se wrote: On Sun, 19 Aug 2012, Ralph Mitchell wrote: static const char *pCertFile = cert.pem; static const char *pCACertFile=cert.pem; This seems like a highly unlikely scenario. The same file, really? A self-signed cert would be its own CA cert, so it's possible. But the pCertFile is the SSL _client_ cert. It would mean that he uses the client cert file to verify the server cert and still also use that to authenticate to the server. It might be possible, but my money is on a mistake or misunderstanding somewhere. Yes, true!! The *server* would need to have the client's CA chain to validate the client cert, and the *client* would need the server's CA chain to validate the server cert. If the OP has a self-signed CA cert that signed both the client and server certs, the chain file would be the same for both ends.. However, command-line curl must have successfully validated the server using just the CApath: * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs so pCACertFile doesn't need setting to cert.pem. so the key would need to be in the same file too. The one thing that I'm not sure about is the compiled program reporting: * found 142 certificates in /etc/ssl/certs/ca-**certificates.crt when it *should* be looking for CA certs in the file designated by pCACertFile. Does libcurl automatically look in ca-certificates as well as any file you designate? Very good remark there! No, libcurl will in that case only use the single file specified in the setopt. It would indicate that he actually has 142 certificates in the file. That also hints that it isn't a client cert at all... In command-line curl, -E filename specifies a file containing both key and matching certificate. Is libcurl going to bail out if it finds multiple unrelated certificates in that file, as well as the expected key/cert entries? Does the order of the entries matter? Personally, I'd prefer to keep the key, cert and chain files separate, so I know what's where, but I can see there might be reasons for keeping the pem files in one big lump. Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: libcurl with client ssl certificate
On Sat, Aug 18, 2012 at 5:52 PM, Chris Baylis chrisba...@gmail.com wrote: Hello, I am trying to use libcurl in C to connect a site that uses client SSL authentication. I can acheive this task with curl, but not libcurl. Successful example with curl $ curl -vv -E cert.pem https://someURL * About to connect() to someURL port 443 (#0) * Trying 1.2.3.4... connected * Connected to someURL (1.2.3.4) port 443 (#0) Enter PEM pass phrase: * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs ... Webpages dowloads succesfully and I can see some debuging output I added in PHP verifying that the client certificate was used. I have taken the simplessl.c example from the your site and changed only the following lines so far. const char *pPassphrase = PASSWORD SET HERE; static const char *pCertFile = cert.pem; static const char *pCACertFile=cert.pem; pKeyName = cert.pem; pKeyType = PEM; pEngine = NULL; curl_easy_setopt(curl, CURLOPT_VERBOSE, 1); The result of this is as follows $ gcc -lcurl simplessl.c $ ./a.out * About to connect() to https://someURL port 443 (#0) * Trying 1.2.3.4... * connected * Connected to someURL (1.2.3.4) port 443 (#0) * found 142 certificates in /etc/ssl/certs/ca-certificates.crt * error reading X.509 key or certificate file * Closing connection #0 * SSL connect error curl_easy_perform() failed: SSL connect error Does your cert.pem contain the chain of CA certs that signed your client certificate? You have a working command-line already, so you can use the --libcurl option to get the equivalent source file (with the exception of formpost stuff as mentioned in the man page): curl --libcurl myCurlSsl.c -w -E cert.pem https://some.site.com Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: cert files
On Mon, Oct 17, 2011 at 9:17 PM, Jeff Pohlmeyer yetanotherg...@gmail.comwrote: On Mon, Oct 17, 2011 at 4:00 PM, Perry Smith wrote: my question is, where can I store the cacert,perm file so that curl, or libcurl, or openssl can find it automatically. I don't know anything about AIX but on Linux there is a tool called strace that can help track down where a program is looking for its files. Maybe AIX has something similar? I haven't touched AIX for a long time, but just for giggles I downloaded from curl.haxx.se the most recent AIX build of curl (7.15.1) that was compiled for AIX by The Written Word. strings on the lpp package showed there's a curl-config parked here: /opt/TWWfsw/curl715/bin/curl-config Your milage may vary, depending on who compiled the version you have, but if curl-config is handy, there's an option which will tell you where the ca-bundle is located. e.g. curl-config --ca /opt/TWWfsw/curl715/share/ca-bundle.crt You should be able to cat your certificate pem file onto the end of the bundle. Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: [PATCH] CURLOPT_CACERTSTORE
On Fri, May 13, 2011 at 6:07 PM, gir...@shankar-software.org wrote: The whole point is to allow the user to decide which CA to trust and which to not trust. Trust is a funny thing but you can't shove trust onto somebody and force them to trust someone. That's not trust, that's something else. This is ok when the user is just a single person, we should of course trust him to do the right thing. But when the user is a large organization it is the authorities in that organization who determines who everyone in the organization should trust. Under that scenario, the individual user should not be allowed to tamper with the list of root CAs. This is the situation we are facing. OK, I have to ask this - could you build your specific list of allowed CA certs into the app and simply not go get the list from a possibly-tampered-with external file?? Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: how to filter redirections
On Mon, Apr 4, 2011 at 4:41 PM, Daniel Stenberg dan...@haxx.se wrote: On Mon, 4 Apr 2011, moi wrote: I have to track some sites using curlpp. When i download, some site open popup coming from ads or porno servers. I'd like to block those popup telling curl to follow redirections, but either to blacklist some sites, or to allow only redirection in the asked site. If CURLOPT_FOLLOWLOCATION isn't good enough (unconditionally following HTTP redirects to the protocols you allow) you can _easily_ do the redirect following yourself using curl_easy_getinfo and CURLINFO_REDIRECT_URL and then you can black/whitelist the URLs all you like or whatever you decide is right! You may also want to examine the page for: 1) meta refresh tags that redirect to a different url. 2) script fragments that use top.location=newurl 3) script fragments that use window.open(some url) to do popups 4) I think window.href(new url) works similarly to top.location 5) and then there's the body onload=submit(something) that can auto-submit a form, thereby pushing you to a new url. 6) script fragments that do a submit(), as in #5 Did I miss any? Yes, I *have* seen all of the above. Mostly separately, but on one occasion most of the above 6 redirect forms happened one after another from the same web server - that wasn't pretty... :) Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: Calling JavaScript Functions using Curl
On Wed, Sep 22, 2010 at 2:26 PM, bharadwaj nagendra
bharadwaj.nagen...@gmail.com wrote:
Hi All,
I am working on a shell script to download data from website . I am using
cURL libraries.
I encountered a page where the button on click event looks like
a id=theForm:downloadLabel1 href=# style=text-decoration: none;
color: rgb(255, 255, 255); onclick=if(typeof jsfcljs ==
'function'){jsfcljs(document.forms['theForm'],'theForm:downloadLabel1,theForm:downloadLabel1','');}return
false class=buttonSummary/a.
can anyone show the way to use cURL libraries and call the onclick event in
the above scenario.
The problem you're going to run into is in evaluating this:
document.forms['theForm']
document.forms is a construct created by the web browser while parsing the
html. Curl doesn't do that, so you can't reference document.forms.
Ralph Mitchell
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: HELP: how can i do POST with setting URL to /
2010/7/29 xiusand...@yahoo.co.jp dear all: thank you for reading this mail~ i need to create a http client for sending a http request to a http server as following format: POST / HTTP/1.1 HOST: 192.168.1.2 Accept: ... ... Content-type: ... ... ... ... so i write a source code like this: easy_set_opt(curl, URL, / ); curl_slist_append(slist, HOST: 192.168.1.2); ... ... curl_easy_setopt(curl, CURLOPT_HTTPHEADER, slist); curl_easy_perform(curl); when i run this program, i got a error message like this: fail to resolve URL / if someone know something about this please give me a help! The url is supposed to be the whole url: curl_easy_setopt (curl, CURLOPT_URL, http://192.168.1.2/;); The man page says if you leave off the protocol part (http://, or ftp://, etc) it will try to guess based on the hostname. You're not even giving a hostname or ip address. Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curling password protected website with hidden dynamic variable on the login page
On Wed, Feb 3, 2010 at 2:10 PM, Maximilian Rausch maxrau...@gmail.comwrote: I need to curl data that is on a password protected site and I am first trying to get by the login page so that I can store the cookies which will allow me to get to the password protected data with a second call to curl. The login webpage is https://secure.ngx.com/sso/login However when the submit button is clicked on the login page it posts a hidden variable lt which changes every time the page reloads (so you can't curl the login page once and parse it to find the value and then post it the next time). You can see this by taking a look at the source code of the page. I have tried a live HTTP replay ( a function of the live http headers plugin for firefox) but the replay does not successfully login because I think that the the new page has a different value for lt while the replay is posting the old value. I can not figure out how to get the value of 'lt' the first time I curl the page so that I can post it along with the username, password, and other necessary variables. If you save that first page, you can grep out all the input fields, including the hidden ones: grep -i 'input' home.html You'll want to leave out the commented out warn field and the reset button. You can also extract the form using Daniel's formfind.pl program, which you can find here: http://curl.haxx.se/cvs.cgi/curl/perl/contrib/formfind?view=markupcontent-type=text/vnd.viewcvs-markuprevision=HEAD That'll give you the correct form elements and also the action url to post to. Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curling password protected website with hidden dynamic variable on the login page
On Wed, Feb 3, 2010 at 9:57 PM, Ralph Mitchell ralphmitch...@gmail.comwrote: On Wed, Feb 3, 2010 at 2:10 PM, Maximilian Rausch maxrau...@gmail.comwrote: I need to curl data that is on a password protected site and I am first trying to get by the login page so that I can store the cookies which will allow me to get to the password protected data with a second call to curl. The login webpage is https://secure.ngx.com/sso/login However when the submit button is clicked on the login page it posts a hidden variable lt which changes every time the page reloads (so you can't curl the login page once and parse it to find the value and then post it the next time). You can see this by taking a look at the source code of the page. I have tried a live HTTP replay ( a function of the live http headers plugin for firefox) but the replay does not successfully login because I think that the the new page has a different value for lt while the replay is posting the old value. I can not figure out how to get the value of 'lt' the first time I curl the page so that I can post it along with the username, password, and other necessary variables. If you save that first page, you can grep out all the input fields, including the hidden ones: grep -i 'input' home.html You'll want to leave out the commented out warn field and the reset button. You can also extract the form using Daniel's formfind.pl program, which you can find here: http://curl.haxx.se/cvs.cgi/curl/perl/contrib/formfind?view=markupcontent-type=text/vnd.viewcvs-markuprevision=HEAD That'll give you the correct form elements and also the action url to post to. Heh - I should have looked harder at the email headers. You don't want a command line solution. What are you doing with the downloaded page?? Are you saving it somewhere? If not, take a look at the getinmemory.c program: http://curl.haxx.se/libcurl/c/getinmemory.html That shows you how to save the downloaded page in a buffer for further processing. Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: curling password protected website with hidden dynamic variable on the login page
On Thu, Feb 4, 2010 at 12:03 AM, Maximilian Rausch maxrau...@gmail.comwrote:
From: William Betts william.betts_at_gmail.com
Date: Wed, 3 Feb 2010 19:14:44 -0800
On Wed, Feb 3, 2010 at 11:10 AM, Maximilian Rausch
maxrausch_at_gmail.comwrote:
I need to curl data that is on a password protected site and I am
first trying to get by the login page so that I can store the cookies
which will allow me to get to the password protected data with a
second call to curl. The login webpage is
https://secure.ngx.com/sso/login
However when the submit button is clicked on the login page it posts a
hidden variable lt which changes every time the page reloads (so you
can't curl the login page once and parse it to find the value and then
post it the next time). You can see this by taking a look at the
source code of the page. I have tried a live HTTP replay ( a function
of the live http headers plugin for firefox) but the replay does not
successfully login because I think that the the new page has a
different value for lt while the replay is posting the old value.
I can not figure out how to get the value of 'lt' the first time I
curl the page so that I can post it along with the username, password,
and other necessary variables.
Max
Hello,
After looking at the login page I assume you're talking about input
type=hidden name=lt
value=_cDA353F3E-64C2-4DF8-A7C6-6478B2BAC095_k3808E2D5-62BA-0552-1E53-C2AC9C4E71AE/.
First you'll want to fetch the login page and store it in some variable.
Then run the regex below on it. The example is in PHP, but the regex
should
work for other languages as well. The regular expression works, but could
probably be made better.
preg_match_all('input\s*type=hidden\s*name=lt\s*value=(.*?)\s*\/',
$html, $matches);
Best of Luck,
William
That is correct I am looking for that value and this idea is what I
had originally thought of as a solution but there is a problem with
it.
I curl the page and use a regex to find the value of 'it', but then
how do I submit the form with this value in the same call to curl?
If you reload the page (or call curl again with the value of 'lt' from
which I found previously) then the login will not work. This is
because when you reload the page the value of 'lt' changes every time.
I tried saving the session ID cookie but the value of 'lt' still
changes every time the page is reloaded.
Does anyone know of another way to accomplish a login to this website
using curl?
It's a form, so you need to do a POST to submit the form, not a GET, which
just gets you a fresh copy of the original page. If you're doing this in C,
it's something like this, with appropriate replacements for ,
:
curl_easy_setopt(hnd, CURLOPT_URL,
https://secure.ngx.com/sso/login;jsessionid=02C40047EDC6229287C1264D2EE9808A
);
curl_easy_setopt(handle, CURLOPT_POSTFIELDS,
username=password=lt=_eventId=submitsubmit=Login);
curl_easy_setopt(handle, CURLOPT_POST, 1);
though I haven't done a lot with libcurl, so I've probably left out some
stuff.
Ralph Mitchell
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: enableEventValidation=true
On Sat, Jan 16, 2010 at 1:04 PM, xml...@aol.com wrote: Greetings. My target site (.aspx pages) requires that I pass it begin and end dates, and as soon as the year became 2010, my PHP cURL script is no longer working. The exact error message is: Invalid postback or callback argument. Event validation is enabled using pages enableEventValidation=true/ in configuration or %@ Page EnableEventValidation=true % in a page. For security purposes, this feature verifies that arguments to postback or callback events originate from the server control that originally rendered them. If the data is valid and expected, use the ClientScriptManager.RegisterForEventValidation method in order to register the postback or callback data for validation. For any dates previous to January 1, 2010, my script works fine. (The data I'm scraping is from county records and is public information) Any insight on how to fix this would be greatly appreciated. All I see there is my program broke, how do I fix it? We're going to need some more detail to be able to make any sensible suggestions. For example, how are you passing the date? What curl version? Have you tried the same page fetch using the command-line tool or a browser, and did it work as expected? Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: libcurlnet 1.3 compatible with libcurl 7.19.6
On Sat, Oct 3, 2009 at 2:33 PM, Joerg Giese giese_gcwcl_l...@arcor.dewrote: Hi all, I have successfully implemented an FTP transfer in my C# project, but only using libcurl 7.13.0 as supplied with libcurlnet 1.3. As soon as I exchange lc 7.13 for lc 7.19, my C# program terminates with an exception, stating that libcurl.dll could not be found. Considering that lc 7.13 is quite a few years old and probably has some unfixed security issues, I would prefer to use the newer library. Did anyone else observe this behaviour? Any hints how to fix it? Any help is greatly appreciated! Just to get the silly questions out of the way - you did recompile your program after updating curl, right?? Ralph Mitchell --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: Question about cookies
On Sep 11, 2009, at 2:37 PM, Ralph Mitchell wrote:
On Fri, Sep 11, 2009 at 1:18 PM, Jonathan Wallace
jwall...@livetechnology.com wrote:
Is it possible to get a list of cookies from the return header so that I
can tokenize it and record them in my application. Currently I'm using a
CURLOPT_DEBUGFUNCTION and CURLOPT_VERBOSE to retrieve the header and
tokenize and I'm wondering if there is a more elegant and faster way to do
this?
How about CURLINFO_COOKIELIST?? That gets you a linked list of all cookies
curl knows about.
Ralph Mitchell
On Fri, Sep 11, 2009 at 1:51 PM, Jonathan Wallace
jwall...@livetechnology.com wrote:
That looks promising, but how do I use it? I see two public methods on the
curl_slist struct, data and next. How do I get the cookie's name and data
out of it?
I wouldn't know a public method if it jumped up and bit me. It's been a
while since I did any C, but what I see in the curl.h include file is:
struct curl_slist {
char *data;
struct curl_slist *next;
};
so I'm thinking if you have a pointer 'p' to that struct, then after calling
curl_easy_getinfo, you'd pick up your first cookie from p-data. To get to
the next cookie, something like p=p-next gets you your next cookie.
Lather, rinse, repeat until p-next is null. Or something like that...
Like I said, it's been awhile, so don't flame me on syntax... :)
Ralph Mitchell
Re: UNSUBSCRIBE
On Sun, Sep 13, 2009 at 7:23 PM, Dick Fleming dickflem...@hypermediadesign.com wrote: -- President - Hypermedia Design - dickflem...@hypermediadesign.com dickflem...@hypermediadesign.com Chief Operating Officer - Collaboration Studios dickflem...@collabortionstudios.com 207-348-5230 (voice) 207-669-8375 (FAX) www.hypermediadesign.com www.collaborationstudios.com --CS Finalist in TechMaine's TECHNOLOGY COMPANY OF THE YEAR AWARD !!! Read all about it: http://www.collaborationstudios.com It's right there in the email headers: List-Unsubscribe: http://cool.haxx.se/cgi-bin/mailman/options/curl-library mailto:curl-library-requ...@cool.haxx.se?subject=unsubscribe And, btw, you might want to fix that typo in your Collaboration Studios email... Ralph Mitchell
Re: Question about cookies
On Fri, Sep 11, 2009 at 1:18 PM, Jonathan Wallace jwall...@livetechnology.com wrote: Is it possible to get a list of cookies from the return header so that I can tokenize it and record them in my application. Currently I'm using a CURLOPT_DEBUGFUNCTION and CURLOPT_VERBOSE to retrieve the header and tokenize and I'm wondering if there is a more elegant and faster way to do this? How about CURLINFO_COOKIELIST?? That gets you a linked list of all cookies curl knows about. Ralph Mitchell
Re: How to install latest version of libcurl
On Wed, Aug 12, 2009 at 5:30 AM, Kamil Dudka kdu...@redhat.com wrote: On Wednesday 12 of August 2009 11:02:54 Raghu PV wrote: Hi, Recently i started using libcurl 7.15.5 version on Redhat Linux 5 system to perform HTTP POST for upload test. I find some issues with libcurl 7.15.5 for multithreaded upload using pthreads in Linux. I would like to install latest version of libcurl (7.19.5) for Redhat Linux 5 system. Since am quite new to using libcurl, i would request anyone who has knowledge about this to list the procedure(commands) used to build the latest version from source? Generally you have two choices: 1) Install it as a RPM package. You can try to build RPM yourself from using a SRPM package. 2) Install the upstream version directly (independently on your distro) - in this case its highly recommended to configure with --prefix=/usr/local to avoid clash with the files managed by rpm. You can follow the instructions here: http://curl.haxx.se/docs/install.html When installing from source, if you specify the version number in the prefix, you could have multiple versions installed in /usr/local: ./configure --prefix=/usr/local/curl-7.19.5 The install step creates all the necessary directories: /usr/local/curl-7.19.5/bin /usr/local/curl-7.19.5/lib etc.I had about 10 or 12 copies installed one time, while I was trying to track down an issue with the curl executable. This structure also fits nicely with testing a new release of curl before pushing it out into production. Ralph Mitchell
Re: File type to download
On Mon, Jun 22, 2009 at 6:15 PM, jvp j...@compu-weigh.com.au wrote: Hi all As I have had no response for the last few days, I have to assume: 1: This is the wrong forum to post these kind of questions or 2: I have done something wrong that is so obvious, nobody is game to tell me. Your original message was dropped into my spam folder by Gmail. The reason given was this may not be from the person it appears to be from. Ralph Mitchell
Re: Error 407 with Proxy authentication
. The timeout may have o 08c0: ccurred due to Internet congestion. 08e5: LI id=L_default_7BCheck spelling:/B Check that you typed t 0925: he Web page address correctly. The address may have been mistype 0965: d. 0969: LI id=L_default_8BAccess from a link:/B If there is a link 09a9: to the page you are looking for, try accessing the page from th 09e9: at link. 09f3: 09f5: /UL 0a02: ID id=L_default_9If you are still not able to view the request 0a42: ed page, try contacting your administrator or Helpdesk./ID BR 0a82: BR 0a89: /TD 0a94: /TR 0a9d: /TBODY 0aa9: /TABLE 0ab3: 0ab5: TABLE id=spacerTBODYTRTD height=15/TD/TR/TBODY/TA 0af5: BLE 0afb: 0afd: TABLE width=400 0b10: TBODY 0b1b: TR 0b23: TD noWrap width=25/TD 0b42: TD width=400 id=L_default_10BTechnical Information (for 0b82: support personnel)/B 0b9b: UL class=adminList 0bb7: LI id=L_default_11Error Code: 407 Proxy Authentication 0bf7: Required. The ISA Server requires authorization to fulfill the 0c37: request. Access to the Web Proxy filter is denied. (12209) 0c73: LI id=L_default_12IP Address: 10.10.82.13 0ca0: LI id=L_default_13Date: 4/29/2009 6:25:30 AM 0cd0: LI id=L_default_14Server: isa02.domain.com 0d07: LI id=L_default_15Source: proxy 0d2a: 0d2c: /UL 0d39: /TD 0d44: /TR 0d4d: /TBODY 0d59: /TABLE 0d63: 0d65: /BODY 0d6e: /HTML 0d77: == Info: Connection #0 to host myproxy left intact == Info: Closing connection #0 Thank you so much!!! -Original Message- From: curl-library-boun...@cool.haxx.se [mailto:curl-library-boun...@cool.haxx.se] On Behalf Of Daniel Stenberg Sent: Tuesday, April 28, 2009 11:34 PM To: libcurl development Subject: Re: Error 407 with Proxy authentication On Tue, 28 Apr 2009, AMBROSINI Marco wrote: curl -u DOMAIN\USER -x myproxy:Port http://www.google.com to get Google page testing the proxy configuration, I get the error Error code: 407 proxy authentication required Add -v too, and check the response headers. I'm sure it says it wants some other authorization method than plain Basic, which curl defaults to. Are you just making the one attempt to get through the proxy?? I once had a problem with a proxy where I had to make the first attempt without the userid/passwd, get the proxy auth required response, then try again *with* the userid/passwd. It would not let me get through on the first try, even with the userid/passwd. BTW, you should change that password - the encoding is really easy to reverse... Ralph Mitchell
Re: About libcurl on Solaris 10
On Wed, Apr 22, 2009 at 9:34 PM, CHEN Xiaolei A xiaolei.a.c...@alcatel-sbell.com.cn wrote: Hello everyone, Nice to see you. I am a new comer to this maillist, I hope to study how to use libcurl to implement SFTP and make friends with you! Just as the subject said, now i am working on sun solaris 10 platform, and want to download libcurl for solaris. But I can not find a corresponding version on url: *http://curl.haxx.se/dlwiz/* http://curl.haxx.se/dlwiz/. Do you know where could i download the libcurl for solaris 10? Thanks a lot! You could download the latest source and compile it. A full installation of Solaris 10 should include GCC, probably in /usr/sfw/bin. Ralph Mitchell
