Just looking at your code briefly in-between college football Saturday
but it looks like you are closing your file inside your loop where you
are adding easy handles to the multi. I think that's probably the
source of your crash since the FILE* is invalid when the getpage call
is done. Make an array of FILE*'s. Keep them around until you're done.
Have each of them point to a different file (which looks like its also
an issue in the code). Good luck!
On Oct 3, 2009, at 12:10 PM, will griffiths wrote:
Hello,
I have written a full text indexing program in C++ and am now
building a spidering program and I intend to use libcurl for the
download side of things.
I am struggling a bit to transfer my curl multi code from PHP to C++
as I relied on the "getcontent" function quite heavily.
I have included the full source code below. When i turned verbose on
I got this output:
* Closing connection #0
* Closing connection #1
* Closing connection #2
followed by a crash. I probably need help understanding how the curl
functions work in C++. I have read the docs for the functions
"individually" and then strung them together by looking at egs in
Google Code Search.
Any help appreciated :)
Will
//#pragma comment(lib, "libcurl.lib") // now appending /link <DIR>
\libcurl.lib to cmd args instead
#include <string>
#include <iostream>
#include <map>
#include "curl/curl.h"
using namespace std;
FILE *fd;
static size_t getpage(void *incoming, size_t size, size_t nmemb,
void *page)
{
return fwrite(incoming, size, nmemb, (FILE *)page);
}
int main()
{
map<int, string> pass;
pass[0] = "http://chemrefer.com/index.html";;
pass[1] = "http://www.chemrefer.com/chemistry_search.php?page_title=news&submit
";
pass[2] = "http://www.chemrefer.com/chemistry_search.php?page_title=toolbar&submit
";
pass[3] = "http://www.chemrefer.com/chemistry_search.php?page_title=about&submit
";
pass[4] = "http://www.chemrefer.com/chemistry_search.php?page_title=contact&submit
";
CURL *h[5];
CURLM *mh;
mh = curl_multi_init();
for (int i = 0; i < 5; ++i) {
char *urlbuf = new char[pass[i].length()];
strcpy(urlbuf, pass[i].c_str());
h[i] = curl_easy_init();
fd = fopen("download.txt", "ab");
curl_easy_setopt(h[i], CURLOPT_URL, urlbuf);
curl_easy_setopt(h[i], CURLOPT_WRITEFUNCTION, getpage);
curl_easy_setopt(h[i], CURLOPT_WRITEDATA, fd);
curl_easy_setopt(h[i], CURLOPT_HEADER, 0);
curl_multi_add_handle(mh, h[i]);
delete[] urlbuf;
fclose(fd);
}
pass.clear();
int running;
CURLMcode result;
do {
result = curl_multi_perform(mh, &running);
if (result != CURLM_CALL_MULTI_PERFORM) break;
}
while (running > 0);
for (int i = 0; i < 5; ++i) curl_multi_remove_handle(mh, h[i]);
for (int i = 0; i < 5; ++i) curl_easy_cleanup(h[i]);
curl_multi_cleanup(mh);
}
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
