________________________________________
From: [email protected] [[email protected]] on
behalf of Dan Fandrich [[email protected]]
Sent: Wednesday, January 25, 2012 7:26 PM
To: [email protected]
Subject: Re: libcurl PUT retry and READFUNCTION
On Wed, Jan 25, 2012 at 08:09:19PM +0000, Wenjun Chen wrote:
> This is the actual code. It uses local variable, not on stack.
>It may be an issue elsewhere in your code (and your previous messages
>imply that), but you didn't show us that code. You'll have to be more
>specific about what the problem is including code if you'd like more
>specific help.
Thanks Dan for your help, the problem is as described in the first e-mail, I
copied below again.
I then use SEEK_FUNC call back to get more info. See code after
put_data_callback.
The normal sequences are
(1) put_data_callback is called and data is sent to server.
In case when TCP is terminated.
(1) put_data_callback is called, data is copied and sent (the first try)
(TCP terminated by server)
(2) seek_data_callback is called, but the dataPtr passed in is not correct.
The seek_data_callback is called when server terminated the TCP, but the
m_data->data_len is 0, Origin is also 0 (SEEK_SET), offset is also 0. I know
the seek_data_call back does not handle SEEK_CUR and SEEK_END correctly. The
handle of SEEK_SET should be ok.
(3) libcurl send 0 bytes to server
I try to debug.
(1) Why the m_data->data_len is 0 when seek_data_callback is called. While
m_data->data_len is correct.
(2) Why the read_data_callback is not called after seek_data_callback is
called. I assume this is the root cause of sending 0 bytes to server.
Thanks again,
Wenjun
### Original Problem description and all related function implementation ####
I use libcurl to put a file to a HTTP server. When HTTP server FIN TCP
connection, the curl retried and resend the http header. However, the data in
retry packet has 0 length.
The sequence of the packet is as follows. The 1.1.1.1 is client IP address, the
2.2.2.2 is server IP address
1 0.000000 1.1.1.1 -> 2.2.2.2 HTTP PUT
/webdav/arxce/1327331782-01XFR48JF-17.metadata HTTP/1.1 #### Initial PUT
request, send HTTP header
2 0.000105 2.2.2.2 -> 1.1.1.1 TCP www > 55199 [FIN, ACK] Seq=0 Ack=0
Win=27 Len=0 #### Server FIN the connection
3 0.000124 1.1.1.1 -> 2.2.2.2 HTTP Continuation or non-HTTP traffic
#### client send DATA 373 bytes. (Probably send before
process the TCP FIN packet).
4 0.000177 1.1.1.1 -> 2.2.2.2 TCP 55199 > www [FIN, ACK] Seq=533 Ack=1
Win=254 [TCP CHECKSUM INCORRECT] Len=0 ### Client FIN above TCP connection
5 0.000294 1.1.1.1 -> 2.2.2.2 TCP 55200 > www [SYN] Seq=0 [TCP CHECKSUM
INCORRECT] Len=0 MSS=1460 WS=8 #### Client started a new TCP connection
6 0.000496 2.2.2.2 -> 1.1.1.1 TCP www > 55199 [ACK] Seq=1 Ack=534 Win=36
Len=0
7 0.000498 2.2.2.2 -> 1.1.1.1 TCP www > 55200 [SYN, ACK] Seq=0 Ack=1
Win=5840 Len=0 MSS=1460 WS=8
8 0.000532 1.1.1.1 -> 2.2.2.2 TCP 55200 > www [ACK] Seq=1 Ack=1 Win=65536
[TCP CHECKSUM INCORRECT] Len=0
9 0.000600 1.1.1.1 -> 2.2.2.2 HTTP PUT
/webdav/arxce/1327331782-01XFR48JF-17.metadata HTTP/1.1 ### Retry PUT
request, send HTTP header
10 0.000785 2.2.2.2 -> 1.1.1.1 TCP www > 55200 [ACK] Seq=1 Ack=161 Win=6912
Len=0 #### Server ACK
11 0.000802 1.1.1.1 -> 2.2.2.2 HTTP Continuation or non-HTTP traffic
#### client resend the data (only 5 bytes
300d0a0d0a), it should send 373 bytes as initial data send.
12 0.000933 2.2.2.2 -> 1.1.1.1 TCP www > 55200 [ACK] Seq=1 Ack=166 Win=6912
Len=0 #### Server ACK
13 0.001284 2.2.2.2 -> 1.1.1.1 HTTP HTTP/1.1 201 Created (text/html)I saw a
similar issue reported here ### Server response with created (but only the file
is only 0 byte).
I read the e-mail trace and look like this is the similar problem and
SEEKFUNCTION is suggested here.
http://curl.haxx.se/mail/lib-2009-08/0401.html
What is the final solution to the issue described above? If SEEKFUNCTION is
used, does anyone have an example of how to use it.
Thank you very much for your help in advance.
Wenjun Chen
PS.
I use READFUNCTION call back to put data to the libcurl buff.
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
// set the put_data_callback function
curl_easy_setopt(curl, CURLOPT_PUT, true);
curl_easy_setopt(curl, CURLOPT_READDATA, &m_data);
curl_easy_setopt(curl, CURLOPT_READFUNCTION, put_data_callback);
curl_easy_perform(curl);
Here is my callback function.
struct put_data_t {
const char * data_start;
unsigned read_pos;
unsigned data_len;
};
size_t put_data_callback(void *buff, size_t size, size_t nmem, void* userp)
{
if (!userp)
return 0;
put_data_t *userdata = (put_data_t *)userp;
if (userdata->data_len <= 0)
return 0;
size_t curl_size = nmem*size;
std::ostringstream ss;
ss << "put_data_callback get " << curl_size << " data passed in
" << userdata->data_len;
DCAL_logCritical(ss.str().c_str());
size_t to_copy = (userdata->data_len < curl_size) ?
userdata->data_len : curl_size;
memcpy(buff, userdata->data_start + userdata->read_pos,
to_copy);
userdata->data_len -= to_copy;
userdata->read_pos += to_copy;
return to_copy;
}
#### The code is changed to add seek_data_callback ####
> // set the put_data_callback function
> curl_easy_setopt(curl, CURLOPT_PUT, true);
> curl_easy_setopt(curl, CURLOPT_READDATA, &m_data);
> curl_easy_setopt(curl, CURLOPT_READFUNCTION,
> put_data_callback);
>
> curl_easy_setopt(curl, CURLOPT_SEEKDATA, &m_data);
> curl_easy_setopt(curl, CURLOPT_SEEKFUNCTION,
> seek_data_callback);
> curl_easy_perform(curl);
> struct put_data_t {
> const char * data_start;
> unsigned read_pos;
> unsigned data_len;
>};
>int seek_data_callback(void *userp, curl_off_t offset, int origin)
>{
> if (!userp)
> return 0;
> put_data_t *userdata = (put_data_t *)userp;
> if (offset > userdata->data_len)
> return CURL_SEEKFUNC_FAIL;
> if (origin == SEEK_SET || origin == SEEK_CUR)
> {
> userdata->read_pos = offset;
> return CURL_SEEKFUNC_OK;
> }
> if (origin == SEEK_END)
> {
> userdata->read_pos = userdata->data_len;
> return CURL_SEEKFUNC_OK;
> }
> return CURL_SEEKFUNC_FAIL;
> }
>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
