Re: sftp download issue (Daniel Stenberg)

[Sumukh Anantha Manohar]

Hi,
  As i had posted earlier,

 Do you always get this same error (and byte amount) no matter how large
file
 you get? If not, do you get the same error and byte amount for the same
file
 if you retry the exact same transfer?

 Eh, so does it work for _any_ sftp files at all? If so, which?

Even if i try to download a 2MB file with sftp protocol, sometimes it will
download completely and sometimes it will show the error : **curl: (18)
transfer closed with 574 bytes
  remaining to read*.  So, this problem is arising for smaller file sizes
also (sometimes).  No matter how large the file is , i will get the same
error.

Libraries: libssh2-1.2.9
I am using Curl-7.21.7 package to generate the libcurl on the 64-bit solaris
machine, here in the file lib/sendf.c, there is a piece of code having:

/* If session can pipeline, check connection buffer  */
  if(pipelining) {
size_t bytestocopy = CURLMIN(conn-buf_len - conn-read_pos,
 sizerequested);

/* Copy from our master buffer first if we have some unread data there*/
if(bytestocopy  0) {
  memcpy(buf, conn-master_buffer + conn-read_pos, bytestocopy);
  conn-read_pos += bytestocopy;
  conn-bits.stream_was_rewound = FALSE;

  *n = (ssize_t)bytestocopy;
  return CURLE_OK;
}
/* If we come here, it means that there is no data to read from the
buffer,
 * so we read from the socket */
bytesfromsocket = CURLMIN(sizerequested, BUFSIZE * sizeof (char));
buffertofill = conn-master_buffer;
  }
  else {
bytesfromsocket = CURLMIN((long)sizerequested,
  conn-data-set.buffer_size ?
  conn-data-set.buffer_size : BUFSIZE);
/* * bytesfromsocket = 1;*  */
   /* *OR* */
   /* * bytesfromsocket = 16384(=BUFSIZE);*  */
buffertofill = buf;
  }

See the else part, where bytesfromsocket is assigned with some values,
normally it will have value like 16384 bytes each time, if the remaining
file size  to be downloaded is large. Once the remaining file size to be
downloaded is between 16384 and 1, bytesfromsocket  will be equal to
sizerequested (less than 16384), then i am getting the error as .*curl: (18)
transfer closed with 4456 bytes remaining to read*.
So, i changed bytesfromsocket to a constant value 1  or to
16384(BUFSIZE) which i have commented, and again rebuilded the library
libcurl.so.4, After doing this, it was perfectly downloading for all the
file sizes. Want to know what i have changed is right or not?

Thanks
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Curl Issue..........

[Sumukh Anantha Manohar]

Hi,
I am trying to download files whose size is greater than 2GB using curl
API's like curl_easy_perform ( ) and curl_easy_setopt ( ) on linux and unix
platforms using c++ language.

The problem is, the above functions are helping me to download upto 2GB file
size and not more than that.
Here what i am doing isI am calling the statement:

curl_easy_setopt(curlHandle_m, CURLOPT_WRITEFUNCTION,
FileTransfer::CallBack);

Then, it calls the CallBack function which tries to download the file part
by part in bytes into the destination place.But once the file reaches 2GB
size it is not able to write the remaining data into the file.

Following is the Callback function:

size_t FileTransfer::CallBack(void *buffer, size_t size, size_t nmemb, void
*stream) {

struct DownloadFileName *out = (struct DownloadFileName *) stream;
if (out  !out-stream) {

LOG_DEBUG(ACE_TEXT(While downloading file name as: %s\n), out-filename);

// open file for writing
out-stream = ACE_OS::fopen(out-filename, wb);

if (!out-stream) {

  // Failure, can't open file to write
  return CURLE_WRITE_ERROR;
}
}
  return ACE_OS::fwrite(buffer, size, nmemb, out-stream);
}

So plz anyone help me regarding this issue..

Thanks,
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html