Re: Avoid copying data in CURLOPT_WRITEFUNCTION callback

2012-02-15 Thread Rich Gray
Konstantin Miller wrote: Hi! Is there any way to avoid copying data between the buffer that is passed to the callback function, which is set with CURLOPT_WRITEFUNCTION, and my own buffer? Can I tell libcurl that I would like to reuse its buffer and that I will free it myself later on? Or, even

Re: Avoid copying data in CURLOPT_WRITEFUNCTION callback

2012-02-15 Thread Daniel Stenberg
On Wed, 15 Feb 2012, Rich Gray wrote: This capability would have to be usable from within a write callback function to allow the application to do things like cause libcurl to place successive returns in contiguous memory. (I don't believe I've seen anything about calling libcurl functions

Avoid copying data in CURLOPT_WRITEFUNCTION callback

2012-02-13 Thread Konstantin Miller
Hi! Is there any way to avoid copying data between the buffer that is passed to the callback function, which is set with CURLOPT_WRITEFUNCTION, and my own buffer? Can I tell libcurl that I would like to reuse its buffer and that I will free it myself later on? Or, even better, that I will give it

Re: Avoid copying data in CURLOPT_WRITEFUNCTION callback

2012-02-13 Thread Dan Fandrich
On Mon, Feb 13, 2012 at 10:36:04AM +0100, Konstantin Miller wrote: Is there any way to avoid copying data between the buffer that is passed to the callback function, which is set with CURLOPT_WRITEFUNCTION, and my own buffer? Can I tell libcurl that I would like to reuse its buffer and that I

Re: Avoid copying data in CURLOPT_WRITEFUNCTION callback

2012-02-13 Thread Daniel Stenberg
On Mon, 13 Feb 2012, Dan Fandrich wrote: It was removed from the list a few months ago, presumably due to lack of interest. Indeed. if I would reconsider it, someone would have to show a significant performance boost with the use of something like that. -- / daniel.haxx.se

RE: Avoid copying data in CURLOPT_WRITEFUNCTION callback

2012-02-13 Thread Yehezkel Horowitz
It was removed from the list a few months ago, presumably due to lack of interest. Indeed. if I would reconsider it, someone would have to show a significant performance boost with the use of something like that. Since such approach will require dynamic buffer allocation, I think this

RE: Avoid copying data in CURLOPT_WRITEFUNCTION callback

2012-02-13 Thread Daniel Stenberg
On Mon, 13 Feb 2012, Yehezkel Horowitz wrote: if I would reconsider it, someone would have to show a significant performance boost with the use of something like that. Since such approach will require dynamic buffer allocation, I think this could improve the memory foot print as well.

RE: Avoid copying data in CURLOPT_WRITEFUNCTION callback

2012-02-13 Thread Daniel Stenberg
On Mon, 13 Feb 2012, Yehezkel Horowitz wrote: Currently each easy handle has its own 2 buffers with fixed size of 16K. This enforces the application that uses libcurl to copy the data when the callbacks are called, since it will not be valid after the callback return. I understand

Re: Avoid copying data in CURLOPT_WRITEFUNCTION callback

2012-02-13 Thread Lars Nilsson
On Mon, Feb 13, 2012 at 7:39 AM, Daniel Stenberg dan...@haxx.se wrote: On Mon, 13 Feb 2012, Yehezkel Horowitz wrote: Currently each easy handle has its own 2 buffers with fixed size of 16K. This enforces the application that uses libcurl to copy the data when the callbacks are called, since

Re: Avoid copying data in CURLOPT_WRITEFUNCTION callback

2012-02-13 Thread Konstantin Miller
An interface as suggested by Yehezkel would be really great! It would give the application the flexibility to implement its own memory management. It still would be very helpful, even if the minimum buffer size would be 2 x 16K, as pointed out by Daniel. I think that if you are targeting a video

RE: Avoid copying data in CURLOPT_WRITEFUNCTION callback

2012-02-13 Thread Aleksandar Lazic
On 13-02-2012 13:20, Daniel Stenberg wrote: On Mon, 13 Feb 2012, Yehezkel Horowitz wrote: I think that 'CURLcode curl_allocate_buffer_cb(char** buff, int* size)' which will be called by curl with the requested buffer size - would be easy to use API for this. Yes, I agree it could probably