I believe it was set down that low (in a couple places! Perlbal,
mogilefs...) because larger values were occasionally crashing perl for
some people. Making it optional looks totally reasonable though.
I'll review/apply this if someone doesn't beat me to it. At a glance it
looks fine but I like doing quick style/test checks ;)
-Dormando
Tomas Doran wrote:
Hiya
I've found (what I think are) 2 problems in the Mogile client library.
1) In the store_file method, there is a loop which reads segments of the
file, and prints them to the file handle (which depending on which
client backend you're using, does different things). The block size is
fixed to 8192 bytes here, I've changed it to use the supplied
readLineChunkSize param.
We're doing chunked (range PUT, with ClientHTTPFile) uploads of some
very big files, and this change makes a *massive* difference to the
throughput that we see when we upload chunks of 4Mb in size, rather than
8k.
2) In the same method - a the print on the filehandle can fail, and this
is currently silently ignored. As we're doing multiple PUT requests per
file then occasionally one of these requests times out, and I'd like to
retry once per file.
Therefore I also added a 'retries' parameter to the options to
store_file, and fixed the bug with not checking the return value of the
print method on the file handle.
Patch attached, any feedback you have welcome :)
Cheers
Tom
