Re: [PATCH] Fix in Git.pm cat_blob crashes on large files

2013-02-22 Thread Joshua Clayton
running git svn fetch on a remote repository (yes I know there are a lot of possible outside variables, including network latency) Code with 1024 reads and 64k writes: real75m19.906s user16m43.919s sys 29m16.326s Code with 1024 reads and 1024 writes: real71m21.006s user

Re: [PATCH] Fix in Git.pm cat_blob crashes on large files

2013-02-22 Thread Jeff King
On Fri, Feb 22, 2013 at 07:11:54AM -0800, Joshua Clayton wrote: running git svn fetch on a remote repository (yes I know there are a lot of possible outside variables, including network latency) Code with 1024 reads and 64k writes: real75m19.906s user16m43.919s sys 29m16.326s

[PATCH] Fix in Git.pm cat_blob crashes on large files (reviewed by Jeff King)

2013-02-22 Thread Joshua Clayton
Read and write each 1024 byte buffer, rather than trying to buffer the entire content of the file. Previous code would crash on all files 2 Gib, when the offset variable became negative (perhaps below the level of perl), resulting in a crash. On a 32 bit system, or a system with low memory it

Re: [PATCH] Fix in Git.pm cat_blob crashes on large files (reviewed by Jeff King)

2013-02-22 Thread Erik Faye-Lund
On Fri, Feb 22, 2013 at 6:03 PM, Joshua Clayton stillcompil...@gmail.com wrote: Read and write each 1024 byte buffer, rather than trying to buffer the entire content of the file. Previous code would crash on all files 2 Gib, when the offset variable became negative (perhaps below the level of

[PATCH] Fix in Git.pm cat_blob crashes on large files (resubmit with reviewed-by)

2013-02-22 Thread Joshua Clayton
Read and write each 1024 byte buffer, rather than trying to buffer the entire content of the file. Previous code would crash on all files 2 Gib, when the offset variable became negative (perhaps below the level of perl), resulting in a crash. On a 32 bit system, or a system with low memory it

Re: [PATCH] Fix in Git.pm cat_blob crashes on large files (resubmit with reviewed-by)

2013-02-22 Thread Jeff King
On Fri, Feb 22, 2013 at 09:30:57AM -0800, Joshua Clayton wrote: Read and write each 1024 byte buffer, rather than trying to buffer the entire content of the file. OK. Did you ever repeat your timing with a larger symmetric buffer? That should probably be a separate patch on top, but it might

Re: [PATCH] Fix in Git.pm cat_blob crashes on large files (resubmit

2013-02-22 Thread Jeff King
On Fri, Feb 22, 2013 at 09:30:57AM -0800, Joshua Clayton wrote: Subject: Re: [PATCH] Fix in Git.pm cat_blob crashes on large files (resubmit with reviewed-by) One thing I forgot to note in my other response: the subject is part of the commit message, so information like resubmit

Re: [PATCH] Fix in Git.pm cat_blob crashes on large files (resubmit with reviewed-by)

2013-02-22 Thread Junio C Hamano
. On a 32 bit system, or a system with low memory it might crash before reaching 2 GiB due to memory exhaustion. Signed-off-by: Joshua Clayton stillcompil...@gmail.com Reviewed-by: Jeff King p...@peff.net --- Thanks. Subject: Re: [PATCH] Fix in Git.pm cat_blob crashes on large files (resubmit

Re: [PATCH] Fix in Git.pm cat_blob crashes on large files (resubmit with reviewed-by)

2013-02-22 Thread Joshua Clayton
Thanks for all the input and patience. On Fri, Feb 22, 2013 at 10:34 AM, Jeff King p...@peff.net wrote: On Fri, Feb 22, 2013 at 09:30:57AM -0800, Joshua Clayton wrote: Read and write each 1024 byte buffer, rather than trying to buffer the entire content of the file. OK. Did you ever repeat

[PATCH] Fix in Git.pm cat_blob crashes on large files

2013-02-21 Thread Joshua Clayton
Greetings. This is my first patch here. Hopefully I get the stylistic political details right... :) Patch applies against maint and master (If I understand the mechanics, in theory a negative offset should work, if the values lined up just right, but would be very wrong, overwriting the lower

Re: [PATCH] Fix in Git.pm cat_blob crashes on large files

2013-02-21 Thread Jeff King
On Thu, Feb 21, 2013 at 02:13:32PM -0800, Joshua Clayton wrote: Greetings. This is my first patch here. Hopefully I get the stylistic political details right... :) Patch applies against maint and master I have some comments. :) The body of your email should contain the commit message

Re: [PATCH] Fix in Git.pm cat_blob crashes on large files

2013-02-21 Thread Joshua Clayton
On Thu, Feb 21, 2013 at 2:43 PM, Jeff King p...@peff.net wrote: On Thu, Feb 21, 2013 at 02:13:32PM -0800, Joshua Clayton wrote: Greetings. This is my first patch here. Hopefully I get the stylistic political details right... :) Patch applies against maint and master I have some comments.

Re: [PATCH] Fix in Git.pm cat_blob crashes on large files

2013-02-21 Thread Jeff King
On Thu, Feb 21, 2013 at 03:18:40PM -0800, Joshua Clayton wrote: By having the read and flush size be the same, it's much simpler. My original bugfix did just read 1024, and write 1024. That works fine and, yes, is simpler. I changed it to be more similar to the original code in case there