>From what I recall we did have one user who did have this problem and it was something like mail processing where all header lines got sliced from an array of the entire email and then retained... :)
In this case, the COW leak was nearly impossible to understand until visually inspecting the lines in the arrays in a debugger. I am guessing you are right that most people probably slice chunks of a larger thing for display purposes, so possibly we can look for gems which do that. I suspect most pagination gems won't do this, but it is a good avenue of inquiry. Even in this case those Arrays will go away after a request is done. Hmm. -Tom On Wed, Oct 26, 2011 at 8:36 AM, Bob McWhirter <bmcwh...@redhat.com> wrote: > Typical case I've used it was to slice things for columnar display. I end up > using the entire array, but just in N chunks of M items at a time. > > I don't recall ever taking a slice and discarding the rest. > > fwiw. > > -Bob > > > On Oct 26, 2011, at 9:28 AM, Thomas E Enebo wrote: > >> Does anyone do a lot of sliciing (e.g. things like [1...100]) in their >> apps. Are they big arrays? Does anyone know a library which does? >> >> -Tom >> >> On Tue, Oct 25, 2011 at 6:46 PM, Charles Oliver Nutter >> <head...@headius.com> wrote: >>> Long ago when the world was new, Marcin Mielzynski modified our Array >>> implementation to be copy-on-write like MRI. And there was much >>> rejoicing. >>> >>> COW helps methods like slice and dup that would normally need to make >>> a copy of all or part of the source array by wrapping the original >>> array in "shared" mode. Subsequent modifications force the original >>> contents to be copied into a new array, but for read-only subsets >>> (especially if they're big) it's a win. >>> >>> However, it suffers from a big down side: if you have a really large >>> array holding onto many large objects, and you slice out a small >>> subset of that array, the original contents stick around. This is like >>> a pseudo-leak, since the memory the rest of the array is rooting never >>> gets collected. We've patched around this in a few key methods. >>> >>> So today I thought I'd try removing COW from Array completely and >>> comparing numbers. Here's the result and the patch: >>> >>> https://gist.github.com/beab95be678c519e8a24 >>> >>> The bottom line is that it definitely has an impact. Any operation >>> that would create a new view of N elements of the original array goes >>> from being O(1) (the time needed to wrap the old array contents in a >>> new object) to O(N) (the time needed to copy all those elements into a >>> new array. >>> >>> I post this here so anyone interested in testing it against real code >>> or real applications can do so. I just used some simple benchmarks >>> from Rubinius, which certainly aren't going to behave like a typical >>> app. Maybe the degradation is small enough for real apps that the >>> GC/memory improvements would be worth it? >>> >>> - Charlie >>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >> >> >> >> -- >> blog: http://blog.enebo.com twitter: tom_enebo >> mail: tom.en...@gmail.com >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> > > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > -- blog: http://blog.enebo.com twitter: tom_enebo mail: tom.en...@gmail.com --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
