No problem, glad we figured it out =)
On Nov 23, 7:57 am, Mads Sig Ager <[email protected]> wrote:
> This is happening because we exhaust map space. Currently, we have an
> 8MB limit on map space because of garbage collection encoding
> constraints. The issue here is that the objects go slow case and each
> object gets it's own map. We should attempt to not create new
> slow-case maps for each object, but use a canonical one if we can.
> Additionally, it would be nice to allow map space to grow larger.
>
> I'll file a V8 bug report on the issue. Simplified case:
>
> V8 version 2.0.2 (candidate)
>
> > var i = 1000000
> > var a = new Array(i)
> > for (var j = 0; j < i; j++) { var o = {}; o.x = 42; delete o.x; a[j] = o; }
>
> #
> # Fatal error in (null)
> # Allocation failed - process out of memory
> #
>
> Thanks for reporting this problem.
>
> -- Mads
>
>
>
> On Mon, Nov 23, 2009 at 1:48 PM, Anton Muhin <[email protected]> wrote:
>
> > Cool, that is reproducible on Linux, digging it further.
>
> > yours,
> > anton.
>
> > On Mon, Nov 23, 2009 at 3:21 PM, Anton Muhin <[email protected]> wrote:
> >> Thanks a lot, I can now reproduce it on my box.
>
> >> My immediate hypothesis would be it's due to virtual address space
> >> fragmentation: notice that v8 dies after performing 10 mark sweeps
> >> which fail to reclaim enough of garbage (last ones, going from 125.1
> >> -> 125.0, the last one 125.0 -> 125.0)---that should be enough to
> >> exhaust process memory space. But it's only a hypothesis. I'd check
> >> if it's reproducible under Linux and update.
>
> >> yours,
> >> anton.
>
> >> On Sat, Nov 21, 2009 at 12:38 AM, dracflamloc
> >> <[email protected]> wrote:
>
> >>> Sorry for so many replies in a row, but I've isolated the code that
> >>> crashes v8. You can test this with the normal d8.exe and it crashes at
> >>> the same point my custom exe does.
>
> >>>http://pastebin.com/m2f610e8
>
> >>> On Nov 20, 4:29 pm, dracflamloc <[email protected]> wrote:
> >>>> Small update: I rebuilt the lib using the visual studio sln instead of
> >>>> scons, and now the splay test runs fine using my executable. However
> >>>> my js still crashes. Can someone take a look at the cpp code and let
> >>>> me know if theres something I'm missing? Thanks
>
> >>>> On Nov 20, 3:53 pm, dracflamloc <[email protected]> wrote:
>
> >>>> > What do you mean by allocate a 'thing' in between?
>
> >>>> >http://pastebin.com/m10af9629
>
> >>>> > Here is the cpp code for my exe.
>
> >>>> > On Nov 20, 3:13 pm, Anton Muhin <[email protected]> wrote:
>
> >>>> > > Could you post the code? If we could reproduce that under Windows,
> >>>> > > it
> >>>> > > might be easier to understand what goes on.
>
> >>>> > > How do you build v8, btw?
>
> >>>> > > I can run Erik's variant of splay just fine:
>
> >>>> > > d:\chromium\src\v8>shell.exe public-splay.txt --trace-g
> >>>> > > Scavenge 0.7 -> 0.6 MB, 2 ms.
> >>>> > > Scavenge 1.0 -> 1.0 MB, 2 ms.
> >>>> > > Scavenge 1.3 -> 1.3 MB, 2 ms.
> >>>> > > Scavenge 2.1 -> 2.0 MB, 5 ms.
> >>>> > > Scavenge 2.8 -> 2.8 MB, 5 ms.
> >>>> > > Mark-sweep 4.3 -> 4.2 MB, 20 ms.
> >>>> > > Scavenge 5.7 -> 5.6 MB, 7 ms.
> >>>> > > Mark-sweep 8.6 -> 8.5 MB, 38 ms.
> >>>> > > Mark-sweep 11.5 -> 11.3 MB, 45 ms.
> >>>> > > Scavenge 17.3 -> 17.1 MB, 31 ms.
> >>>> > > Mark-sweep 23.1 -> 22.8 MB, 99 ms.
> >>>> > > Mark-sweep 28.8 -> 28.5 MB, 119 ms.
> >>>> > > Scavenge 34.5 -> 34.2 MB, 34 ms.
> >>>> > > Mark-sweep 39.3 -> 39.1 MB, 156 ms.
> >>>> > > Scavenge 45.1 -> 44.8 MB, 35 ms.
> >>>> > > Mark-sweep 50.8 -> 50.5 MB, 191 ms.
> >>>> > > Scavenge 56.5 -> 56.3 MB, 34 ms.
> >>>> > > Scavenge 62.3 -> 62.0 MB, 33 ms.
> >>>> > > Mark-sweep 68.0 -> 67.7 MB, 249 ms.
> >>>> > > Scavenge 73.7 -> 73.4 MB, 34 ms.
> >>>> > > Scavenge 79.4 -> 79.2 MB, 35 ms.
> >>>> > > Scavenge 85.2 -> 84.9 MB, 35 ms.
> >>>> > > Mark-sweep 90.9 -> 90.6 MB, 323 ms.
> >>>> > > Scavenge 96.6 -> 96.4 MB, 36 ms.
> >>>> > > Scavenge 102.4 -> 102.1 MB, 36 ms.
> >>>> > > Scavenge 108.1 -> 107.8 MB, 36 ms.
> >>>> > > Scavenge 113.8 -> 113.5 MB, 37 ms.
> >>>> > > Mark-sweep 119.5 -> 119.3 MB, 371 ms.
> >>>> > > Scavenge 125.3 -> 125.0 MB, 37 ms.
> >>>> > > Scavenge 131.0 -> 130.7 MB, 37 ms.
> >>>> > > Scavenge 136.8 -> 134.5 MB, 28 ms.
> >>>> > > Scavenge 140.5 -> 134.5 MB, 16 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > Time (splay): 1954 us.
>
> >>>> > > (I am at trunk's 3339 now)
>
> >>>> > > Are you sure you do not allocate a thing in between? (it could lead
> >>>> > > to fragmentation of address space and eventual OOM).
>
> >>>> > > yours,
> >>>> > > anton.
>
> >>>> > > On Fri, Nov 20, 2009 at 10:29 PM, dracflamloc
>
> >>>> > > <[email protected]> wrote:
>
> >>>> > > > Thanks for the help, btw =)
>
> >>>> > > > I ran your splay test and it crashes as well. here is the trace:
> >>>> > > > Scavenge 34.5 -> 34.2 MB, 43 ms.
> >>>> > > > Mark-sweep 39.2 -> 38.9 MB, 199 ms.
> >>>> > > > Scavenge 45.0 -> 44.7 MB, 41 ms.
> >>>> > > > Mark-sweep 50.7 -> 50.4 MB, 243 ms.
> >>>> > > > Scavenge 56.4 -> 56.1 MB, 41 ms.
> >>>> > > > Scavenge 62.1 -> 61.9 MB, 42 ms.
> >>>> > > > Mark-sweep 67.9 -> 67.6 MB, 316 ms.
> >>>> > > > Scavenge 73.6 -> 73.3 MB, 43 ms.
> >>>> > > > Scavenge 79.3 -> 79.1 MB, 42 ms.
> >>>> > > > Scavenge 85.1 -> 84.8 MB, 42 ms.
> >>>> > > > Mark-sweep 90.8 -> 90.5 MB, 409 ms.
> >>>> > > > Scavenge 96.5 -> 96.2 MB, 44 ms.
> >>>> > > > Scavenge 102.2 -> 102.0 MB, 43 ms.
> >>>> > > > Scavenge 108.0 -> 107.7 MB, 44 ms.
> >>>> > > > Scavenge 113.7 -> 113.4 MB, 46 ms.
> >>>> > > > Mark-sweep 119.4 -> 119.1 MB, 519 ms.
> >>>> > > > Scavenge 125.2 -> 124.9 MB, 44 ms.
> >>>> > > > Scavenge 130.9 -> 130.6 MB, 45 ms.
> >>>> > > > Scavenge 136.6 -> 134.5 MB, 35 ms.
> >>>> > > > Scavenge 140.5 -> 134.5 MB, 18 ms.
> >>>> > > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > > Scavenge 142.5 -> 134.5 MB, 6 ms.
> >>>> > > > Scavenge 142.5 -> 134.5 MB, 7 ms.
> >>>> > > > crash...
>
> >>>> > > > I could post the code, however I'm not just using the vanilla v8
> >>>> > > > engine. I've wrapped pdcurses functions and created my own. So my
> >>>> > > > code
> >>>> > > > won't work in plain v8. However since even the splay test is
> >>>> > > > crashing,
> >>>> > > > we already have a common code we can work from to try and figure
> >>>> > > > this
> >>>> > > > out.
>
> >>>> > > > On Nov 20, 1:58 pm, Erik Corry <[email protected]> wrote:
> >>>> > > >> I attached the file. I am running with the command line
>
> >>>> > > >> ./v8 public-splay.txt --trace-gc
>
> >>>> > > >> I am using the Linux build.
>
> >>>> > > >> 2009/11/20 dracflamloc <[email protected]>:
>
> >>>> > > >> > Perhaps I'm missing something here. Could you post the flags
> >>>> > > >> > you're
> >>>> > > >> > passing in, if any?
> >>>> > > >> > I assume the 140 you're getting is from the gc trace output?
> >>>> > > >> > Are there any special build parameters you're using?
>
> >>>> > > >> > I don't get why its happening, but it is. Is there anything I
> >>>> > > >> > could do
> >>>> > > >> > to investigate this better?
>
> >>>> > > >> If the JS file is something you can share then sending it to us
> >>>> > > >> would
> >>>> > > >> help us find out what is going on.
>
> >>>> > > >> > On Nov 20, 1:12 pm, Erik Corry <[email protected]> wrote:
> >>>> > > >> >> It seems to work for me. I quadrupled the size of the splay
> >>>> > > >> >> benchmark
> >>>> > > >> >> and I can run that without problems. It tops out at over
> >>>> > > >> >> 250Mbytes in
> >>>> > > >> >> the 64 bit version, around 140Mbytes in the 32 bit version
> >>>> > > >> >> (those fat
> >>>> > > >> >> pointers take up space!).
>
> >>>> > > >> >> 2009/11/20 dracflamloc <[email protected]>:
>
> >>>> > > >> >> > trace shows it gets to about 125MB and then can no longer
> >>>> > > >> >> > free memory
> >>>> > > >> >> > so it throws OOM.
> >>>> > > >> >> > I've attempted to use --max-old-space-size but always get
> >>>> > > >> >> > the same
> >>>> > > >> >> > result
>
> >>>> > > >> >> > I have tried this with 3339 and still it caps out at 125 in
> >>>> > > >> >> > the trace.
>
> >>>> > > >> >> > The thing is, I know where the memory is going (Lots of
> >>>> > > >> >> > arrays),
> >>>> > > >> >> > however I don't know why its capping out at such a
>
> ...
>
> read more »
--~--~---------~--~----~------------~-------~--~----~
v8-users mailing list
[email protected]
http://groups.google.com/group/v8-users
-~----------~----~----~----~------~----~------~--~---
