[v8-users] Re: GC cost of adding to an object with 50,000+ keys
I switched to a Map which looks like it may have solved the issue. See this: https://jsperf.com/es6-map-vs-object-properties/ And, running Node.js --perf with the code from that jsperf yields the following (on Node v8.9.1 which, according to process.versions, is V8 6.1.534.47): *Map (15.7% in GC)* [Summary]: ticks total nonlib name 903 68.1% 68.4% JavaScript 395 29.8% 29.9% C++ 208 15.7% 15.7% GC 50.4% Shared libraries 231.7% Unaccounted [C++]: ticks total nonlib name 916.9%6.9% t v8::internal::Scavenger::ScavengeObject(v8:: internal::HeapObject**, v8::internal::HeapObject*) 382.9%2.9% t v8::internal::OrderedHashTable::Rehash(v8::internal::Handle, int) 362.7%2.7% T v8::internal::Runtime_GenerateRandomNumbers(int, v8::internal::Object**, v8::internal::Isolate*) 362.7%2.7% T v8::internal::IncrementalMarking::Step(unsigned long, v8::internal::IncrementalMarking::CompletionAction, v8::internal:: IncrementalMarking::ForceCompletionAction, v8::internal::StepOrigin) 302.3%2.3% T v8::internal::Heap::Scavenge() 251.9%1.9% T v8::internal::Heap::AllocateFixedArrayWithFiller( int, v8::internal::PretenureFlag, v8::internal::Object*) 241.8%1.8% t node::(anonymous namespace)::ContextifyScript:: New(v8::FunctionCallbackInfo const&) 141.1%1.1% t v8::internal::(anonymous namespace):: GetSimpleHash(v8::internal::Object*) *Object (26.8% in GC)* [Summary]: ticks total nonlib name 698 32.2% 32.2% JavaScript 1417 65.3% 65.4% C++ 581 26.8% 26.8% GC 20.1% Shared libraries 532.4% Unaccounted [C++]: ticks total nonlib name 269 12.4% 12.4% T v8::internal::IncrementalMarking::Step(unsigned long, v8::internal::IncrementalMarking::CompletionAction, v8::internal:: IncrementalMarking::ForceCompletionAction, v8::internal::StepOrigin) 1486.8%6.8% t v8::internal::HashTable::Rehash(v8::internal:: NameDictionary*) 1225.6%5.6% T v8::internal::Heap::Scavenge() 1074.9%4.9% t v8::internal::Scavenger::ScavengeObject(v8:: internal::HeapObject**, v8::internal::HeapObject*) 1034.7%4.8% T v8::internal::HashTable::Rehash(v8::internal::StringTable*) 914.2%4.2% T v8::internal::StringTable::LookupKey(v8::internal ::Isolate*, v8::internal::StringTableKey*) 823.8%3.8% t v8::internal::LookupIterator::State v8::internal ::LookupIterator::LookupInRegularHolder(v8::internal::Map*, v8:: internal::JSReceiver*) 482.2%2.2% T v8::internal::Heap::AllocateFixedArrayWithFiller( int, v8::internal::PretenureFlag, v8::internal::Object*) 411.9%1.9% T v8::internal::String::SlowEquals(v8::internal:: String*) If I plot out the actual duration of the add/update operation to either a Map or Object, they're both almost equivalent as N increases, but the memory usage and GC activity *around* that add/update for the Object is much higher. In fact, depending on the length of the random string being created, the Object often maxes out the heap and crashes Node.js. (I experimented with increasing --max_old_space_size up to 1) There's also a point (random strings of length 5, with N of 8,000,000) where even if your heap is large enough to hold all the strings, the adding/appending to an Object just seizes up (and Map still works). On Thursday, November 23, 2017 at 12:59:06 AM UTC-8, Camillo Bruni wrote: > > Hi Jonathan, > > this seems suspicious. Would it be possible for you to write a standalone > repro? > Without further information it is hard to asses what exactly might be > going wrong here. > > Cheers, > Camillo > > On Wednesday, November 22, 2017 at 1:29:50 AM UTC+1, Jonathan Otto wrote: >> >> I have a plain JavaScript object that looks like: >> >> { >> sha256a: { id, fieldA, fieldB, fieldC }, >> sha256b: { id, fieldA, fieldB, fieldC }, >> ... >> } >> >> and it seems like when I get over ~50,000 sha's then I start to have high >> CPU usage (~3%) when adding a key or changing a value for an existing key - >> note I am adding/changing keys once or twice per second. Naively I'm >> wondering if the GC is traversing the entire object on each addition or >> change and whether there's a cliff where that falls off. >> >> Is there a better way to manage a local store in JavaScript? >> > -- -- v8-users mailing list v8-users@googlegroups.com http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [v8-users] How to build a fixed version of V8 and its dependencies.
The depot_tools system makes this easy, actually: git checkout gclient sync ninja -C out/ d8 (If you go back far enough before ninja was supported, replace the last command with the equivalent make/GYP based build.) On Mon, Nov 27, 2017 at 9:23 AM Edd Barrett wrote: > Hi Mathias, > > Thanks for your swift response. > > On Monday, November 27, 2017 at 1:40:05 PM UTC, Mathias Bynens wrote: >> >> This is not a direct answer to your question, but we already host >> prebuilt `d8` binaries for Linux on 32-bit and 64-bit architectures. We >> plan on doing the same for other operating systems: >> https://bugs.chromium.org/p/v8/issues/detail?id=5918 By using a >> precompiled binary, you could bypass compiling from source together. >> > > > That's good to know, but for our purposes we really do need to build from > source: > > * We have patches to grant access to the monotonic clock and performance > counters. > * The experiment should be fully repeatable from source. > > I think it's something the V8 team should consider for the future. > > Cheers > > -- > -- > v8-users mailing list > v8-users@googlegroups.com > http://groups.google.com/group/v8-users > --- > You received this message because you are subscribed to the Google Groups > "v8-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to v8-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- -- v8-users mailing list v8-users@googlegroups.com http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [v8-users] How to build a fixed version of V8 and its dependencies.
Hi Mathias, Thanks for your swift response. On Monday, November 27, 2017 at 1:40:05 PM UTC, Mathias Bynens wrote: > > This is not a direct answer to your question, but we already host prebuilt > `d8` binaries for Linux on 32-bit and 64-bit architectures. We plan on > doing the same for other operating systems: > https://bugs.chromium.org/p/v8/issues/detail?id=5918 By using a > precompiled binary, you could bypass compiling from source together. > That's good to know, but for our purposes we really do need to build from source: * We have patches to grant access to the monotonic clock and performance counters. * The experiment should be fully repeatable from source. I think it's something the V8 team should consider for the future. Cheers -- -- v8-users mailing list v8-users@googlegroups.com http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [v8-users] How to build a fixed version of V8 and its dependencies.
This is not a direct answer to your question, but we already host prebuilt `d8` binaries for Linux on 32-bit and 64-bit architectures. We plan on doing the same for other operating systems: https://bugs.chromium.org/p/v8/issues/detail?id=5918 By using a precompiled binary, you could bypass compiling from source together. On Mon, Nov 27, 2017 at 12:19 PM, Edd Barrett wrote: > Hi, > > I raised this as a bug, but was asked to re-post here. > > For a benchmarking experiment I need to build a frozen version of V8. For > most VMs this means just checking out a specific git version of the code > and building it. For V8 it is complicated by the depot_tools system. I've > yet to find a reliable way to do this. > > In the past, I've gotten by with cloning a specific version of depot > tools, fetching V8, then walking the dependencies resetting the git > versions of everything to a fixed date (this hack is needed because `fetch` > gets moving targets). When this stopped working (it was a hack anyway), I > resorted to rolling my own tarballs, which is not really ideal either. > > Is there an official way to build an frozen version of V8? How do you guys > build (e.g.) V8-5.8.283.32 and using the dependencies of the time of that > release? > > Without the ability to do this, it's difficult to conduct repeatable > benchmarking experiments on V8. > > Thanks > > -- > -- > v8-users mailing list > v8-users@googlegroups.com > http://groups.google.com/group/v8-users > --- > You received this message because you are subscribed to the Google Groups > "v8-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to v8-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Mathias Bynens | V8 | Google Germany GmbH | Erika-Mann-Str. 33, 80636 München Geschäftsführer: Paul Manicle, Halimah DeLaine Prado Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Diese E-Mail ist vertraulich. Falls sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde. This e-mail is confidential. If you received this communication by mistake, please don’t forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person. -- -- v8-users mailing list v8-users@googlegroups.com http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[v8-users] How to build a fixed version of V8 and its dependencies.
Hi, I raised this as a bug, but was asked to re-post here. For a benchmarking experiment I need to build a frozen version of V8. For most VMs this means just checking out a specific git version of the code and building it. For V8 it is complicated by the depot_tools system. I've yet to find a reliable way to do this. In the past, I've gotten by with cloning a specific version of depot tools, fetching V8, then walking the dependencies resetting the git versions of everything to a fixed date (this hack is needed because `fetch` gets moving targets). When this stopped working (it was a hack anyway), I resorted to rolling my own tarballs, which is not really ideal either. Is there an official way to build an frozen version of V8? How do you guys build (e.g.) V8-5.8.283.32 and using the dependencies of the time of that release? Without the ability to do this, it's difficult to conduct repeatable benchmarking experiments on V8. Thanks -- -- v8-users mailing list v8-users@googlegroups.com http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.