On Monday, June 5, 2017 at 4:58:40 PM UTC+3, Alexandr Nikitin wrote: > > Gil, Kirk, Thank you for the great and thorough explanation! It is an > awesome read! Almost everything fell into place. > > Indeed, Gil is right about the application pattern change. It's a > high-load API. There's a background thread that changes an internal state. > The state is ~3.5G and requires some similar amount of memory to build > itself. It happens in one background thread with lower priority. The change > doesn't happen often, once per hour or so. > > The G1 is new for me and I couldn't intuitively understand why it reacts > to the allocation pattern change like that. There's the main allocation/ > collection pattern that lasts for hours. And it needs just one background > low priority thread (which allocates less memory than worker threads) to > change it :) That region number change leads to more copying and promotions > and longer pauses as a consequence (it changes max age threshold to 1 and > happens more often) > > And the actual bottom line is very simple: You need a concurrent newgen to >> handle this workload, with these phase changes, without any of those >> "hiccups". We can discuss that in other posts if you want ;-). >> > > Do you mean Azul Zing by that? >
Yup. Zing's newgen would happily copy and promote those 3.5GB concurrently, and wouldn't break a sweat doing it. It will take a couple of seconds, but not stop-the-world seconds. Since all the collection work is done in the background and is concurrent, you won't feel it. And since Zing doesn't need to worry much a "bigger pause" happening if it lets newgen grow, it doesn't have to hurry up and collect too early, so collections would still ~110GB of allocation apart given your heap size and apparent live set. At your allocation rate, that looks like newgens can stay several hundreds of seconds apart, keeping them very efficient. Newgen will end up copying those 3.5GB once (to promote), or maybe twice (if the collection happened less than 2 seconds after the new 3.5GB of state was allocated). But not several times. Zing doesn't need to keep things in newgen for multiple cycles to age them. We use time instead. In fact, you could probably run this workload (with this behavior) on Zing with about about half or 2/3 the heap you are using for HotSpot, and still not break a sweat. We'll probably also end up running an oldgen every 10 minutes or so as well. Just for fun. Not because of any pressure, but simply because the collector gets bored after a while of not doing oldgens when other stuff gets to have fun. Since oldgen is concurrent too (including compaction), it's ok to do it lazily to make sure we don't delay some strange work to later. We like to make sure that unless you are idle, you've seen the full behavior of the system if you've been running for an hour or so. Our rule is: if an newgen happens, and no oldgen has run in the past ten minutes, do one. -- You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
