Re: Java GC pauses, reality check

From:* Kant Kodali <k...@peernova.com> > *Sent:* Monday, November 28, 2016 6:56 AM > > *To:* user@cassandra.apache.org > *Subject:* Re: Java GC pauses, reality check > > Hi Hari, > > I am a little bit confused. > > What you mean 11/11 ? > > "We are using g1GC

Re: Java GC pauses, reality check

Hi @Kant Kodali, 11 /11 , 11 nodes in DC1 and 11 nodes in DC2. From: Kant Kodali <k...@peernova.com> Sent: Monday, November 28, 2016 6:56 AM To: user@cassandra.apache.org Subject: Re: Java GC pauses, reality check Hi Hari, I am a little bit confused. Wh

Re: Java GC pauses, reality check

; the clusters having 10- 100 KB of read data. > > Regards > > Hari > -- > *From:* Kant Kodali <k...@peernova.com> > *Sent:* Saturday, November 26, 2016 8:39:01 PM > *To:* user@cassandra.apache.org > *Subject:* Re: Java GC pauses, reality ch

Re: Java GC pauses, reality check

ters having 10- 100 KB of read data. > > Regards > > Hari > -- > *From:* Kant Kodali <k...@peernova.com> > *Sent:* Saturday, November 26, 2016 8:39:01 PM > *To:* user@cassandra.apache.org > *Subject:* Re: Java GC pauses, reality check > > @

Re: Java GC pauses, reality check

hours and 99% latency is under 5-10 ms for most of the clusters having 10- 100 KB of read data. Regards Hari From: Kant Kodali <k...@peernova.com> Sent: Saturday, November 26, 2016 8:39:01 PM To: user@cassandra.apache.org Subject: Re: Java GC pauses, r

Re: Java GC pauses, reality check

Maybe I was not totally clear. Reference counting is of course done at runtime but the compiler automates where + when to do the counting. Before, the developer had to retain + release objects manually. Since ARC, this is done by the compiler at file level. Nothing is "free" in this world. There

Re: Java GC pauses, reality check

Reference counting happens at run time, not compile time. It's not free either. Every time a reference is added, there's overhead in tracking it. It also doesn't catch cycles. You still need garbage collection to avoid memory leaks. On Sun, Nov 27, 2016 at 12:31 AM Benjamin Roth

Re: Java GC pauses, reality check

I didn't even know there are plans to move to TPC in Cs. Thanks for that update. After all I will follow the development of both Scylla and Cs and am excited about the future of both! Am 27.11.2016 10:02 schrieb "Kant Kodali" : > Yes I am well aware of Scyalldb. It might be

Re: Java GC pauses, reality check

Yes I am well aware of Scyalldb. It might be well written in C++ but the performance gain they are claiming has very little to do with moving from Java to C++. They had major design changes such as moving away from SEDA to TPC and so on. Moreover I would say it still needs to mature. Lot of users

Re: Java GC pauses, reality check

You are of course right. There is no solution and no language that is a perfect match for every situation and every solution and language has it's own pros, cons, pitfalls and drawbacks. Actually that article you posted points at some aspect of ARC, I wasn't aware of, yet. Nevertheless, GC is an

Re: Java GC pauses, reality check

Automatic Reference counting sounds like college level idea that we all have been hearing for since GC is born! There seem to be bunch of cons of ARC as explained here https://www.quora.com/Why-doesnt-Apple-Swift-adopt-the-memory-management-method-of-garbage-collection-like-in-Java Maintaining C

Re: Java GC pauses, reality check

Arc means Automatic Reference counting which is done at compilen time. Eg Objektive c and Swift use this technique. There are absolutely No gc's. Its a completely different memory Management technique. Why i dont like Java on Server side? Because gc is a pain in the ass. I am doing this Business

Re: Java GC pauses, reality check

@Harikrishnan Pillai: How many nodes you guys are running? and what is an approximate read size and an approximate write size? On Fri, Nov 25, 2016 at 7:32 PM, Harikrishnan Pillai < hpil...@walmartlabs.com> wrote: > We are running azul zing in prod with 1 million reads/s and 100 K writes/s >

Re: Java GC pauses, reality check

Good to know about Zing! I will have to take a look. On Sat, Nov 26, 2016 at 8:27 PM, Kant Kodali wrote: > Benjamin Roth: How do you know Arc eliminates GC pauses completely? By > completely I mean no GC pauses whatsoever. > > When you say Java is NOT the First choice for

Re: Java GC pauses, reality check

Benjamin Roth: How do you know Arc eliminates GC pauses completely? By completely I mean no GC pauses whatsoever. When you say Java is NOT the First choice for Server Applications you are generalizing it too much I would say since many of them fall under that category. Either way the statement

Re: Java GC pauses, reality check

On Nov 26, 2016 20:52, "Graham Sanderson" wrote: It was removed in the 3.0.x line, but not in the 3.x line (post 9472) as far as I can tell. It looks to be available in 3.11 and in 3.X branches Thanks, you are correct. I'm confused. On Nov 26, 2016, at 1:17 PM, Oleksandr

Re: Java GC pauses, reality check

It was removed in the 3.0.x line, but not in the 3.x line (post 9472) as far as I can tell. It looks to be available in 3.11 and in 3.X branches > On Nov 26, 2016, at 1:17 PM, Oleksandr Shulgin > wrote: > > On Nov 26, 2016 20:04, "Graham Sanderson"

Re: Java GC pauses, reality check

On Nov 26, 2016 20:04, "Graham Sanderson" wrote: Not AFAIK; https://issues.apache.org/jira/browse/CASSANDRA-9472 is marked as resolved in 3.4, though we are not running it so I can’t say much about it. But I was referring to

Re: Java GC pauses, reality check

Not AFAIK; https://issues.apache.org/jira/browse/CASSANDRA-9472 is marked as resolved in 3.4, though we are not running it so I can’t say much about it. It looks like Zing is no longer tied price wise per core which was a show stopper for

Re: Java GC pauses, reality check

Thanks! But getting back to the original issue: I think the GC itself is not the root cause for such a long pause. I remember having had issues with 1 minute GCs in the beginning. I also tried around with larger and smaller heap sizes and different GCs (G1, CMS), different settings but what

Re: Java GC pauses, reality check

I'm not affiliated with them, I've just been impressed by them. They have done amazing work in performance measurement. They discovered a major flaw in most performance testing ... I've never seen their pricing. But, recently, they made their product available for testing by developers. And the

Re: Java GC pauses, reality check

This sounds amazing but also expensive - I don't see pricing on their page. Are you able and allowed to tell a rough pricing range? Am 26.11.2016 04:33 schrieb "Harikrishnan Pillai" : > We are running azul zing in prod with 1 million reads/s and 100 K writes/s > with

Re: Java GC pauses, reality check

Hi Ahmed, obviously, 20-30 sec. pause is unacceptable. I suppose check the following: - disable swapping completely - check Java version, v8. is desirable (depending on Cassandra version) - use multiprocessor machine (it allows concurrent GC) Best regards, Vladimir Yudovin, Winguzone

Re: Java GC pauses, reality check

On Nov 25, 2016 23:47, "Graham Sanderson" wrote: If you are seeing 25-30 second GC pauses then (unless you are so badly configured) seeing full GC under CMS (though G1 may have similar problems). With CMS eventual fragmentation causing promotion failure is inevitable (unless

Re: Java GC pauses, reality check

We are running azul zing in prod with 1 million reads/s and 100 K writes/s with azul .we never had a major gc above 10 ms . Sent from my iPhone > On Nov 25, 2016, at 3:49 PM, Martin Schröder wrote: > > 2016-11-25 23:38 GMT+01:00 Kant Kodali : >> I would

Re: Java GC pauses, reality check

Zing jvm reduces the pause under 10ms for most use cases. Sent from my iPhone On Nov 25, 2016, at 2:44 PM, Kant Kodali > wrote: +1 Chris Lohfink response I would also restate the following sentence "java GC pauses are pretty much a fact of life" to

Re: Java GC pauses, reality check

2016-11-25 23:38 GMT+01:00 Kant Kodali : > I would also restate the following sentence "java GC pauses are pretty much > a fact of life" to "Any GC based system pauses are pretty much a fact of > life". > > I would be more than happy to see if someone can counter prove. Azul

Re: Java GC pauses, reality check

If you are seeing 25-30 second GC pauses then (unless you are so badly configured) seeing full GC under CMS (though G1 may have similar problems). With CMS eventual fragmentation causing promotion failure is inevitable (unless you cycle your nodes before it happens). Either your heap has way

Re: Java GC pauses, reality check

Lol. The counter proof is to use another memory Model like Arc. Thats why i personally think Java is NOT the First choice for Server Applications. But thats a philosophic discussion. Am 25.11.2016 23:38 schrieb "Kant Kodali" : > +1 Chris Lohfink response > > I would also

Re: Java GC pauses, reality check

+1 Chris Lohfink response I would also restate the following sentence "java GC pauses are pretty much a fact of life" to "Any GC based system pauses are pretty much a fact of life". I would be more than happy to see if someone can counter prove. On Fri, Nov 25, 2016 at 1:41 PM, Chris Lohfink

Re: Java GC pauses, reality check

No tuning will eliminate gcs. 20-30 seconds is horrific and out of the ordinary. Most likely implementing antipatterns and/or poorly configured. Sub 1s is realistic but with some workloads still may require some tuning to maintain. Some workloads are very unfriendly to GCs though (ie heavy