Re: Have a CDC commitLog process option in Cassandra

2017-02-14 Thread Josh McKenzie
The furthest we got to a reference daemon was Carl's work on 11575. Hasn't been prioritized since then since the interface refactors were the lion's share of the necessary complexity to allow people to build off the feature. As for any follow-up work, https://issues.apache.org/jira/browse/CASSANDR

Re: Have a CDC commitLog process option in Cassandra

2017-02-13 Thread Jay Zhuang
Thanks Josh for your response. I agree that it may cause the JVM unstable if that feature is enabled. (we may still want to try that out internally and do some perf/stress tests, will share more information if we have the results. If you have any suggestions on testing, please let me know.) A

Re: Have a CDC commitLog process option in Cassandra

2017-02-10 Thread Josh McKenzie
The primary reason I avoided integrating a daemon into the Cassandra process was the increase in heap pressure and further muddying of the profile of heap usage. We've already seen that mixing read/write, compaction, streaming, and repair in the same JVM causes a nasty mix of allocation patterns th

Re: Have a CDC commitLog process option in Cassandra

2017-02-09 Thread Jay Zhuang
No. It's going to have Cassandra to manage the CDC logs, instead of having another daemon process to handle that. Here is CDC design JIRA: CASSANDRA-8844. The pain point is to develop and manage the daemon. If they're integrated, it's going to be easier to manage and monitor that. Thanks, Ja

Re: Have a CDC commitLog process option in Cassandra

2017-02-09 Thread Dikang Gu
Is it for testing purpose? On Thu, Feb 9, 2017 at 3:54 PM, Jay Zhuang wrote: > Hi, > > To process the CDC commitLogs, it requires a separate Daemon process, Carl > has a Daemon example here: CASSANDRA-11575. > > Does it make sense to integrate it into Cassandra? So the user doesn't > have to man

Have a CDC commitLog process option in Cassandra

2017-02-09 Thread Jay Zhuang
Hi, To process the CDC commitLogs, it requires a separate Daemon process, Carl has a Daemon example here: CASSANDRA-11575. Does it make sense to integrate it into Cassandra? So the user doesn't have to manage another JVM on the same box. Then provide an ITrigger like interface (https://gith