Re: Time for a new 3.0/3.11 release?

2019-07-03 Thread Jay Zhuang
I'd like to raise some attention for the following 2 tickets, they're patch-ready and deployed on all our production clusters: * CASSANDRA-15098: "Endpoints no longer owning tokens are not removed for vnode" For vNode cluster, the replaced node may not be removed from gossiper (and system.peers,

Re: Recommended circleci settings for DTest

2018-09-28 Thread Jay Zhuang
*with_dtest_jobs > +build_and_run_tests: *with_dtest_jobs > docker_image: _image kjellman/cassandra-test:0.4.3 > version: 2 > jobs: > > Ariel > > On Fri, Sep 28, 2018, at 5:47 PM, Jay Zhuang wrote: > > Hi, > > > > Do we have a recommended circleci setu

Recommended circleci settings for DTest

2018-09-28 Thread Jay Zhuang
Hi, Do we have a recommended circleci setup for DTest? For example, what's the minimal container number I need to finish the DTest in a reasonable time. I know the free account (4 containers) is not good enough for the DTest. But if the community member can pay for the cost, what's the

Re: QA signup

2018-09-26 Thread Jay Zhuang
+1 for publishing official snapshot artifacts for 4.0 and even other branches. We're publishing snapshot artifacts to our internal artifactory. One minor bug we found is: currently build.xml won't publish any snapshot artifact: https://issues.apache.org/jira/browse/CASSANDRA-12704 On Thu, Sep

Re: NGCC 2018?

2018-08-31 Thread Jay Zhuang
Are we going to have a dev event next month? Or anything this year? We may also be able to provide space in bay area and help to organize it. (Please let us know, so we could get final approval for that). On Fri, Jul 27, 2018 at 10:05 AM Jonathan Haddad wrote: > My interpretation of Nate's

Re: [Discuss] Accept GoCQL driver donation

2018-08-31 Thread Jay Zhuang
That's great. Could that be in the same repo as Cassandra or a separate repo? On Fri, Aug 31, 2018 at 7:14 AM Nate McCall wrote: > Hi folks, > So I was recently talking with, Chris Bannister the gocql [0] > maintainer, and he expressed an interest in donating the driver to the > ASF. > > We

Re: Tombstone passed GC period causes un-repairable inconsistent data

2018-06-25 Thread Jay Zhuang
ut if you're > not running repairs, you're nearly guaranteed problems with resurrection > after gcgs anyway. > > > > On Thu, Jun 21, 2018 at 11:33 AM, Jay Zhuang > wrote: > > > Yes, I also agree that the user should run (incremental) repair within > GCGS > > to

Re: Tombstone passed GC period causes un-repairable inconsistent data

2018-06-21 Thread Jay Zhuang
s.apache.org/jira/browse/CASSANDRA-13912. RRs are certainly > > useful, but can be very risky if not very very carefully implemented. So > > I'm wondering if we shouldn't disable RRs for everything but unrepaired > > data. I'd btw also be interested to hear any opinions on this in

Tombstone passed GC period causes un-repairable inconsistent data

2018-06-19 Thread Jay Zhuang
Hi, We know that the deleted data may re-appear if repair is not run within gc_grace_seconds. When the tombstone is not propagated to all nodes, the data will re-appear. But it's also causing following 2 issues before the tombstone is compacted away: a. inconsistent query result With consistency

Re: Rocksandra performance test result

2018-06-03 Thread Jay Zhuang
We just do double (triple) writes on the application side. We're shadowing partial of the traffic to a smaller staging cluster for new release test, performance/configuration tuning. On Sat, Jun 2, 2018 at 7:47 PM Nate McCall wrote: > > Thanks for sharing, Jay. > > > > Could you say a bit more

Rocksandra performance test result

2018-06-01 Thread Jay Zhuang
We're shadowing some production traffics to a Rocksandra cluster ( https://github.com/Instagram/cassandra/tree/rocks_3.0), the P99 latency is significantly improved (about 6x for read, 12x for write). Here are the test details:

Re: CommitLogSegmentManager verbose debug log

2018-04-07 Thread Jay Zhuang
itself does not provide useful information (like segment number or sthg like that), so IMHO trace would be a better level for this one I agree that one log per sec may not be seen that verbose ! Thank you On 30 March 2018 at 06:36, Jay Zhuang <jay.zhu...@yahoo.com> wrote: > It's c

Re: CommitLogSegmentManager verbose debug log

2018-03-29 Thread Jay Zhuang
It's changed to trace() in cassandra-3.0 with  CASSANDRA-10241:https://github.com/pauloricardomg/cassandra/commit/3ef1b18fa76dce7cd65b73977fc30e51301f3fed#diff-d07279710c482983e537aed26df80400 In cassandra-3.11 (and trunk), it's changed back to debug() with 

Re: RE: how to fix constantly getting out of memory (3.11)

2018-03-19 Thread Jay Zhuang
Hi, For CASSANDRA-13929, The patch is available for review. Anyone interested in reviewing it? Thanks,Jay On Tuesday, December 12, 2017, 5:02:14 AM PST, Steinmaurer, Thomas wrote: Hi, if you are talking about on-heap troubles, then the following

Re: penn state academic paper - "scalable" bloom filters

2018-02-22 Thread Jay Zhuang
I think there's a similar idea here to dynamically resize the BF: https://issues.apache.org/jira/browse/CASSANDRA-6633, but I don't quite understand the idea there. On Thu, Feb 22, 2018 at 7:45 AM, Carl Mueller wrote: >

Re: CDC usability and future development

2018-02-01 Thread Jay Zhuang
We did a POC to improve CDC feature as an interface ( https://github.com/ngcc/ngcc2017/blob/master/CassandraDataIngestion.pdf), so the user doesn't have to read the commit log directly. We deployed the change to a test cluster and doing more tests for production traffics, will send out the design

Re: Cassandra Dtests: skip upgrade tests

2017-12-08 Thread Jay Zhuang
Here is how cassandra-builds jenkins job do:$ rm -r upgrade_tests/ https://github.com/apache/cassandra-builds/blob/master/build-scripts/cassandra-dtest.sh#L50 On Friday, December 8, 2017, 1:28:34 AM PST, Sergey wrote: Hi! How to completely skip upgrade

Re: Flakey Dtests

2017-11-27 Thread Jay Zhuang
I fixed one CDC uTest, please review:https://issues.apache.org/jira/browse/CASSANDRA-14066 On Friday, November 17, 2017 6:34 AM, Josh McKenzie wrote: > > Do we have any volunteers to fix the broken Materialized Views and CDC > DTests? I'll try to take a look at

Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+

2017-08-28 Thread Jay Zhuang
We're using 3.0.12+ for a few months and haven't seen the issue like that. Do we know what could trigger the problem? Or is 3.0.x really impacted? Thanks, Jay On 8/28/17 6:02 AM, Hannu Kröger wrote: > Hello, > > Current latest Cassandra version (3.11.0, possibly also 3.0.12+) has a race >

Re: CASSANDRA-9472 Reintroduce off heap memtables - patch to 3.0

2017-07-29 Thread Jay Zhuang
eff Jirsa >> >> >>> On Jul 27, 2017, at 10:33 PM, Andrew Whang <andrewgwh...@gmail.com> >> wrote: >>> >>> Jay, >>> >>> We see ~20% write latency improvement on 3.0.13 in a write-heavy >> workload, >>> using

Re: CASSANDRA-9472 Reintroduce off heap memtables - patch to 3.0

2017-07-27 Thread Jay Zhuang
Hi Andrew, Do you see performance gain from reintroducing off-heap memtables for 3.0.x? When we were on 2.2.x we saw big improvements from enabling off-heap memtables. Thanks, Jay On 7/27/17 9:37 PM, Andrew Whang wrote: > I'm wondering if anyone has been able to patch CASSANDRA-9472 to 3.0, >

commitlog_total_space_in_mb tuning

2017-07-05 Thread Jay Zhuang
Hi, commitlog_total_space_in_mb is increased from 1G to 8G in CASSANDRA-7031. Sometimes we saw the number of dropped mutations spikes. Not sure if it's a sign that we should increase the commitlog_total_space_in_mb? For bean:

Re: NGCC Proposal (Was Re: NGCC?)

2017-06-20 Thread Jay Zhuang
Just one day this year? On 6/13/17 12:34 PM, Jonathan Haddad wrote: > Agreed with Jeff & Jason. > > On Tue, Jun 13, 2017 at 11:45 AM Jeff Jirsa wrote: > >> Looks great to me - especially the venue. Date wise, Tuesday (19th) lets >> people fly in on Monday instead of costing

Re: Is concurrent_batchlog_writes option used/implemented?

2017-06-13 Thread Jay Zhuang
Looks like it's a document issue. The option is removed in 3.0.0 for CASSANDRA-9673: https://github.com/apache/cassandra/commit/53a177a9150586e56408f25c959f75110a2997e7 Thanks, Jay On 6/13/17 5:33 AM, Tomas Repik wrote: Hi, while browsing the options for setting up Cassandra at [1] I found

Re: Status on new nodes for builds.apache.org

2017-06-05 Thread Jay Zhuang
circleci is pretty good. Thanks for adding that. Would be better to have DTest too. One problem for circleci is it takes much longer than a typical host (70 minutes vs. 40 minutes on a macbook pro). With more parallel test.runners, local host could run even faster:

Re: Status on new nodes for builds.apache.org

2017-06-05 Thread Jay Zhuang
Is there any plan to give the CI permission to non-committers? It would be great if we could also use that. Thanks, Jay On 6/2/17 10:24 AM, Stefan Podkowinski wrote: Just a quick heads up for everyone interested in the jobs history at builds.apache.org or who wants to run devbranch jobs

Fwd: Potential block issue for 3.0.13: schema version id mismatch while upgrading

2017-05-30 Thread Jay Zhuang
Seems the mail is marked as spam. So try forwarding with another email account. Thanks, Jay -- Forwarded message -- From: Jay Zhuang <jay.zhu...@yahoo.com.invalid> Date: Tue, May 30, 2017 at 2:22 PM Subject: Potential block issue for 3.0.13: schema version id mismatch

Potential block issue for 3.0.13: schema version id mismatch while upgrading

2017-05-30 Thread Jay Zhuang
Hi, While upgrading to 3.0.13 we found that the schema id is changed for the same schema. Which could cause cassandra unable to start and other issues related to UnknownColumnFamilyException. Ticket: CASSANDRA-13559 The problem is because the order of SchemaKeyspace tables is changed. Then

Re: How to read CDC from Cassandra?

2017-02-15 Thread Jay Zhuang
I tried this CASSANDRA-11575 for 3.8. Works great. Thanks, Jay On 2/15/17 3:08 PM, S G wrote: Hi, I have gone through several resources mentioned in http://cassandra.apache.org/doc/latest/operating/cdc.html The only thing mentioned about reading the CDC is that it is fairly straightforward

Re: Have a CDC commitLog process option in Cassandra

2017-02-09 Thread Jay Zhuang
, Jay On 2/9/17 3:57 PM, Dikang Gu wrote: Is it for testing purpose? On Thu, Feb 9, 2017 at 3:54 PM, Jay Zhuang <jay.zhu...@yahoo.com.invalid> wrote: Hi, To process the CDC commitLogs, it requires a separate Daemon process, Carl has a Daemon example here: CASSANDRA-11575. Does it make

Have a CDC commitLog process option in Cassandra

2017-02-09 Thread Jay Zhuang
Hi, To process the CDC commitLogs, it requires a separate Daemon process, Carl has a Daemon example here: CASSANDRA-11575. Does it make sense to integrate it into Cassandra? So the user doesn't have to manage another JVM on the same box. Then provide an ITrigger like interface

Re: Current Branch Merge Path - PLEASE READ!

2016-12-12 Thread Jay Zhuang
. If it’s a minor minor bug, fix it in 3.0 and generate patches for versions newer than that. If it’s a critical bug, go back to 2.1 and see if it exists there as well. On 12/10/16, 6:03 PM, "Jay Zhuang" <jay.zhu...@yahoo.com.INVALID> wrote: I'm new to the community, sorry

Re: Current Branch Merge Path - PLEASE READ!

2016-12-10 Thread Jay Zhuang
I'm new to the community, sorry if it’s obvious question. Are there any general guidance on choosing which branch we should start with? For example, if I find a bug in 3.0, should I try to reproduce it in the lowest version (2.1) and work from there? Thanks, Jay On 12/8/16 10:29 AM, Michael