Re: Welcome Dinesh Joshi as Cassandra PMC member

Congrats Dinesh! On Thu, Jun 3, 2021 at 3:59 PM Patrick McFadin wrote: > This is great. Congratulations Dinesh! > > On Thu, Jun 3, 2021 at 11:51 AM Jordan West wrote: > > > Congratulations Dinesh! > > > > Jordan > > > > On Thu, Jun 3, 2021 at 1:40 AM Mick Semb Wever wrote: > > > > > Congrats

Re: Welcome Yifan Cai as Cassandra committer

Congrats Yifan! Dikang On Mon, Dec 21, 2020 at 9:11 AM Benjamin Lerer wrote: > The PMC's members are pleased to announce that Yifan Cai has accepted the > invitation to become committer last Friday. > > Thanks a lot, Yifan, for everything you have done! > > Congratulations and welcome > >

Re: Apache Cassandra meetup @ Instagram HQ

Yeah, we have recorded the meetup, will publish it online in next couple weeks. Thanks Dikang On Fri, Feb 22, 2019 at 3:35 PM Sundaramoorthy, Natarajan < natarajan_sundaramoor...@optum.com> wrote: > Dinesh - Please let us know if they were able to record and post it > online? Thanks > > > >

Re: CASSANDRA-14482

+1 On Fri, Feb 15, 2019 at 10:27 AM Vinay Chella wrote: > We have been using Zstd compressor across different products/services here > and have seen significant improvements, getting this in 4.0 would be a big > win. > > +1 > > Thanks, > Vinay Chella > > > On Fri, Feb 15, 2019 at 10:19 AM Jeff

Re: [DISCUSS] changing default token behavior for 4.0

We are using 8 or 16 tokens internally, with the token allocation algorithm enabled. The range distribution is good for us. Dikang. On Fri, Sep 21, 2018 at 9:30 PM Dinesh Joshi wrote: > Jon, thanks for starting this thread! > > I have created CASSANDRA-14784 to track this. > > Dinesh > > > On

Re: Apache Cassandra Blog is now live

The fast streaming is very cool! We are also interested in contributing to the blog, what's the process? Thanks Dikang. On Tue, Aug 7, 2018 at 7:01 PM Nate McCall wrote: > You can tell how psyched we are about it because we cross posted! > > Seriously though - this is by the community for the

Re: NGCC 2018?

We are interested in bay area C* developers event as well. On Thu, Jul 26, 2018 at 10:42 PM Jeff Jirsa wrote: > Bay area event is interesting to me, in any format. > > > On Thu, Jul 26, 2018 at 9:03 PM, Ben Bromhead wrote: > > > It sounds like there may be an appetite for something, but the

Re: Improve the performance of CAS

at context. > > > > Thanks, > > > > -Jason > > > >> On Wed, May 16, 2018 at 2:57 PM, Ariel Weisberg <ar...@weisberg.ws> > wrote: > >> > >> Hi, > >> > >> I think you are looking at the right low hanging fruit. Cassandra >

Re: Improve the performance of CAS

alitarian paxos or epaxos*? > > That might be helpful for the discussion. > > > > Jeremy > > > > * https://issues.apache.org/jira/browse/CASSANDRA-6246 > > > > > On May 16, 2018, at 3:37 PM, Dikang Gu <dikan...@gmail.com> wrote: > > > > > >

Improve the performance of CAS

Hello C* developers, I'm working on some performance improvements of the lightweight transitions (compare and set), I'd like to hear your thoughts about it. As you know, current CAS requires 4 round trips to finish, which is not efficient, especially in cross DC case. 1) Prepare 2) Quorum read

Re: IN restrictions are not supported on indexed columns

r 23, 2018 at 6:08 PM, Dikang Gu <dikan...@gmail.com> wrote: > > > Hello C* developers: > > > > I have one question, does anyone know why we can not support the IN > > restrictions on indexed columns? Is it just because no one is working it? > > Or are t

IN restrictions are not supported on indexed columns

Hello C* developers: I have one question, does anyone know why we can not support the IN restrictions on indexed columns? Is it just because no one is working it? Or are there any other reasons? Below is an example query: cqlsh:ks1> describe keyspace; CREATE KEYSPACE ks1 WITH replication

Rocksandra blog post

As some of you already know, Instagram Cassandra team is working on the project to use RocksDB as Cassandra's storage engine. Today, we just published a blog post about the work we have done, and more excitingly, we published the benchmark metrics in AWS environment. Check it out here:

Re: Cassandra Wrapup: Feb 2018 Edition

Congratulations Jay! On Sun, Mar 4, 2018 at 6:38 PM, Jeff Jirsa wrote: > I'm late. Mea culpa. I blame February for only having 28 days. > > The following contributors had their first ever commit into the project > (since the last time I made this list, which was late 2017)! >

Re: Timeout unit tests in trunk

ikang, Do you have any details to share about the > failures? > > -Jason > > On Tue, Feb 27, 2018 at 5:16 PM, Dinesh Joshi < > dinesh.jo...@yahoo.com.invalid> wrote: > > > Yes, give it a go. I am not 100% sure if it will help a whole lot but try > > it

Re: Timeout unit tests in trunk

rote: > > Some tests might require additional resources to spin up the required > components. 2 CPU / 4GB might not be sufficient. You may need to bump up > the resources to 8CPU / 16GB. > Dinesh > > On Tuesday, February 27, 2018, 11:24:34 AM PST, Dikang Gu < > dik

Timeout unit tests in trunk

Looks like there are a few flaky/timeout unit tests in trunk, wondering is there anyone looking at them already? testBuildRange - org.apache.cassandra.db.view.ViewBuilderTaskTest testUnloggedPartitionsPerBatch - org.apache.cassandra.metrics.BatchMetricsTest testViewBuilderResume -

Re: [Proposal] Pluggable storage engine design

in the plan. Thanks again for your help! Dikang. On Tue, Nov 28, 2017 at 12:23 PM, Dikang Gu <dikan...@gmail.com> wrote: > I create a more formal design proposal for the pluggable storage engine > project, please take a look and let me know your comments. > > > https://docs.

RocksDB meetup bayarea

Hi Dev, Sorry for the lame promo but Pengchao Wang (@wpc), one of the main contributors to our Cassandra on RocksDB hack will be presenting at the annual RockDB meetup hosted at FB HQ. I just wanted to put this on your radar in case some might be interested.

[Proposal] Pluggable storage engine design

I create a more formal design proposal for the pluggable storage engine project, please take a look and let me know your comments. https://docs.google.com/document/d/1suZlvhzgB6NIyBNpM9nxoHxz_Ri7qAm-UEO8v8AIFsc At this moment, I'm focus on the purpose, scope, guideline and high level plan for

Pluggable storage engine discussion

Hi, We are having discussions about the pluggable storage engine plan on the jira: https://issues.apache.org/jira/browse/CASSANDRA-13475. We are trying to figure out a plan for the pluggable storage engine effort. Right now, the discussion is mainly happening between couple C* committers, like

Re: Cassandra pluggable storage engine (update)

echnical storage engine choice rather than functional > splitting > > Regards > > On Wed, Oct 4, 2017 at 10:47 PM, Dikang Gu <dikan...@gmail.com> wrote: > > > Hi Blake, > > > > Great questions! > > > > 1. Yeah, we implement the encoding algorit

Re: Cassandra pluggable storage engine (update)

y categories (ie > repaired and unrepaired data living in the same token range). Is support > for incremental repair planned for v1? > > Thanks, > > Blake > > > On October 4, 2017 at 1:28:01 PM, Dikang Gu (dikan...@gmail.com) wrote: > > Hello C* developers: &

Cassandra pluggable storage engine (update)

Hello C* developers: In my previous email ( https://www.mail-archive.com/dev@cassandra.apache.org/msg11024.html), I presented that Instagram was kicking off a project to make C*'s storage engine to be pluggable, as other modern databases, like mysql, mongoDB etc, so that users will be able to

Commitlog without header

Hello, In our production cluster, we had multiple times that after a *unclean* shutdown, cassandra sever can not start due to commit log exceptions: 2017-09-17_06:06:32.49830 ERROR 06:06:32 [main]: Exiting due to error while processing commit log during initialization. 2017-09-17_06:06:32.49831

Re: Definition of QUORUM consistency level

https://issues.apache.org/jira/browse/CASSANDRA-13645 On Wed, Jun 28, 2017 at 4:59 PM, Dikang Gu <dikan...@gmail.com> wrote: > We implement the patch internally, and deploy to our production clusters, > we see 2X drop of the P99 quorum read latency, because we can reduce one > un

Re: Definition of QUORUM consistency level

e never liked TWO/THREE. This is >> > somewhat relevant: https://issues.apache.org/jira/browse/CASSANDRA-2338 >> > >> > I don't think going to CL.FOUR, etc, is a good long-term solution, but >> I'm >> > also not sure what is. >> > >> > >> >

Re: Definition of QUORUM consistency level

ase. > > Justin > > On Fri, 9 Jun 2017 at 12:29 Dikang Gu <dikan...@gmail.com> wrote: > >> Hello there, >> >> We have some use cases are doing consistent read/write requests, and we >> have 4 replicas in that cluster, according to our setup. >

Definition of QUORUM consistency level

Hello there, We have some use cases are doing consistent read/write requests, and we have 4 replicas in that cluster, according to our setup. What's interesting to me is that, for both read and write quorum requests, they are blocked for 4/2+1 = 3 replicas, so we are accessing 3 (for write) + 3

Re: Cassandra on RocksDB experiment result

t particular storage engine alone? > > On Wed, Apr 26, 2017 at 5:07 AM, Dikang Gu <dikan...@gmail.com> wrote: > > > I created several tickets to start the discussion, please free feel to > > comment on the JIRAs. I'm also open for suggestions about other efficient > >

Re: Cassandra on RocksDB experiment result

://issues.apache.org/jira/browse/CASSANDRA-13476 Thanks Dikang. On Mon, Apr 24, 2017 at 9:53 PM, Dikang Gu <dikan...@gmail.com> wrote: > Thanks everyone for the feedback and suggestions! They are all very > helpful. I'm looking forward to having more discussions about the > implementation details.

Re: Cassandra on RocksDB experiment result

Thanks everyone for the feedback and suggestions! They are all very helpful. I'm looking forward to having more discussions about the implementation details. As the next step, we will be focus on three areas: 1. Pluggable storage engine interface. 2. Wide column support on RocksDB. 3. Streaming

Re: Weekly Cassandra Status

Jeff, thanks for the summary! I will take a look at the token jira, https://issues.apache.org/ jira/browse/CASSANDRA-13348, since I was working on that recently. --Dikang. On Sun, Mar 26, 2017 at 3:35 PM, Jeff Jirsa wrote: > Email stuff: > - We've moved github pull requests

Re: Hopefully Weekly Apache Cassandra JIRA Topics of Interest

This is awesome, Jeff! On Sun, Mar 12, 2017 at 2:52 PM, DuyHai Doan wrote: > Static columns can already be indexed by custom 2nd index impl, for example > SASI : https://issues.apache.org/jira/browse/CASSANDRA-11183 > > > On Sun, Mar 12, 2017 at 10:40 PM, Jeff Jirsa

Re: Have a CDC commitLog process option in Cassandra

Is it for testing purpose? On Thu, Feb 9, 2017 at 3:54 PM, Jay Zhuang wrote: > Hi, > > To process the CDC commitLogs, it requires a separate Daemon process, Carl > has a Daemon example here: CASSANDRA-11575. > > Does it make sense to integrate it into Cassandra? So

Re: Dropped messages on random nodes.

ropping based on time, but > despite the eager dropping you are still facing overload. > > That local, non-gc pause is also troubling. (I assume non-gc since there > wasn't anything logged by the GC inspector.) > > On Mon, Jan 23, 2017 at 12:36 AM, Dikang Gu <dikan...@gmail.com> wr

Re: Dropped messages on random nodes.

Btw, the C* version is 2.2.5, with several backported patches. On Sun, Jan 22, 2017 at 10:36 PM, Dikang Gu <dikan...@gmail.com> wrote: > Hello there, > > We have a 100 nodes ish cluster, I find that there are dropped messages on > random nodes in the cluster, which caused er

Dropped messages on random nodes.

Hello there, We have a 100 nodes ish cluster, I find that there are dropped messages on random nodes in the cluster, which caused error spikes and P99 latency spikes as well. I tried to figure out the cause. I do not see any obvious bottleneck in the cluster, the C* nodes still have plenty of

Re: Wrapping up tick-tock

+1 to 6 months *major* release. I think we still need *minor* release containing bug fixes (or small features maybe?), which I think would make sense to release more frequently, like monthly. So that we won't need to wait for 6 months for bug fixes, or have to maintain a lot of patches

Re: Slow performance after upgrading from 2.0.9 to 2.1.11

-- so i'm not sure how "hacky" it is if it > works.. > > best, > kjellman > > > On Nov 8, 2016, at 11:31 AM, Dikang Gu <dikan...@gmail.com<mailto:dik > an...@gmail.com>> wrote: > > This is very expensive: > > "MessagingService-

Re: Slow performance after upgrading from 2.0.9 to 2.1.11

433? > This is just a wild guess as it's in a related codepath, but maybe worth > trying out the patch available to see if it helps anything... > > 2016-10-28 15:03 GMT-02:00 Dikang Gu <dikan...@gmail.com>: > >> We are seeing huge cpu regression when upgrading one of our 2.0.16

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

My 2 cents. I'm wondering is it a good idea to have some high level goals for the major release? For example, the goals could be something like: 1. Improve the scalability/reliability/performance by X%. 2. Add Y new features (feature A, B, C, D...). 3. Fix Z known issues (issue A, B, C, D...). I

Batch read requests to one physical host?

Hi there, We have couple use cases that are doing fanout read for their data, means one single read request from client contains multiple keys which live on different physical hosts. (I know it's not recommended way to access C*). Right now, on the coordinator, it will issue separate read

Hints handoff is memory intensive

In our 2.1 cluster, I find that hints handoff is using a lot of memory on our proxy nodes, when delivering hints to a data node that was dead for 3+ hours (our hints window is 3 hours). It makes the young gen GC time as long as 2 secs. I'm using 64G max heap size, and 4G young gen size. I'm

Re: Can we change the number of vnodes on existing nodes?

, Brandon Williams <dri...@gmail.com> wrote: > I am curious exactly what gossip issues you are encountering. > > On Fri, Sep 9, 2016 at 7:45 PM, Dikang Gu <dikan...@gmail.com> wrote: > > > Hi, > > > > We have some big cluster (500+ nodes), they have 256 vno

Can we change the number of vnodes on existing nodes?

Hi, We have some big cluster (500+ nodes), they have 256 vnodes on physical host, which is causing a lot of problems to us, especially make the gossip to be in-efficient. There seems no way to change the number of vnodes on existing nodes, is there any reason that we can not support it? It

Re: Counter values become under-counted when running repair.

4 March 2016 at 23:20:55, Dikang Gu (dikan...@gmail.com) wrote: > > @Aleksey, we are writing to cluster with CL = 2, and reading with CL = 1. > And overall we have 6 copies across 3 different regions. Do you have > comments about our setup? > > During the repair, the counter value

Re: Counter values become under-counted when running repair.

> > -- > AY > > On 24 March 2016 at 06:17:27, Dikang Gu (dikan...@gmail.com) wrote: > > Hello there, > > We are experimenting Counters in Cassandra 2.2.5. Our setup is that we > have > 6 nodes, across three different regions, and in each region, the > replication fact

Counter values become under-counted when running repair.

Hello there, We are experimenting Counters in Cassandra 2.2.5. Our setup is that we have 6 nodes, across three different regions, and in each region, the replication factor is 2. Basically, each nodes holds a full copy of the data. When are doing 30k/s counter increment/decrement per node, and

Re: How to measure the write amplification of C*?

docs/jesd219a >> >> >> Matt Kennedy >> >> Sr. Product Manager, DSE Core >> >> matt.kenn...@datastax.com | Public Calendar <http://goo.gl/4Ui04Z> >> >> *DataStax Enterprise - the database for cloud applications.* >> >> On Th

Re: Compaction Filter in Cassandra

Fyi, this is the jira, https://issues.apache.org/jira/browse/CASSANDRA-11348 . We can move the discussion to the jira if want. On Thu, Mar 17, 2016 at 11:46 AM, Dikang Gu <dikan...@gmail.com> wrote: > Hi Eric, > > Thanks for sharing the information! > > We al

Re: Compaction Filter in Cassandra

ng test it out, I'd love to hear from you. > > On Sat, Mar 12, 2016 at 5:12 AM Marcus Eriksson <krum...@gmail.com> wrote: > >> We don't have anything like that, do you have a specific use case in mind? >> >> Could you create a JIRA ticket and we can discuss th

Compaction Filter in Cassandra

Hello there, RocksDB has the feature called "Compaction Filter" to allow application to modify/delete a key-value during the background compaction. https://github.com/facebook/rocksdb/blob/v4.1/include/rocksdb/options.h#L201-L226 I'm wondering is there a plan/value to add this into C* as well?