Re: Using CDC Feature to Stream C* to Kafka (Design Proposal)

Re Rahul: "Although DSE advanced replication does one way, those are use cases with limited value to me because ultimately it’s still a master slave design." Completely agree. I'm not familiar with Calvin protocol, but that sounds interesting (reading time...). On Tue, Sep 11, 2018 at 8:38 PM Joy

Re: Using CDC Feature to Stream C* to Kafka (Design Proposal)

Thank you all for the feedback so far. The immediate use case for us is setting up a real-time streaming data pipeline from C* to our Data Warehouse (BigQuery), where other teams can access the data for reporting/analytics/ad-hoc query. We already do this with MySQL

Re: Scrub a single SSTable only?

Doing this can resurrect deleted data and violate consistency - if that’s a problem for you, it may be easier to treat the whole host as failed, run repairs and replace it. -- Jeff Jirsa > On Sep 11, 2018, at 2:41 PM, Rahul Singh wrote: > > What’s the RF for that data ? If you can manage do

Re: Scrub a single SSTable only?

What’s the RF for that data ? If you can manage downtime one node I’d recommend just bringing it down, and then repairing after you delete the bad file and bring it back up. Rahul Singh Chief Executive Officer m 202.905.2818 Anant Corporation 1010 Wisconsin Ave NW, Suite 250 Washington, D.C. 20

Re: Fresh SSTable files (due to repair?) in a static table (was Re: Drop TTLd rows: upgradesstables -a or scrub?)

On Tue, 11 Sep 2018, 19:26 Jeff Jirsa, wrote: > Repair or read-repair > Jeff, Could you be more specific please? Why any data would be streamed in if there is no (as far as I can see) possibilities for the nodes to have inconsistency? -- Alex On Tue, Sep 11, 2018 at 12:58 AM Oleksandr Shulgi

Re: impact/incompatibility of patch backport on Cassandra 3.11.2

https://issues.apache.org/jira/browse/CASSANDRA-14672 is almost certainly due to pre-existing corruption . That the user is seeing 14672 is due to extra guards added in 3.11.3, but 14672 isn't likely going to hit you unless you're subject to https://issues.apache.org/jira/browse/CASSANDRA-14515 , w

Re: Fresh SSTable files (due to repair?) in a static table (was Re: Drop TTLd rows: upgradesstables -a or scrub?)

Repair or read-repair On Tue, Sep 11, 2018 at 12:58 AM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > On Tue, Sep 11, 2018 at 9:47 AM Oleksandr Shulgin < > oleksandr.shul...@zalando.de> wrote: > >> On Tue, Sep 11, 2018 at 9:31 AM Steinmaurer, Thomas < >> thomas.steinmau...@dynatrace.

Re: High IO and poor read performance on 3.11.2 cassandra cluster

A few reasons I can think of offhand why your test setup might not see problems from large readahead: Your sstables are <4MB or your reads are typically <4MB from the end of the file Your queries tend to use the 4MB of data anyways Your dataset is small enough that most of it fits in the VM cache,

Speakers needed for Apache DC Roadshow

We need your help to make the Apache Washington DC Roadshow on Dec 4th a success. What do we need most? Speakers! We're bringing a unique DC flavor to this event by mixing Open Source Software with talks about Apache projects as well as OSS CyberSecurity, OSS in Government and and OSS Career

Re: Using CDC Feature to Stream C* to Kafka (Design Proposal)

You know what they say: Go big or go home. Right now candidates are Cassandra itself but embedded or on the side not on the actual data clusters, zookeeper (yuck) , Kafka (which needs zookeeper, yuck) , S3 (outside service dependency, so no go. ) Jeff, Those are great patterns. ESP. Second one.

Re: impact/incompatibility of patch backport on Cassandra 3.11.2

Any opinion please ? Le jeu. 6 sept. 2018 à 22:18, Ahmed Eljami a écrit : > Hi, > > We are testing Cassandra 3.11.2 and we sawed that it contains a critcal > bug wich was fixed in 3.11.3 ( > https://issues.apache.org/jira/browse/CASSANDRA-13929). > > After about 1 months of testing, we haven't

Re: Drop TTLd rows: upgradesstables -a or scrub?

On Tue, Sep 11, 2018 at 10:04 AM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > > Yet another surprising aspect of using `nodetool compact` is that it > triggers major compaction on *all* nodes in the cluster at the same time. > I don't see where this is documented and this was contrar

Re: Default Single DataCenter -> Multi DataCenter

It’s self respond. Step3 is wrong. Even if it was a SimpleSnitch, changing the dc information will not start CassandraDaemon with the error log. ERROR [main] 2018-09-11 18:36:30,272 CassandraDaemon.java:708 - Cannot start node if snitch's data center (pg1) differs from previous data center (d

Re: Drop TTLd rows: upgradesstables -a or scrub?

On Tue, Sep 11, 2018 at 11:07 AM Steinmaurer, Thomas < thomas.steinmau...@dynatrace.com> wrote: > > a single (largish) SSTable or any other SSTable for a table, which does > not get any writes (with e.g. deletes) anymore, will most likely not be > part of an automatic minor compaction anymore, thu

RE: Drop TTLd rows: upgradesstables -a or scrub?

Alex, a single (largish) SSTable or any other SSTable for a table, which does not get any writes (with e.g. deletes) anymore, will most likely not be part of an automatic minor compaction anymore, thus may stay forever on disk, if I don’t miss anything crucial here. Might be different though, i

Re: Drop TTLd rows: upgradesstables -a or scrub?

On Tue, Sep 11, 2018 at 9:47 AM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > On Tue, Sep 11, 2018 at 9:31 AM Steinmaurer, Thomas < > thomas.steinmau...@dynatrace.com> wrote: > >> As far as I remember, in newer Cassandra versions, with STCS, nodetool >> compact offers a ‘-s’ command-l

Fresh SSTable files (due to repair?) in a static table (was Re: Drop TTLd rows: upgradesstables -a or scrub?)

On Tue, Sep 11, 2018 at 9:47 AM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > On Tue, Sep 11, 2018 at 9:31 AM Steinmaurer, Thomas < > thomas.steinmau...@dynatrace.com> wrote: > >> As far as I remember, in newer Cassandra versions, with STCS, nodetool >> compact offers a ‘-s’ command-l

Re: Drop TTLd rows: upgradesstables -a or scrub?

On Tue, Sep 11, 2018 at 9:31 AM Steinmaurer, Thomas < thomas.steinmau...@dynatrace.com> wrote: > As far as I remember, in newer Cassandra versions, with STCS, nodetool > compact offers a ‘-s’ command-line option to split the output into files > with 50%, 25% … in size, thus in this case, not a sin

RE: Drop TTLd rows: upgradesstables -a or scrub?

As far as I remember, in newer Cassandra versions, with STCS, nodetool compact offers a ‘-s’ command-line option to split the output into files with 50%, 25% … in size, thus in this case, not a single largish SSTable anymore. By default, without -s, it is a single SSTable though. Thomas From:

RE: Drop TTLd rows: upgradesstables -a or scrub?

From: Jeff Jirsa Sent: Montag, 10. September 2018 19:40 To: cassandra Subject: Re: Drop TTLd rows: upgradesstables -a or scrub? I think it's important to describe exactly what's going on for people who just read the list but who don't have context. This blog does a really good job: http://the

Re: Drop TTLd rows: upgradesstables -a or scrub?

On Mon, Sep 10, 2018 at 10:03 PM Jeff Jirsa wrote: > How much free space do you have, and how big is the table? > So there are 2 tables, one is around 120GB and the other is around 250GB on every node. On the node with the most free disk space we still have around 500GB available and on the nod