Re: cqlsh -e output - How to change the default delimiter '|' in the output

[Jon Haddad]

Using COPY .. TO you can export using the DELIMITER option, does that help?


> On Aug 15, 2017, at 9:01 PM, Harikrishnan A  wrote:
> 
> Thank you all
> 
> Regards,
> Hari
> 
> 
> On Tuesday, August 15, 2017 12:55 AM, Erick Ramirez  
> wrote:
> 
> 
> +1 to Jim and Tobin. cqlsh wasn't designed for what you're trying to achieve. 
> Cheers!
> 
> On Tue, Aug 15, 2017 at 1:34 AM, Tobin Landricombe  > wrote:
> Can't change the delimiter (I'm on cqlsh 5.0.1). Best I can offer is 
> https://docs.datastax.com/en/ cql/3.3/cql/cql_reference/ cqlshExpand.html 
> 
> 
> > On 14 Aug 2017, at 16:17, Jim Witschey  > > wrote:
> >
> > Not knowing the problem you're trying to solve, I'm going to guess
> > cqlsh is a bad tool for this job. If you want to put the results of
> > CQL queries into a shell pipeline, a small custom script using a
> > driver is probably a better fit, and should be possible to write
> > without much effort.
> >
> > -- -- -
> > To unsubscribe, e-mail: user-unsubscribe@cassandra. apache.org 
> > 
> > For additional commands, e-mail: user-h...@cassandra.apache.org 
> > 
> >
> 
> 
> -- -- -
> To unsubscribe, e-mail: user-unsubscribe@cassandra. apache.org 
> 
> For additional commands, e-mail: user-h...@cassandra.apache.org 
> 
> 
> 
> 
> 

Re: cqlsh -e output - How to change the default delimiter '|' in the output

[Harikrishnan A]

Thank you all
Regards,Hari 

On Tuesday, August 15, 2017 12:55 AM, Erick Ramirez  
wrote:
 

 +1 to Jim and Tobin. cqlsh wasn't designed for what you're trying to achieve. 
Cheers!
On Tue, Aug 15, 2017 at 1:34 AM, Tobin Landricombe  wrote:

Can't change the delimiter (I'm on cqlsh 5.0.1). Best I can offer is 
https://docs.datastax.com/en/ cql/3.3/cql/cql_reference/ cqlshExpand.html

> On 14 Aug 2017, at 16:17, Jim Witschey  wrote:
>
> Not knowing the problem you're trying to solve, I'm going to guess
> cqlsh is a bad tool for this job. If you want to put the results of
> CQL queries into a shell pipeline, a small custom script using a
> driver is probably a better fit, and should be possible to write
> without much effort.
>
> -- -- -
> To unsubscribe, e-mail: user-unsubscribe@cassandra. apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>


-- -- -
To unsubscribe, e-mail: user-unsubscribe@cassandra. apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org





   

Re: Migrate from DSE (Datastax) to Apache Cassandra

[Jon Haddad]

I agree with Jeff, it’s not necessary to launch a new cluster for this 
operation.

> On Aug 15, 2017, at 7:39 PM, Jeff Jirsa  wrote:
> 
> Or just alter the key space replication strategy and remove the DSE specific 
> strategies in favor of network topology strategy
> 
> 
> -- 
> Jeff Jirsa
> 
> 
> On Aug 15, 2017, at 7:26 PM, Erick Ramirez  > wrote:
> 
>> Ioannis, it's not a straightforward process to migrate from DSE to COSS. 
>> There are some parts of DSE which are not recognised by COSS, e.g. 
>> EverywhereStrategy for replication only known to DSE.
>> 
>> You are better off standing up a new COSS 3.11 cluster and restore app 
>> keyspaces to the new cluster. Cheers!
>> 
>> On Wed, Aug 16, 2017 at 6:33 AM, Ioannis Zafiropoulos > > wrote:
>> Hi all,
>> 
>> We have setup a new cluster DSE 5.1.2 (with Cassandra 3.11.0.1758) and we 
>> want to migrate it to Apache Cassandra 3.11.0 without loosing schema or data.
>> 
>> Anybody, has done it before?
>> 
>> Obviously we are going to test this, but it would be nice to hear if 
>> somebody else has gone through with the procedure.
>> 
>> Thank you!
>> 

Re: Migrate from DSE (Datastax) to Apache Cassandra

[Jeff Jirsa]

Or just alter the key space replication strategy and remove the DSE specific 
strategies in favor of network topology strategy


-- 
Jeff Jirsa


> On Aug 15, 2017, at 7:26 PM, Erick Ramirez  wrote:
> 
> Ioannis, it's not a straightforward process to migrate from DSE to COSS. 
> There are some parts of DSE which are not recognised by COSS, e.g. 
> EverywhereStrategy for replication only known to DSE.
> 
> You are better off standing up a new COSS 3.11 cluster and restore app 
> keyspaces to the new cluster. Cheers!
> 
>> On Wed, Aug 16, 2017 at 6:33 AM, Ioannis Zafiropoulos  
>> wrote:
>> Hi all,
>> 
>> We have setup a new cluster DSE 5.1.2 (with Cassandra 3.11.0.1758) and we 
>> want to migrate it to Apache Cassandra 3.11.0 without loosing schema or data.
>> 
>> Anybody, has done it before?
>> 
>> Obviously we are going to test this, but it would be nice to hear if 
>> somebody else has gone through with the procedure.
>> 
>> Thank you!
> 

Re: Migrate from DSE (Datastax) to Apache Cassandra

[Erick Ramirez]

Ioannis, it's not a straightforward process to migrate from DSE to COSS.
There are some parts of DSE which are not recognised by COSS, e.g.
EverywhereStrategy for replication only known to DSE.

You are better off standing up a new COSS 3.11 cluster and restore app
keyspaces to the new cluster. Cheers!

On Wed, Aug 16, 2017 at 6:33 AM, Ioannis Zafiropoulos 
wrote:

> Hi all,
>
> We have setup a new cluster DSE 5.1.2 (with Cassandra 3.11.0.1758) and we
> want to migrate it to Apache Cassandra 3.11.0 without loosing schema or
> data.
>
> Anybody, has done it before?
>
> Obviously we are going to test this, but it would be nice to hear if
> somebody else has gone through with the procedure.
>
> Thank you!
>

Re: Attempted to write commit log entry for unrecognized table

[Erick Ramirez]

Myron, it just means that while the node was down, one of the tables got
dropped. When you eventually brought the node back online and the commit
logs were getting replayed, it tried to replay a mutation for a table which
no longer exists. Cheers!

On Wed, Aug 16, 2017 at 5:56 AM, Myron A. Semack 
wrote:

> We have a Cassandra 2.2.10 cluster of 9 nodes, hosted in AWS.  One of the
> nodes had a problem where it ran out of space on its root volume (NOT the
> Cassandra volume which holds the Cassandra data and commit logs).  I
> resolved the issue with free space on the root volume and restarted the
> node.  But now the node is showing this error in the Cassandra system.log:
>
> ERROR [SharedPool-Worker-4] 2017-08-15 19:45:54,519
> CommitLogSegment.java:421 - Attempted to write commit log entry for
> unrecognized table: 1b58bb96-a35d-366a-bfac-b3bb75a8bf97
>
> I did some searching, but was unable to find any guidance on how to deal
> with this error.  Is there a way to recover from this error?  (Preferably
> without terminating and rebuilding the node.)
>
> Sincerely,
> Myron A. Semack
>
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

Re: Creating a copy of a C* cluster

[Erick Ramirez]

Fay, it's normal to need to increase the max heap for sstableloader if you
have large data sets. Cheers!

On Wed, Aug 16, 2017 at 1:18 AM, Fay Hou [Storage Service] ­ <
fay...@coupang.com> wrote:

> We do snapshot and sstableloader. with sstableloader, it is ok to have
> different configuration of the "stand by" cluster (i.e. number of the
> nodes).
> however, there is a issue we ran into with the sstableloader 
> (java.lang.OutOfMemoryError:
> GC overhead limit exceeded)
>  https://issues.apache.org/jira/browse/CASSANDRA-7385
>
> Thanks,
> Fay
>
> On Tue, Aug 15, 2017 at 12:58 AM, Erick Ramirez 
> wrote:
>
>> A slight variation to Ben Slater's idea is to build the second cluster
>> like-for-like and assigning the same tokens used by the original nodes. If
>> you restore the data onto the equivalent nodes with the same tokens, the
>> data will be accessible as normal. Cheers!
>>
>> On Tue, Aug 8, 2017 at 7:08 AM, Robert Wille  wrote:
>>
>>> We need to make a copy of a cluster. We’re going to do some testing
>>> against the copy and then discard it. What’s the best way of doing that? I
>>> created another datacenter, and then have tried to divorce it from the
>>> original datacenter, but have had troubles doing so.
>>>
>>> Suggestions?
>>>
>>> Thanks in advance
>>>
>>> Robert
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
>>> For additional commands, e-mail: user-h...@cassandra.apache.org
>>>
>>
>>
>

Re: SASI index returns no results

[Erick Ramirez]

Have you tried tracing (TRACING ON) the query? That would usually give you
clues as to where it's failing. Cheers!

On Wed, Aug 16, 2017 at 12:03 AM, Vladimir Yudovin 
wrote:

> Hi,
>
> I recently encountered with strange issue.
> Assuming there is table
>
> id PRIMARY KEY
> indexed text
> column text
>
> CREATE custom index on table(indexed) using '...SASIIndex'
>
> I inserted row like *id=0, indexed='string1', column='just string'*
>
> When I did *SELECT * FROM table WHERE id=0 AND indexed='string1' *no rows
> is returned, while *SELECT * FROM table WHERE id=0 AND column =' just
> string' ALLOW FILTERING* returned the row.
>
> In reality this table consist about 70 columns, 20M rows and probably
> total 30G on disk with RF=2 and three nodes. I guess this issue is somehow
> linked to memory usage, as I run some different heavy queries, then cluster
> become unresponsible to cqlsh and nodetool. I sow GC messages in gc.log, so
> I restated cluster and all returned to normal.
>
> I would expect the whole query to fail with timeout or some error message,
> but not to silently return zero rows.
> Any thoughts?
>
> Best regards, Vladimir Yudovin,
> *Winguzone  - Cloud Cassandra Hosting*
>
>

Re: Migrate from DSE (Datastax) to Apache Cassandra

[kurt greaves]

Haven't done it for 5.1 but went smoothly for earlier versions. If you're
not using any of the additional features of DSE, it should be OK. Just
change any custom replication strategies before migrating and also make
sure your yaml options are compatible.

Re: Attempted to write commit log entry for unrecognized table

[kurt greaves]

what does nodetool describecluster show?
stab in the dark but you could try nodetool resetlocalschema  or a rolling
restart of the cluster if it's schema issues.

Migrate from DSE (Datastax) to Apache Cassandra

[Ioannis Zafiropoulos]

Hi all,

We have setup a new cluster DSE 5.1.2 (with Cassandra 3.11.0.1758) and we
want to migrate it to Apache Cassandra 3.11.0 without loosing schema or
data.

Anybody, has done it before?

Obviously we are going to test this, but it would be nice to hear if
somebody else has gone through with the procedure.

Thank you!

Attempted to write commit log entry for unrecognized table

[Myron A. Semack]

We have a Cassandra 2.2.10 cluster of 9 nodes, hosted in AWS.  One of the nodes 
had a problem where it ran out of space on its root volume (NOT the Cassandra 
volume which holds the Cassandra data and commit logs).  I resolved the issue 
with free space on the root volume and restarted the node.  But now the node is 
showing this error in the Cassandra system.log:

ERROR [SharedPool-Worker-4] 2017-08-15 19:45:54,519 CommitLogSegment.java:421 - 
Attempted to write commit log entry for unrecognized table: 
1b58bb96-a35d-366a-bfac-b3bb75a8bf97

I did some searching, but was unable to find any guidance on how to deal with 
this error.  Is there a way to recover from this error?  (Preferably without 
terminating and rebuilding the node.)

Sincerely,
Myron A. Semack


-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Creating a copy of a C* cluster

[Fay Hou [Storage Service] ­]

We do snapshot and sstableloader. with sstableloader, it is ok to have
different configuration of the "stand by" cluster (i.e. number of the
nodes).
however, there is a issue we ran into with the sstableloader
(java.lang.OutOfMemoryError:
GC overhead limit exceeded)
 https://issues.apache.org/jira/browse/CASSANDRA-7385

Thanks,
Fay
On Tue, Aug 15, 2017 at 12:58 AM, Erick Ramirez 
wrote:

> A slight variation to Ben Slater's idea is to build the second cluster
> like-for-like and assigning the same tokens used by the original nodes. If
> you restore the data onto the equivalent nodes with the same tokens, the
> data will be accessible as normal. Cheers!
>
> On Tue, Aug 8, 2017 at 7:08 AM, Robert Wille  wrote:
>
>> We need to make a copy of a cluster. We’re going to do some testing
>> against the copy and then discard it. What’s the best way of doing that? I
>> created another datacenter, and then have tried to divorce it from the
>> original datacenter, but have had troubles doing so.
>>
>> Suggestions?
>>
>> Thanks in advance
>>
>> Robert
>>
>>
>> -
>> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: user-h...@cassandra.apache.org
>>
>
>

Re: SASI index returns no results

[Fay Hou [Storage Service] ­]

SASI is in early experiment and had many major problems. for example,

"nodetool repair breaks SASI index"

https://issues.apache.org/jira/browse/CASSANDRA-13403

"OOM when using SASI index"
https://issues.apache.org/jira/browse/CASSANDRA-12662

I would not use SASI index for production.

Fay

On Tue, Aug 15, 2017 at 7:03 AM, Vladimir Yudovin 
wrote:

> Hi,
>
> I recently encountered with strange issue.
> Assuming there is table
>
> id PRIMARY KEY
> indexed text
> column text
>
> CREATE custom index on table(indexed) using '...SASIIndex'
>
> I inserted row like *id=0, indexed='string1', column='just string'*
>
> When I did *SELECT * FROM table WHERE id=0 AND indexed='string1' *no rows
> is returned, while *SELECT * FROM table WHERE id=0 AND column =' just
> string' ALLOW FILTERING* returned the row.
>
> In reality this table consist about 70 columns, 20M rows and probably
> total 30G on disk with RF=2 and three nodes. I guess this issue is somehow
> linked to memory usage, as I run some different heavy queries, then cluster
> become unresponsible to cqlsh and nodetool. I sow GC messages in gc.log, so
> I restated cluster and all returned to normal.
>
> I would expect the whole query to fail with timeout or some error message,
> but not to silently return zero rows.
> Any thoughts?
>
> Best regards, Vladimir Yudovin,
> *Winguzone  - Cloud Cassandra Hosting*
>
>

SASI index returns no results

[Vladimir Yudovin]

Hi,



I recently encountered with strange issue.

Assuming there is table



id PRIMARY KEY

indexed text

column text



CREATE custom index on table(indexed) using '...SASIIndex'



I inserted row like id=0, indexed='string1', column='just string'



When I did SELECT * FROM table WHERE id=0 AND indexed='string1' no rows is 
returned, while SELECT * FROM table WHERE id=0 AND column =' just string' ALLOW 
FILTERING returned the row.



In reality this table consist about 70 columns, 20M rows and probably total 30G 
on disk with RF=2 and three nodes. I guess this issue is somehow linked to 
memory usage, as I run some different heavy queries, then cluster become 
unresponsible to cqlsh and nodetool. I sow GC messages in gc.log, so I restated 
cluster and all returned to normal.



I would expect the whole query to fail with timeout or some error message, but 
not to silently return zero rows.

Any thoughts?



Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting

Re: MUTATION messages were dropped in last 5000 ms for cross node timeout

[Erick Ramirez]

Mutations get dropped because a node can't keep up with writes. If you
understand the Cassandra write path, writes are ACKed when the mutation is
appended to the commitlog which is why it's very fast.

Knowing that, dropped mutations mean that the disk is not able to keep up
with the IO. Another word for it is "overloaded".

So what do you do? Use fast SSDs (HDDs and EBS won't do) and size your
cluster correctly to deal with the peak load, i.e. add more nodes. Cheers!

On Fri, Jul 21, 2017 at 1:27 AM, ZAIDI, ASAD A  wrote:

> Hello Folks –
>
>
>
> I’m using apache-cassandra 2.2.8.
>
>
>
> I see many messages like below in my system.log file. In Cassandra.yaml
> file [ cross_node_timeout: true] is set and NTP server is also running
> correcting clock drift on 16node cluster. I do not see pending or blocked
> HintedHandoff  in tpstats output though there are bunch of MUTATIONS
> dropped observed.
>
>
>
> 
>
> INFO  [ScheduledTasks:1] 2017-07-20 08:02:52,511 MessagingService.java:946
> - MUTATION messages were dropped in last 5000 ms: 822 for internal timeout
> and 2152 for cross node timeout
>
> 
>
>
>
> I’m seeking help here if you please let me know what I need to check in
> order to address these cross node timeouts.
>
>
>
> Thank you,
>
> Asad
>
>
>

Re: Error Exception in Repair Thread

[Erick Ramirez]

2 common causes of interrupted streams are (a) network interruptions, or
(b) nodes becoming unresponsive, e.g. GC pause during high loads.

As far as network is concerned, is there a firewall in the middle? If so,
it's quite common for firewalls to close sockets when it thinks the
connection is idle and happens for long-running streams. Set the TCP
keep-alive variables on the nodes. For example:

$ sudo sysctl -w net.ipv4.tcp_keepalive_time=60
net.ipv4.tcp_keepalive_probes=3 net.ipv4.tcp_keepalive_intvl=10

On Fri, Jul 28, 2017 at 4:50 AM, Meg Mara  wrote:

> Hello Cassandra Experts!
>
>
>
> I have seen the following errors in the system log when a scheduled
> nodetool repair operation runs on the cluster. Cassandra Version 3.0.10.
> Any thoughts or suggestions are welcome!
>
>
>
> ERROR [Repair#3794:3] 2017-07-27 17:47:22,000 CassandraDaemon.java:207 -
> Exception in thread Thread[Repair#3794:3,5,RMI Runtime]
>
> java.lang.AssertionError: java.lang.InterruptedException
>
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.
> extractThrowable(DebuggableThreadPoolExecutor.java:265)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
>
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.
> logExceptionsAfterExecute(DebuggableThreadPoolExecutor.java:225)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
>
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.
> afterExecute(DebuggableThreadPoolExecutor.java:196)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
>
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> ~[na:1.8.0_101]
>
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_101]
>
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
>
> Caused by: java.lang.InterruptedException: null
>
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.
> acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
> ~[na:1.8.0_101]
>
> at com.google.common.util.concurrent.AbstractFuture$
> Sync.get(AbstractFuture.java:285) ~[guava-18.0.jar:na]
>
> at 
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
> ~[guava-18.0.jar:na]
>
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.
> extractThrowable(DebuggableThreadPoolExecutor.java:261)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
>
> ... 5 common frames omitted
>
>
>
> Thank you,
>
> Meg Mara
>
>
>

Re: Questions on time series use case, tombstones, TWCS

[Erick Ramirez]

Not sure if these are what Jeff was referring to but as a workaround, you
can configure the following STCS compaction subproperties:
- min_threshold - set to 2 so that only a minimum of 2 similar-sized
sstables are required to trigger a minor compaction instead of the default 4
- tombstone_threshold - set to 0.1 so that if at least 10% of an sstable
are tombstones, Cassandra will compact the table alone instead of waiting
for the higher default ratio of 0.2
- unchecked_tombstone_compaction - set to true to allow Cassandra to run
tombstone compaction without having to check if an sstable is eligible for
compaction

WARNING - For future reference, this is just a workaround. It isn't a fix
for clusters with bad data models. Consider these as buying your cluster
some breathing space while you redesign your data model. Cheers!

On Thu, Aug 10, 2017 at 12:27 AM, Jeff Jirsa  wrote:

> The deleting compaction strategy from protectwise (https://github.com/
> protectwise/cassandra-util/blob/master/deleting-
> compaction-strategy/README.md) was written (I believe) to solve a similar
> problem - business based deletion rules to enable flexible TTLs. May want
> to glance at that.
>
> Other answers inline below
>
>
> --
> Jeff Jirsa
>
>
> On Aug 9, 2017, at 1:41 AM, Steinmaurer, Thomas <
> thomas.steinmau...@dynatrace.com> wrote:
>
> Hello,
>
>
>
> our top contributor from a data volume perspective is time series data. We
> are running with STCS since our initial production deployment in 2014 with
> several clusters with a varying number of nodes, but currently with max. 9
> nodes per single cluster per different region in AWS with m4.xlarge / EBS
> gp2 storage. We have a road of Cassandra versions starting with 1.2 to
> actually using DSC 2.1.15 soon to be replaced by Apache Cassandra 2.1.18
> across all deployments. Lately we switched from Thrift (Astyanax) to
> Native/CQL (DataStax driver). Overall we are pretty happy with stability
> and the scale out offering.
>
>
>
> We store time series data in different resolutions, from 1min up to 1day
> aggregates per “time slot”. Each resolution has its own column family /
> table and a periodic worker is executing our business logic regarding time
> series aging from e.g. 1min => 5min => … resolution + deletion in source
> resolutions according to our retention per resolution policy. So deletions
> will happen way later (e.g. at least > 14d). We don’t use TTLs on written
> time series data (in production, see TWCS testing below), so purging is
> exclusively handled by explicit DELETEs in our aging business logic
> creating tombstones.
>
>
>
> Naturally with STCS and late explicit deletions / tombstones, it will take
> a lot of time to finally reclaim disk space, even worse, we are now running
> a major compaction every X weeks. We currently are also testing with STCS
> min_threshold = 2 etc., but all in all, this all feels not being a
> long-term solution. I guess there is nothing else we are missing from a
> configuration/setting side with STCS? Single SSTable compaction might not
> kick in as well, cause checking with sstablemeta, estimated droppable
> tombstones for our time series based SSTables is pretty much 0.0 all the
> time. I guess as we don’t write with TTL?
>
>
>
> Or you aren't issuing deletes, explicit deletes past GCGS will cause that
> number to increase
>
>
>
> TWCS caught my eyes in 2015 I think, and even more at the Cassandra Summit
> 2016 + other Tombstone related talks. Cassandra 3.0 is around 6 months
> ahead for us, thus initial testing was with 2.1.18 patched with TWCS from
> GitHub.
>
>
>
> Looks like TWCS is exactly what we need, thus test-wise, once we start
> writing with TTL we end up with a single SSTable per passed window size and
> data (SSTables) older than TTL + grace get automatically removed from disk.
> Even with enabled out-of-orders DELETEs from our business logic, purging
> SSTables seems not be stucked. Not sure if this is expected. Writing with
> TTL is also a bit problematic, in case our retention policy changes in
> general or for particular customers.
>
>
> Search for my Cassandra summit talk from 2016 - there's a few other
> compaction options you probably want to set to more aggressively trigger
> single sstable compaction to help unstick it.
>
>
>
> A few questions, as we need some short-term (with C* 2.1) and long-term
> (with C* 3.0) mitigation:
>
> · With STCS, estimated droppable tombstones being always 0.0
> (thus also no automatic single SSTable compaction may happen): Is this a
> matter of not writing with TTL? If yes, would enabling TTL with STCS
> improve the disk reclaim situation, cause then single SSTAble compactions
> will kick in?
>
> · What is the semantic of “default_time_to_live” at table level?
> From: http://docs.datastax.com/en/cql/3.1/cql/cql_using/use_expire_c.html
> : “After the default_time_to_live TTL value has been exceed, Cassandra
> tombstones the entire table”. 

Re: Creating a copy of a C* cluster

[Erick Ramirez]

A slight variation to Ben Slater's idea is to build the second cluster
like-for-like and assigning the same tokens used by the original nodes. If
you restore the data onto the equivalent nodes with the same tokens, the
data will be accessible as normal. Cheers!

On Tue, Aug 8, 2017 at 7:08 AM, Robert Wille  wrote:

> We need to make a copy of a cluster. We’re going to do some testing
> against the copy and then discard it. What’s the best way of doing that? I
> created another datacenter, and then have tried to divorce it from the
> original datacenter, but have had troubles doing so.
>
> Suggestions?
>
> Thanks in advance
>
> Robert
>
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>

Re: cqlsh -e output - How to change the default delimiter '|' in the output

[Erick Ramirez]

+1 to Jim and Tobin. cqlsh wasn't designed for what you're trying to
achieve. Cheers!

On Tue, Aug 15, 2017 at 1:34 AM, Tobin Landricombe 
wrote:

> Can't change the delimiter (I'm on cqlsh 5.0.1). Best I can offer is
> https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlshExpand.html
>
> > On 14 Aug 2017, at 16:17, Jim Witschey 
> wrote:
> >
> > Not knowing the problem you're trying to solve, I'm going to guess
> > cqlsh is a bad tool for this job. If you want to put the results of
> > CQL queries into a shell pipeline, a small custom script using a
> > driver is probably a better fit, and should be possible to write
> > without much effort.
> >
> > -
> > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: user-h...@cassandra.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

Re: live dsc upgrade from 2.0 to 2.1 behind the scenes

[Erick Ramirez]

1) You should not perform any streaming operations (repair, bootstrap,
decommission) in the middle of an upgrade. Note that an upgrade is not
complete until you have completed upgradesstables on all nodes in the
cluster.

2) No streaming involved with writes so it's not an issue.

3) It doesn't matter whichever way you do it. My personal experience is
that it's best to do it as you go but YMMV.

4) It depends on a lot of factors including type of disks (e.g. SSDs vs
HDDs), data model, access patterns, cluster load, etc. The only way you'll
be able to estimate it is by running your own tests.

5) There is no "max" time but it is preferable that you complete the
upgrades in the shortest amount of time. Until you have completed
upgradesstables on all nodes, there is a performance hit with reading older
generations of sstables. I'm sure you're about to ask "how much perf hit?"
and the answer is "test it".

6) It is not advisable to perform schema changes in mixed-mode -- the
schema version on upgraded nodes are different and there will be a mismatch
until all nodes are upgraded.

Good luck!

On Mon, Aug 14, 2017 at 12:14 PM, Park Wu  wrote:

> Hi, folks: I am planning to upgrade our production from dsc 2.0.16 to
> 2.1.18 for 2 DC (20 nodes each, 600GB per node). Few questions:
> 1), what happen when doing rolling upgrade. Let's say we only upgrade one
> node to new version, before upgrade sstable, the data coming in will stay
> in the node and not be able to stream to other nodes?
> 2), What if I have very active writes? how much data it can hold until it
> sees other nodes with new version so it can stream?
> 3), Should I upgrade sstable when all nodes in one DC upgraded? or wait
> until all 2 DC upgraded?
> 4), any idea or experience how long it will take to upgrade sstable for
> 600 GB data on each node?
> 5), what is the max time I can take for rolling upgrade on each DC?
> 6), I was doing a test with 3-nodes cluster, one node with 2.1.18, other
> two are 2.0.16. I got a warning on the node with newer version when I tried
> to create keyspace and insert some sample data:
> "Warning: schema version mismatch detected, which might be caused by DOWN
> nodes; if this is not the case, check the schema versions of your nodes in
> system.local and system.peers. OperationTimedOut: errors={},
> last_host=xxx"
> But data upserted successfully, even not be seen on other nodes. Any
> suggestion?
> Great thanks for any help or comments!
> - Park
>

Re: Dropping down replication factor

[Erick Ramirez]

I would discourage dropping to RF=2 because if you're using CL=*QUORUM, it
won't be able to tolerate a node outage.

You mentioned a couple of days ago that there's an index file that is
corrupted on 10.40.17.114. Could you try moving out the sstable set
associated with that corrupt file and try again? Though I echo Jeff's
comments and I'm concerned you have a hardware issue on that node since
OpsCenter tables got corrupted too. The replace method certainly sounds
like a good idea.

On Sun, Aug 13, 2017 at 7:58 AM,  wrote:

> Hi folks, hopefully a quick one:
>
> We are running a 12 node cluster (2.1.15) in AWS with Ec2Snitch.  It's all
> in one region but spread across 3 availability zones.  It was nicely
> balanced with 4 nodes in each.
>
> But with a couple of failures and subsequent provisions to the wrong az we
> now have a cluster with :
>
> 5 nodes in az A
> 5 nodes in az B
> 2 nodes in az C
>
> Not sure why, but when adding a third node in AZ C it fails to stream
> after getting all the way to completion and no apparent error in logs.
> I've looked at a couple of bugs referring to scrubbing and possible OOM
> bugs due to metadata writing at end of streaming (sorry don't have ticket
> handy).  I'm worried I might not be able to do much with these since the
> disk space usage is high and they are under a lot of load given the small
> number of them for this rack.
>
> Rather than troubleshoot this further, what I was thinking about doing was:
> - drop the replication factor on our keyspace to two
> - hopefully this would reduce load on these two remaining nodes
> - run repairs/cleanup across the cluster
> - then shoot these two nodes in the 'c' rack
> - run repairs/cleanup across the cluster
>
> Would this work with minimal/no disruption?
> Should I update their "rack" before hand or after ?
> What else am I not thinking about?
>
> My main goal atm is to get back to where the cluster is in a clean
> consistent state that allows nodes to properly bootstrap.
>
> Thanks for your help in advance.
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

Re: MemtablePostFlush pending

[Erick Ramirez]

Check what you have set for memtable_cleanup_threshold and if it's set too
low which means more flushing triggered. Cheers!

On Sat, Aug 12, 2017 at 5:05 AM, ZAIDI, ASAD A  wrote:

> Hello Folks,
>
>
>
> I’m using Cassandra 2.2 on 14 node cluster.
>
>
>
> Now a days, I’m observing memtablepostflush pending number going high ,
> this happens intermittently. I’m looking if  Is there way to ‘tune’
> memtablepostflush stage?
>
>
>
> Thanks/ASad
>
>
>