Re: partition sizes reported by nodetool tablehistograms

[Chris Lohfink]

Its the decompressed size of the partitions. Each sstable has stats
component that contains histograms for the size and number of columns in
the partitions (among other things, can see with sstablemetadata tool),
tablehistograms merges it for each sstable and gives the results.

Chris

On Fri, Feb 24, 2017 at 4:53 PM, John Sanda  wrote:

> I am working on some issues involving really big partitions. I have been
> making extensive use of nodetool tablehistograms. What exactly is the
> partition size being reported? I have a table for which the max value
> reported is about 3.5 GB, but running du -h against the table data
> directory reports 548 MB. Are the partition sizes reported by
> tablehistograms the decompressed size on disk?
>
> - John
>

partition sizes reported by nodetool tablehistograms

[John Sanda]

I am working on some issues involving really big partitions. I have been
making extensive use of nodetool tablehistograms. What exactly is the
partition size being reported? I have a table for which the max value
reported is about 3.5 GB, but running du -h against the table data
directory reports 548 MB. Are the partition sizes reported by
tablehistograms the decompressed size on disk?

- John

Re: Read after Write inconsistent at times

[Charulata Sharma (charshar)]

Hi All,
  Thanks for your replies. I do not see an issue with NTP or with dropped 
messages.
However the tombstones count on the specific CF shows me this. This essentially 
indicates that there are as many tombstones as Live cells in the CF isin't?
Now is that an issue and can this cause inconsistent read ?


Average live cells per slice (last five minutes): 0.8631938498408056

Maximum live cells per slice (last five minutes): 1.0

Average tombstones per slice (last five minutes): 1.1477603751799115E-5

Maximum tombstones per slice (last five minutes): 1.0


Thanks,

Charu

From: Jonathan Haddad >
Reply-To: "user@cassandra.apache.org" 
>
Date: Friday, February 24, 2017 at 9:42 AM
To: "user@cassandra.apache.org" 
>
Subject: Re: Read after Write inconsistent at times

WRT to NTP, I first encountered this issue on my first cluster.  The problem 
with ntp isn't just if you're doing inserts, it's if you're doing inserts in 
combination with deletes, and using server timestamps with a greater variance 
than the period between the delete and the insert.  Basically, you end up with 
a delete in the future and an insert in the past, and the delete timestamp > 
insert timestamp.

+1 to Jan's recommendation on checking for dropped messages.


On Fri, Feb 24, 2017 at 9:35 AM Petrus Gomes 
> wrote:
Hi,

Check the tombstone count, If is it to high, your query will be impacted.

If tombstone is a problem, you can try to reduce your "gc_grace_seconds" to 
reduce tombstone count(Carefully because you use cross data centers).

Tchau,
Petrus Silva



On Fri, Feb 24, 2017 at 12:07 AM, Jan Kesten 
> wrote:
Hi,

are your nodes at high load? Are there any dropped messages (nodetool tpstats) 
on any node?

Also have a look at your system clocks. C* needs them in thight sync - via ntp 
for example. Side hint: if you use ntp use the same set of upstreams on all of 
your nodes - ideal your own one. Using pool.ntp.org might 
lead to minimal dirfts in time across your cluster.

Another thing that could help you out is using client side timestamps: 
https://docs.datastax.com/en/developer/java-driver/3.1/manual/query_timestamps/ 
(of course only when you are using a single client or all clients are in sync 
via ntp).



Am 24.02.2017 um 07:29 schrieb Charulata Sharma (charshar):

Hi All,

In my application sometimes I cannot read data that just got inserted. This 
happens very intermittently. Both write and read use LOCAL QUOROM.

We have a cluster of 12 nodes which spans across 2 Data Centers and a RF of 3.

Has anyone encountered this problem and if yes what steps have you taken to 
solve it

Thanks,
Charu

--
Jan Kesten, mailto:j.kes...@enercast.de
Tel.: +49 561/4739664-0 FAX: -9 Mobil: +49 160 / 
90 98 41 68
enercast GmbH Universitätsplatz 12 D-34127 Kassel HRB15471
http://www.enercast.de Online-Prognosen für erneuerbare Energien
Geschäftsführung: Thomas Landgraf (CEO), Bernd Kratz (CTO), Philipp Rinder (CSO)

Diese E-Mail und etwaige Anhänge können vertrauliche und/oder rechtlich 
geschützte Informationen enthalten. Falls Sie nicht der angegebene Empfänger 
sind oder falls diese E-Mail irrtümlich an Sie adressiert wurde, 
benachrichtigen Sie uns bitte sofort durch Antwort-E-Mail und löschen Sie diese 
E-Mail nebst etwaigen Anlagen von Ihrem System. Ebenso dürfen Sie diese E-Mail 
oder ihre Anlagen nicht kopieren oder an Dritte weitergeben. Vielen Dank.

This e-mail and any attachment may contain confidential and/or privileged 
information. If you are not the named addressee or if this transmission has 
been addressed to you in error, please notify us immediately by reply e-mail 
and then delete this e-mail and any attachment from your system. Please 
understand that you must not copy this e-mail or any attachment or disclose the 
contents to any other person. Thank you for your cooperation.

Re: High disk io read load

[Benjamin Roth]

It was only the schema change.

2017-02-24 19:18 GMT+01:00 kurt greaves :

> How many CFs are we talking about here? Also, did the script also kick off
> the scrubs or was this purely from changing the schemas?
> ​
>



-- 
Benjamin Roth
Prokurist

Jaumo GmbH · www.jaumo.com
Wehrstraße 46 · 73035 Göppingen · Germany
Phone +49 7161 304880-6 · Fax +49 7161 304880-1
AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: High disk io read load

[kurt greaves]

How many CFs are we talking about here? Also, did the script also kick off
the scrubs or was this purely from changing the schemas?
​

Re: Read after Write inconsistent at times

[Jonathan Haddad]

WRT to NTP, I first encountered this issue on my first cluster.  The
problem with ntp isn't just if you're doing inserts, it's if you're doing
inserts in combination with deletes, and using server timestamps with a
greater variance than the period between the delete and the insert.
Basically, you end up with a delete in the future and an insert in the
past, and the delete timestamp > insert timestamp.

+1 to Jan's recommendation on checking for dropped messages.


On Fri, Feb 24, 2017 at 9:35 AM Petrus Gomes  wrote:

> Hi,
>
> Check the tombstone count, If is it to high, your query will be impacted.
>
> If tombstone is a problem, you can try to reduce your "gc_grace_seconds"
> to reduce tombstone count(Carefully because you use cross data centers).
>
> Tchau,
> Petrus Silva
>
>
>
> On Fri, Feb 24, 2017 at 12:07 AM, Jan Kesten  wrote:
>
> Hi,
>
> are your nodes at high load? Are there any dropped messages (nodetool
> tpstats) on any node?
>
> Also have a look at your system clocks. C* needs them in thight sync - via
> ntp for example. Side hint: if you use ntp use the same set of upstreams on
> all of your nodes - ideal your own one. Using pool.ntp.org might lead to
> minimal dirfts in time across your cluster.
>
> Another thing that could help you out is using client side timestamps:
> https://docs.datastax.com/en/developer/java-driver/3.1/manual/query_timestamps/
> (of course only when you are using a single client or all clients are in
> sync via ntp).
>
>
>
> Am 24.02.2017 um 07:29 schrieb Charulata Sharma (charshar):
>
>
> Hi All,
>
> In my application sometimes I cannot read data that just got inserted.
> This happens very intermittently. Both write and read use LOCAL QUOROM.
>
> We have a cluster of 12 nodes which spans across 2 Data Centers and a RF
> of 3.
>
> Has anyone encountered this problem and if yes what steps have you taken
> to solve it
>
> Thanks,
> Charu
>
>
> --
> Jan Kesten, mailto:j.kes...@enercast.de
> Tel.: +49 561/4739664-0 FAX: -9 Mobil: +49 160 / 90 98 41 68
> enercast GmbH Universitätsplatz 12 D-34127 Kassel HRB15471
> http://www.enercast.de Online-Prognosen für erneuerbare Energien
> Geschäftsführung: Thomas Landgraf (CEO), Bernd Kratz (CTO), Philipp Rinder
> (CSO)
>
> Diese E-Mail und etwaige Anhänge können vertrauliche und/oder rechtlich
> geschützte Informationen enthalten. Falls Sie nicht der angegebene
> Empfänger sind oder falls diese E-Mail irrtümlich an Sie adressiert wurde,
> benachrichtigen Sie uns bitte sofort durch Antwort-E-Mail und löschen Sie
> diese E-Mail nebst etwaigen Anlagen von Ihrem System. Ebenso dürfen Sie
> diese E-Mail oder ihre Anlagen nicht kopieren oder an Dritte weitergeben.
> Vielen Dank.
>
> This e-mail and any attachment may contain confidential and/or privileged
> information. If you are not the named addressee or if this transmission has
> been addressed to you in error, please notify us immediately by reply
> e-mail and then delete this e-mail and any attachment from your system.
> Please understand that you must not copy this e-mail or any attachment or
> disclose the contents to any other person. Thank you for your cooperation.
>
>
>

Re: Read after Write inconsistent at times

[Petrus Gomes]

Hi,

Check the tombstone count, If is it to high, your query will be impacted.

If tombstone is a problem, you can try to reduce your "gc_grace_seconds" to
reduce tombstone count(Carefully because you use cross data centers).

Tchau,
Petrus Silva



On Fri, Feb 24, 2017 at 12:07 AM, Jan Kesten  wrote:

> Hi,
>
> are your nodes at high load? Are there any dropped messages (nodetool
> tpstats) on any node?
>
> Also have a look at your system clocks. C* needs them in thight sync - via
> ntp for example. Side hint: if you use ntp use the same set of upstreams on
> all of your nodes - ideal your own one. Using pool.ntp.org might lead to
> minimal dirfts in time across your cluster.
>
> Another thing that could help you out is using client side timestamps:
> https://docs.datastax.com/en/developer/java-driver/3.1/manua
> l/query_timestamps/ (of course only when you are using a single client or
> all clients are in sync via ntp).
>
>
>
> Am 24.02.2017 um 07:29 schrieb Charulata Sharma (charshar):
>
>>
>> Hi All,
>>
>> In my application sometimes I cannot read data that just got inserted.
>> This happens very intermittently. Both write and read use LOCAL QUOROM.
>>
>> We have a cluster of 12 nodes which spans across 2 Data Centers and a RF
>> of 3.
>>
>> Has anyone encountered this problem and if yes what steps have you taken
>> to solve it
>>
>> Thanks,
>> Charu
>>
>
> --
> Jan Kesten, mailto:j.kes...@enercast.de
> Tel.: +49 561/4739664-0 FAX: -9 Mobil: +49 160 / 90 98 41 68
> enercast GmbH Universitätsplatz 12 D-34127 Kassel HRB15471
> http://www.enercast.de Online-Prognosen für erneuerbare Energien
> Geschäftsführung: Thomas Landgraf (CEO), Bernd Kratz (CTO), Philipp Rinder
> (CSO)
>
> Diese E-Mail und etwaige Anhänge können vertrauliche und/oder rechtlich
> geschützte Informationen enthalten. Falls Sie nicht der angegebene
> Empfänger sind oder falls diese E-Mail irrtümlich an Sie adressiert wurde,
> benachrichtigen Sie uns bitte sofort durch Antwort-E-Mail und löschen Sie
> diese E-Mail nebst etwaigen Anlagen von Ihrem System. Ebenso dürfen Sie
> diese E-Mail oder ihre Anlagen nicht kopieren oder an Dritte weitergeben.
> Vielen Dank.
>
> This e-mail and any attachment may contain confidential and/or privileged
> information. If you are not the named addressee or if this transmission has
> been addressed to you in error, please notify us immediately by reply
> e-mail and then delete this e-mail and any attachment from your system.
> Please understand that you must not copy this e-mail or any attachment or
> disclose the contents to any other person. Thank you for your cooperation.
>
>

Re: Read exceptions after upgrading to 3.0.10

[kurt greaves]

That stacktrace generally implies your clients are resetting connections.
The reconnection policy probably handles the issue automatically, however
worth investigating. I don't think it normally causes statuslogger output
however, what were the log messages prior to the stacktrace?

On 24 February 2017 at 11:57, Carlos Rolo  wrote:

> By any chances are you using the PHP/C++ driver?
>
> --
>
>
>
>

Re: Which compaction strategy when modeling a dumb set

[kurt greaves]

Probably LCS although what you're implying (read before write) is an
anti-pattern in Cassandra. Something like this is a good indicator that you
should review your model.
​

Re: Read exceptions after upgrading to 3.0.10

[Carlos Rolo]

By any chances are you using the PHP/C++ driver?

-- 


--

Which compaction strategy when modeling a dumb set

[Vincent Rischmann]

Hello,



I'm using a table like this:



   CREATE TABLE myset (id uuid PRIMARY KEY)



which is basically a set I use for deduplication, id is a unique id for
an event, when I process the event I insert the id, and before
processing I check if it has already been processed for deduplication.


It works well enough, but I'm wondering which compaction strategy I
should use. I expect maybe 1% or less of events will end up duplicated
(thus not generating an insert), so the workload will probably be 50%
writes 50% read.


Is LCS a good strategy here or should I stick with STCS ?

Re: Read after Write inconsistent at times

[Jan Kesten]

Hi,

are your nodes at high load? Are there any dropped messages (nodetool 
tpstats) on any node?


Also have a look at your system clocks. C* needs them in thight sync - 
via ntp for example. Side hint: if you use ntp use the same set of 
upstreams on all of your nodes - ideal your own one. Using pool.ntp.org 
might lead to minimal dirfts in time across your cluster.


Another thing that could help you out is using client side timestamps: 
https://docs.datastax.com/en/developer/java-driver/3.1/manual/query_timestamps/ 
(of course only when you are using a single client or all clients are in 
sync via ntp).



Am 24.02.2017 um 07:29 schrieb Charulata Sharma (charshar):


Hi All,

In my application sometimes I cannot read data that just got inserted. 
This happens very intermittently. Both write and read use LOCAL QUOROM.


We have a cluster of 12 nodes which spans across 2 Data Centers and a 
RF of 3.


Has anyone encountered this problem and if yes what steps have you 
taken to solve it


Thanks,
Charu


--
Jan Kesten, mailto:j.kes...@enercast.de
Tel.: +49 561/4739664-0 FAX: -9 Mobil: +49 160 / 90 98 41 68
enercast GmbH Universitätsplatz 12 D-34127 Kassel HRB15471
http://www.enercast.de Online-Prognosen für erneuerbare Energien
Geschäftsführung: Thomas Landgraf (CEO), Bernd Kratz (CTO), Philipp Rinder (CSO)

Diese E-Mail und etwaige Anhänge können vertrauliche und/oder rechtlich 
geschützte Informationen enthalten. Falls Sie nicht der angegebene Empfänger 
sind oder falls diese E-Mail irrtümlich an Sie adressiert wurde, 
benachrichtigen Sie uns bitte sofort durch Antwort-E-Mail und löschen Sie diese 
E-Mail nebst etwaigen Anlagen von Ihrem System. Ebenso dürfen Sie diese E-Mail 
oder ihre Anlagen nicht kopieren oder an Dritte weitergeben. Vielen Dank.

This e-mail and any attachment may contain confidential and/or privileged 
information. If you are not the named addressee or if this transmission has 
been addressed to you in error, please notify us immediately by reply e-mail 
and then delete this e-mail and any attachment from your system. Please 
understand that you must not copy this e-mail or any attachment or disclose the 
contents to any other person. Thank you for your cooperation.