Re: Question: Large partition warning

2017-06-16 Thread Jeff Jirsa
On 2017-06-16 10:18 (-0700), "Jeff Jirsa" wrote: > > > On 2017-06-15 16:33 (-0700), kurt greaves wrote: > > fyi ticket already existed for this, I've submitted a patch that fixes this > > specific issue but it looks like there are a few other

Re: Question: Large partition warning

2017-06-16 Thread Jeff Jirsa
On 2017-06-15 16:33 (-0700), kurt greaves wrote: > fyi ticket already existed for this, I've submitted a patch that fixes this > specific issue but it looks like there are a few other properties that will > suffer from the same. As I said on the ticket, we should probably

Re: Question: Large partition warning

2017-06-15 Thread kurt greaves
fyi ticket already existed for this, I've submitted a patch that fixes this specific issue but it looks like there are a few other properties that will suffer from the same. As I said on the ticket, we should probably fix these up even though setting things this high is generally bad practice. If

Re: Question: Large partition warning

2017-06-14 Thread Fay Hou [Data Pipeline & Real-time Analytics] ­
> Jayesh > > > > From: kurt greaves > Sent: Wednesday, June 14, 5:53 PM > Subject: Re: Question: Large partition warning > To: Fay Hou [Data Pipeline & Real-time Analytics] ­ > Cc: Thakrar, Jayesh, User > > > Looks like you've hit a

Re: Question: Large partition warning

2017-06-14 Thread Thakrar, Jayesh
Thank you Kurt - that makes sense. Will certainly reduce it to 1024. Greatly appreciate your quick reply. Thanks, Jayesh From: kurt greaves Sent: Wednesday, June 14, 5:53 PM Subject: Re: Question: Large partition warning To: Fay Hou [Data Pipeline & Real-time Analytics] ­ Cc: Thakrar, Ja

RE: Question: Large partition warning

2017-06-14 Thread ZAIDI, ASAD A
ttps://stackoverflow.com/questions/20512710/cassandra-has-a-limit-of-2-billion-cells-per-partition-but-whats-a-partition] From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com] Sent: Wednesday, June 14, 2017 3:14 PM To: User <user@cassandra.apache.org> Subject: Question: Large partition w

Re: Question: Large partition warning

2017-06-14 Thread kurt greaves
Looks like you've hit a bug (not the first time I've seen this in relation to C* configs). compaction_large_partition_warning_threshold_mb resolves to an int, and in the codebase is represented in bytes. 4096 * 1024 * 1024 and you've got some serious overflow. Granted, you should have this warning

Re: Question: Large partition warning

2017-06-14 Thread Fay Hou [Data Pipeline & Real-time Analytics] ­
nodetool tablehistograms tsg_ae/logs_by_user will give you an idea about the estimate of the partition size. It is recommended that the partition size is not over 100MB Large partitions are also creating heap pressure during compactions, which will issue warnings in the logs (look for "large

Question: Large partition warning

2017-06-14 Thread Thakrar, Jayesh
We are on Cassandra 2.2.5 and I am constantly seeing warning messages about large partitions in system.log even though our setting for partition warning threshold is set to 4096 (MB). WARN [CompactionExecutor:43180] 2017-06-14 20:02:13,189 BigTableWriter.java:184 - Writing large partition