[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative
[
https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216154#comment-16216154
]
Jon Haddad commented on CASSANDRA-13853:
It would be great to have a list of all the keyspace in the cluster along with
each one's replication configuration.
By version I mean an output that's the same as showing the schema version.
Like this:
{code}
Database versions:
3.11.0: [127.0.0.3]
3.11.1: [127.0.0.1, 127.0.0.2]
{code}
You can find the release version in Gossip, take a look at
{{org.apache.cassandra.gms.ApplicationState}} and
{{org.apache.cassandra.gms.Gossiper#getReleaseVersion}}.
> nodetool describecluster should be more informative
> ---
>
> Key: CASSANDRA-13853
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13853
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability
>Reporter: Jon Haddad
>Assignee: Preetika Tyagi
> Labels: lhf
>
> Additional information we should be displaying:
> * Total node count
> * List of datacenters, RF, with number of nodes per dc, how many are down,
> * Version(s)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216144#comment-16216144 ] Preetika Tyagi commented on CASSANDRA-13853: Jon Haddad - Can you please explain what RF info you are looking for? RF can vary by keyspaces and it doesn't seem to directly related to DC or Cluster info. Can you elaborate? Also, do you mean Cassandra version by Version(s)? > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Jon Haddad >Assignee: Preetika Tyagi > Labels: lhf > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216074#comment-16216074
]
Jeff Jirsa edited comment on CASSANDRA-13973 at 10/24/17 12:03 AM:
---
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it, so *this is
not a recommendation* , only a pointer to a config option that exists). In
fact, CASSANDRA-5454 notes that this used to cause corruption, so I'm
definitely not saying you should do this, though it's possible that
CASSANDRA-5454 makes this safe to do.
(FWIW, on the index summary, we downsample if we get over 2GB - see
CASSANDRA-12014 - and I think instead of changing this to a long, we should
consider that here as well, until CASSANDRA-9754 is done, which makes this a
moot point).
was (Author: jjirsa):
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it, so *this is
not a recommendation* , only a pointer to a config option that exists). In
fact, CASSANDRA-5454 notes that this used to cause corruption, so I'm
definitely not saying you should do this, though it's possible that
CASSANDRA-5454 makes this safe to do.
(FWIW, on the index summary, we downsample if we get over 2GB - see
CASSANDRA-12014 - and I think instead of changing this to a long, we should
consider that here as well, until CASSANDRA-9754 is done, which makes this a
moot point).
[~krummas] / [~bdeggleston] , what do you guys think about the safety of this,
especially re: streaming?
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
[jira] [Comment Edited] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216074#comment-16216074
]
Jeff Jirsa edited comment on CASSANDRA-13973 at 10/23/17 11:59 PM:
---
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it, so *this is
not a recommendation* , only a pointer to a config option that exists). In
fact, CASSANDRA-5454 notes that this used to cause corruption, so I'm
definitely not saying you should do this, though it's possible that
CASSANDRA-5454 makes this safe to do.
(FWIW, on the index summary, we downsample if we get over 2GB - see
CASSANDRA-12014 - and I think instead of changing this to a long, we should
consider that here as well, until CASSANDRA-9754 is done, which makes this a
moot point).
[~krummas] / [~bdeggleston] , what do you guys think about the safety of this,
especially re: streaming?
was (Author: jjirsa):
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it, so *this is
not a recommendation* , only a pointer to a config option that exists). In
fact, CASSANDRA-5454 notes that this used to cause corruption, so I'm
definitely not saying you should do this, though it's possible that
CASSANDRA-5454 makes this safe to do.
(FWIW, on the index summary, we downsample if we get over 2GB - see
CASSANDRA-12014 - and I think instead of changing this to a long, we should
consider that here as well, until CASSANDRA-9754 is done, which makes this a
moot point)
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
[jira] [Comment Edited] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216074#comment-16216074
]
Jeff Jirsa edited comment on CASSANDRA-13973 at 10/23/17 11:47 PM:
---
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it, so *this is
not a recommendation* , only a pointer to a config option that exists). In
fact, CASSANDRA-5454 notes that this used to cause corruption, so I'm
definitely not saying you should do this, though it's possible that
CASSANDRA-5454 makes this safe to do.
(FWIW, on the index summary, we downsample if we get over 2GB - see
CASSANDRA-12014 - and I think instead of changing this to a long, we should
consider that here as well, until CASSANDRA-9754 is done, which makes this a
moot point)
was (Author: jjirsa):
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it, so *this is
not a recommendation* , only a pointer to a config option that exists).
(FWIW, on the index summary, we downsample if we get over 2GB - see
CASSANDRA-12014 - and I think instead of changing this to a long, we should
consider that here as well, until CASSANDRA-9754 is done, which makes this a
moot point)
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216074#comment-16216074
]
Jeff Jirsa edited comment on CASSANDRA-13973 at 10/23/17 11:46 PM:
---
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it, so *this is
not a recommendation* , only a pointer to a config option that exists).
(FWIW, on the index summary, we downsample if we get over 2GB - see
CASSANDRA-12014 - and I think instead of changing this to a long, we should
consider that here as well, until CASSANDRA-9754 is done, which makes this a
moot point)
was (Author: jjirsa):
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it, so *this is
not a recommendation* , only a pointer to a config option that exists).
(FWIW, on the index summary, we downsample if we get over 2GB, and I think
instead of changing this to a long, we should consider that here as well, until
CASSANDRA-9754 is done, which makes this a moot point)
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216074#comment-16216074
]
Jeff Jirsa edited comment on CASSANDRA-13973 at 10/23/17 11:45 PM:
---
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it, so *this is
not a recommendation* , only a pointer to a config option that exists).
(FWIW, on the index summary, we downsample if we get over 2GB, and I think
instead of changing this to a long, we should consider that here as well, until
CASSANDRA-9754 is done, which makes this a moot point)
was (Author: jjirsa):
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it, so *this is
not a recommendation* , only a pointer to a config option that exists).
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216074#comment-16216074
]
Jeff Jirsa edited comment on CASSANDRA-13973 at 10/23/17 11:42 PM:
---
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it, so *this is
not a recommendation* , only a pointer to a config option that exists).
was (Author: jjirsa):
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it).
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216074#comment-16216074
]
Jeff Jirsa commented on CASSANDRA-13973:
Those knobs are actually for a slightly different index (the partition summary
itself). Though in yaml / {{Config.java}} there's {{column_index_size_in_kb}}
(defaults to 64), which I'm *pretty sure* does this, but not sure enough to
tell you to change it without testing in a lab (in particular, I don't know
exactly how it behaves if you change this with pre-existing data; I'd hope it'd
do the right thing, but I certainly haven't personally tested it).
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216043#comment-16216043
]
Dan Kinder commented on CASSANDRA-13973:
I see... messing with min/max_index_interval wouldn't help would it?
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216031#comment-16216031
]
Jeff Jirsa commented on CASSANDRA-13973:
OK, leaving some quick notes for whoever gets around to handling this (maybe
me, not self assigning because I don't have bandwidth right now, and to be
honest I haven't thought about the right fix yet).
The code here is trying to calculate the serialized size, so it can write the
index out as it rewrites that partition to a data file :
{code}
long size = TypeSizes.sizeofUnsignedVInt(headerLength)
+ DeletionTime.serializer.serializedSize(deletionTime)
+ TypeSizes.sizeofUnsignedVInt(columnsIndex.size()); //
number of entries
for (IndexHelper.IndexInfo info : columnsIndex)
size += idxSerializer.serializedSize(info);
size += columnsIndex.size() * TypeSizes.sizeof(0);
return Ints.checkedCast(size);
{code}
With 394GB and an index entry every 64k, you're going to write something like
{{6169617}} index markers, and the field there to handle it is a (signed)
integer (4 bytes), giving you a maximum size for all of the index markers of
{{2147483648}} , about 348 bytes per marker. The size of a marker is here:
{code}
long size = clusteringSerializer.serializedSize(info.firstName)
+ clusteringSerializer.serializedSize(info.lastName)
+ TypeSizes.sizeofUnsignedVInt(info.offset)
+ TypeSizes.sizeofVInt(info.width - WIDTH_BASE)
+ TypeSizes.sizeof(info.endOpenMarker != null);
if (info.endOpenMarker != null)
size +=
DeletionTime.serializer.serializedSize(info.endOpenMarker);
return size;
{code}
Note that it has both the first and last clustering within that marker - so for
you not to overflow, assuming no range tombstones which would take up even more
space, your clustering markers would have to average less than ~165 bytes each,
which clearly isn't happening, so we overflow that int and stop.
That's the short version of what's happening. I'm not sure why it's an {{int}}
instead of a {{long}} , and I'm not immediately sure why you're hitting it here
with {{upgradesstables}} when you didn't hit it previously.
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
[jira] [Commented] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216010#comment-16216010
]
Dan Kinder commented on CASSANDRA-13973:
The partition key is a string, and there are 4 clustering columns (text, text,
text, timestamp). The partition key and two of the clustering columns are on
average small, on the order of 10 bytes. The third could be larger but have a
maximum of 2048 bytes.
Also, I don't recall running upgradesstables before, if that matters.
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13291) Replace usages of MessageDigest with Guava's Hasher
[
https://issues.apache.org/jira/browse/CASSANDRA-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216001#comment-16216001
]
Michael Kjellman commented on CASSANDRA-13291:
--
[~beobal] The profiling I did was with MurmurPartitioner so RP was completely
out of the picture.
I do see the usage of {{MD5Digest}} in CASSANDRA-10786, but that was just added
so I think that's why I missed it... given that it's a static final reference
to an object that includes an MD5, I'm not exactly sure how to make this play
nicely with a rolling upgrade in the future "change to something not MD5 digest
ticket".
The goal of this ticket (and the follow up one once we get this fun refactoring
out of the way) was do deal with server-to-server communication... not anything
that went thru the native transport and was client-to-server. Looking at how
CASSANDRA-10786 was implemented, we would need to bump the native transport
protocol version as MD5 is assumed and it's part of the protocol at this point,
so I think we could deal with this in a 3rd ticket to deal with straggler MD5
usages like that.
Another usage of the TL is {{MD5Digest#compute(String toHash)}}, which is only
used by {{QueryProcessorr#computeId(String queryString, String keyspace)}}... I
think again this could be taken care of in a separate client native transport
MD5 ticket.. so tl;dr: the explicit goal of this ticket (and the smaller follow
up one to switch to murmur or something else once we've profiled it) was to
deal with all usages of MD5 in regards to server-to-server communication -- and
not anything todo with client MD5 usage at this time; although clearly we would
want to address that usage in the future too.
Re: {{GuidGenerator}} I'm personally preferential to leaving that as a utility
class vs. pulling it all into RandomPartitioner... I know that it's only used
by {{RandomPartitioner}} but it's generic enough and there is no real cost and
I think personally it's cleaner to have it as a utility class...
I pushed up a small change to revert the change of switching
{{ResultSet#computeResultMetadataId}} (just very recently added as part of
CASSANDRA-10786) to Hasher and put a comment in.
> Replace usages of MessageDigest with Guava's Hasher
> ---
>
> Key: CASSANDRA-13291
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13291
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
> Attachments: CASSANDRA-13291-trunk.diff
>
>
> During my profiling of C* I frequently see lots of aggregate time across
> threads being spent inside the MD5 MessageDigest implementation. Given that
> there are tons of modern alternative hashing functions better than MD5
> available -- both in terms of providing better collision resistance and
> actual computational speed -- I wanted to switch out our usage of MD5 for
> alternatives (like adler128 or murmur3_128) and test for performance
> improvements.
> Unfortunately, I found given the fact we use MessageDigest everywhere --
> switching out the hashing function to something like adler128 or murmur3_128
> (for example) -- which don't ship with the JDK -- wasn't straight forward.
> The goal of this ticket is to propose switching out usages of MessageDigest
> directly in favor of Hasher from Guava. This means going forward we can
> change a single line of code to switch the hashing algorithm being used
> (assuming there is an implementation in Guava).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215992#comment-16215992
]
Jeff Jirsa commented on CASSANDRA-13973:
Ok, so a 394GB row on a 5TB table. Can you tell me just a little bit about your
clustering columns - how many and how long are they typically? I'm pretty sure
I see what the issue is, but I haven't yet checked the difference against 2.2
to see why it's only a problem now, and wasn't before.
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215984#comment-16215984
]
Dan Kinder commented on CASSANDRA-13973:
[~jjirsa]
This happens on about 12 out of 16 nodes, much greater than {{RF}}. (No real
need to anonymize the names.)
{noformat}
$ nodetool tablehistograms walker links
walker/links histograms
Percentile SSTables Write Latency Read LatencyPartition Size
Cell Count
(micros) (micros) (bytes)
50% 4.00 51.011155149.91 258
6
75% 5.00 73.463449259.15 1597
42
95% 5.00152.327152383.77 73457
2299
98% 5.00182.798582860.53315852
9887
99% 5.00219.34 53142810.15943127
24601
Min 0.00 8.24 88.1543
0
Max 5.001155149.91 53142810.15 394855527800
962624926
$ nodetool tablestats walker.links
Keyspace: walker
Read Count: 72932
Read Latency: 834.917616670323 ms.
Write Count: 15501219
Write Latency: 0.4392410533003888 ms.
Pending Flushes: 0
Table: links
SSTable count: 14269
SSTables in each level: [0, 9, 68, 349, 2843, 11000, 0, 0, 0]
Space used (live): 5310634842695
Space used (total): 5310634842695
Space used by snapshots (total): 12868175419329
Off heap memory used (total): 16339462
SSTable Compression Ratio: 0.0
Number of partitions (estimate): 20112302
Memtable cell count: 2216835
Memtable data size: 347672840
Memtable off heap memory used: 0
Memtable switch count: 4
Local read count: 80395
Local read latency: 747.603 ms
Local write count: 16891401
Local write latency: 0.118 ms
Pending flushes: 0
Bloom filter false positives: 178
Bloom filter false ratio: 0.00081
Bloom filter space used: 14912616
Bloom filter off heap memory used: 14798464
Index summary off heap memory used: 1540998
Compression metadata off heap memory used: 0
Compacted partition minimum bytes: 43
Compacted partition maximum bytes: 394855527800
Compacted partition mean bytes: 249068
Average live cells per slice (last five minutes):
11010.451219512195
Maximum live cells per slice (last five minutes): 11864
Average tombstones per slice (last five minutes): 1.0
Maximum tombstones per slice (last five minutes): 1
{noformat}
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
>
[jira] [Updated] (CASSANDRA-13944) Throw descriptive errors for mixed mode repair attempts
[ https://issues.apache.org/jira/browse/CASSANDRA-13944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-13944: Reviewer: Marcus Eriksson Status: Patch Available (was: Open) [trunk|https://github.com/bdeggleston/cassandra/tree/13944] [utests|https://circleci.com/gh/bdeggleston/cassandra/135] > Throw descriptive errors for mixed mode repair attempts > --- > > Key: CASSANDRA-13944 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13944 > Project: Cassandra > Issue Type: Bug > Components: Repair >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Minor > Fix For: 4.0 > > > We often make breaking changes to streaming and repair between major > versions, and don't usually support either in mixed mode clusters. Streaming > connections check protocol versions, but repair message handling doesn't, > which means cryptic exceptions show up in the logs when operators forget to > turn off whatever's scheduling repairs on their cluster. Refusing to send or > receive repair messages to/ from incompatible messaging service versions, and > throwing a descriptive exception would make it clearer why repair is not > working, as well as prevent any potentially unexpected behavior. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13508) Make system.paxos table compaction strategy configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215827#comment-16215827
]
Jeff Jirsa commented on CASSANDRA-13508:
That's an interesting idea. In the linked CASSANDRA-13548 , I noted that the
current partition key excludes the CFID, which causes write amplification when
system.paxos is LCS. With a paxos-table-per-table, that problem would be
eliminated.
> Make system.paxos table compaction strategy configurable
>
>
> Key: CASSANDRA-13508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13508
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
> Labels: core, paxos
> Fix For: 4.0, 4.x
>
> Attachments: test11.png, test2.png
>
>
> The default compaction strategy for {{system.paxos}} table is LCS for
> performance reason: CASSANDRA-7753. But for CAS heavily used cluster, the
> system is busy with {{system.paxos}} compaction.
> As the data in {{paxos}} table are TTL'ed, TWCS might be a better fit. In our
> test, it significantly reduced the number of compaction without impacting the
> latency too much:
> !test11.png!
> The time window for TWCS is set to 2 minutes for the test.
> Here is the p99 latency impact:
> !test2.png!
> the yellow one is LCS, the purple one is TWCS. Average p99 has about 10%
> increase.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13508) Make system.paxos table compaction strategy configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215821#comment-16215821
]
Blake Eggleston commented on CASSANDRA-13508:
-
Coming back to this, I think making {{system.paxos}} configurable might be the
wrong approach. The system keyspaces typically store system metadata, where the
data and query volumes are low enough that the table configuration shouldn’t be
a concern for operators. The paxos table (and the batch log, but let's stick to
the paxos table for now) is directly involved in the processing of queries. In
applications that are heavy cas users, it can be the most heavily used table in
the cluster.
Maybe it would be better to treat these tables more like index tables, where
each table involved in paxos operations gets it’s own local sidecar table? We
could either then make the paxos table inherit the compaction settings from
it’s parent table, or we could enable separately tuning the paxos table with a
{{paxos_compaction}} schema option or something.
Any thoughts on this?
> Make system.paxos table compaction strategy configurable
>
>
> Key: CASSANDRA-13508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13508
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
> Labels: core, paxos
> Fix For: 4.0, 4.x
>
> Attachments: test11.png, test2.png
>
>
> The default compaction strategy for {{system.paxos}} table is LCS for
> performance reason: CASSANDRA-7753. But for CAS heavily used cluster, the
> system is busy with {{system.paxos}} compaction.
> As the data in {{paxos}} table are TTL'ed, TWCS might be a better fit. In our
> test, it significantly reduced the number of compaction without impacting the
> latency too much:
> !test11.png!
> The time window for TWCS is set to 2 minutes for the test.
> Here is the p99 latency impact:
> !test2.png!
> the yellow one is LCS, the purple one is TWCS. Average p99 has about 10%
> increase.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215703#comment-16215703
]
Jeff Jirsa commented on CASSANDRA-13973:
Is this happening on exactly 3 (or {{RF}}) nodes, or more than that?
Can you paste (anonymized) {{nodetool tablestats}} or {{nodetool
tablehistograms}} for that table?
> IllegalArgumentException in upgradesstables compaction
> --
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
Dan Kinder created CASSANDRA-13973:
--
Summary: IllegalArgumentException in upgradesstables compaction
Key: CASSANDRA-13973
URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
Project: Cassandra
Issue Type: Bug
Components: Compaction
Reporter: Dan Kinder
After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try to
run upgradesstables, most of them upgrade fine but I see the exception below on
several nodes, and it doesn't complete.
CASSANDRA-12717 looks similar but the stack trace is not the same, so I assumed
it is not identical. The various nodes this happens on all give the same trace.
Might be notable that this is an analytics cluster with some large partitions,
in the GB size.
{noformat}
error: Out of range: 7316844981
-- StackTrace --
java.lang.IllegalArgumentException: Out of range: 7316844981
at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
at
org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
at
org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
at
org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
at
org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
at
org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
at
org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
at
org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
at java.lang.Thread.run(Thread.java:748)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10857) Allow dropping COMPACT STORAGE flag from tables in 3.X
[
https://issues.apache.org/jira/browse/CASSANDRA-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-10857:
--
Labels: client-impacting (was: )
> Allow dropping COMPACT STORAGE flag from tables in 3.X
> --
>
> Key: CASSANDRA-10857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10857
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Blocker
> Labels: client-impacting
> Fix For: 3.0.x, 3.11.x
>
>
> Thrift allows users to define flexible mixed column families - where certain
> columns would have explicitly pre-defined names, potentially non-default
> validation types, and be indexed.
> Example:
> {code}
> create column family foo
> and default_validation_class = UTF8Type
> and column_metadata = [
> {column_name: bar, validation_class: Int32Type, index_type: KEYS},
> {column_name: baz, validation_class: UUIDType, index_type: KEYS}
> ];
> {code}
> Columns named {{bar}} and {{baz}} will be validated as {{Int32Type}} and
> {{UUIDType}}, respectively, and be indexed. Columns with any other name will
> be validated by {{UTF8Type}} and will not be indexed.
> With CASSANDRA-8099, {{bar}} and {{baz}} would be mapped to static columns
> internally. However, being {{WITH COMPACT STORAGE}}, the table will only
> expose {{bar}} and {{baz}} columns. Accessing any dynamic columns (any column
> not named {{bar}} and {{baz}}) right now requires going through Thrift.
> This is blocking Thrift -> CQL migration for users who have mixed
> dynamic/static column families. That said, it *shouldn't* be hard to allow
> users to drop the {{compact}} flag to expose the table as it is internally
> now, and be able to access all columns.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13952) C* still sending WriteFailure instead of CDCWriteFailure
[
https://issues.apache.org/jira/browse/CASSANDRA-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215516#comment-16215516
]
Jaume M commented on CASSANDRA-13952:
-
Yes, that's what's happening, I'll close this ticket and the PR then.
> C* still sending WriteFailure instead of CDCWriteFailure
>
>
> Key: CASSANDRA-13952
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13952
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jaume M
>Assignee: Jaume M
> Fix For: 4.0
>
>
> I've tested this setting setting cdc_total_space_in_mb: 1 and with the
> current python driver master
> {code}
> from cassandra.cluster import Cluster
> cluster = Cluster(protocol_version=5, allow_beta_protocol_version=True)
> session = cluster.connect()
> session.execute("""
> CREATE KEYSPACE IF NOT EXISTS %s
> WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '2'
> }
> """ % "query_per_range")
> session.execute("""
> CREATE TABLE IF NOT EXISTS query_per_range.mytable_int (
> key_one int,
> key_two int,
> col1 text,
> col2 text,
> PRIMARY KEY ((key_one, key_two), col1)
> ) WITH cdc=true;
> """)
> for i in range(1000):
> session.execute("INSERT INTO query_per_range.mytable_int(key_one,
> key_two, col1, col2) VALUES (%s, %s, %s, %s)",
> (i, i, str(i), str(i)))
> {code}
> I'm receiving a {{WriteFailure}} every time but I was expecting a
> {{CDCWriteFailure}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13952) C* still sending WriteFailure instead of CDCWriteFailure
[
https://issues.apache.org/jira/browse/CASSANDRA-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jaume M updated CASSANDRA-13952:
Resolution: Not A Problem
Status: Resolved (was: Patch Available)
> C* still sending WriteFailure instead of CDCWriteFailure
>
>
> Key: CASSANDRA-13952
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13952
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jaume M
>Assignee: Jaume M
> Fix For: 4.0
>
>
> I've tested this setting setting cdc_total_space_in_mb: 1 and with the
> current python driver master
> {code}
> from cassandra.cluster import Cluster
> cluster = Cluster(protocol_version=5, allow_beta_protocol_version=True)
> session = cluster.connect()
> session.execute("""
> CREATE KEYSPACE IF NOT EXISTS %s
> WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '2'
> }
> """ % "query_per_range")
> session.execute("""
> CREATE TABLE IF NOT EXISTS query_per_range.mytable_int (
> key_one int,
> key_two int,
> col1 text,
> col2 text,
> PRIMARY KEY ((key_one, key_two), col1)
> ) WITH cdc=true;
> """)
> for i in range(1000):
> session.execute("INSERT INTO query_per_range.mytable_int(key_one,
> key_two, col1, col2) VALUES (%s, %s, %s, %s)",
> (i, i, str(i), str(i)))
> {code}
> I'm receiving a {{WriteFailure}} every time but I was expecting a
> {{CDCWriteFailure}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13952) C* still sending WriteFailure instead of CDCWriteFailure
[
https://issues.apache.org/jira/browse/CASSANDRA-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215473#comment-16215473
]
Joshua McKenzie commented on CASSANDRA-13952:
-
bq. are you ok with this returning WriteFailureException with a reason code
(the way it currently works),
I am, yes.
> C* still sending WriteFailure instead of CDCWriteFailure
>
>
> Key: CASSANDRA-13952
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13952
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jaume M
>Assignee: Jaume M
> Fix For: 4.0
>
>
> I've tested this setting setting cdc_total_space_in_mb: 1 and with the
> current python driver master
> {code}
> from cassandra.cluster import Cluster
> cluster = Cluster(protocol_version=5, allow_beta_protocol_version=True)
> session = cluster.connect()
> session.execute("""
> CREATE KEYSPACE IF NOT EXISTS %s
> WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '2'
> }
> """ % "query_per_range")
> session.execute("""
> CREATE TABLE IF NOT EXISTS query_per_range.mytable_int (
> key_one int,
> key_two int,
> col1 text,
> col2 text,
> PRIMARY KEY ((key_one, key_two), col1)
> ) WITH cdc=true;
> """)
> for i in range(1000):
> session.execute("INSERT INTO query_per_range.mytable_int(key_one,
> key_two, col1, col2) VALUES (%s, %s, %s, %s)",
> (i, i, str(i), str(i)))
> {code}
> I'm receiving a {{WriteFailure}} every time but I was expecting a
> {{CDCWriteFailure}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13952) C* still sending WriteFailure instead of CDCWriteFailure
[
https://issues.apache.org/jira/browse/CASSANDRA-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215412#comment-16215412
]
Andy Tolbert commented on CASSANDRA-13952:
--
Just to clarify your last comment, are you ok with this returning
{{WriteFailureException}} with a reason code (the way it currently works), or
do you want there to be a new error type?
I prefer the former since it seems to be a good fit and doesn't require any
client changes.
> C* still sending WriteFailure instead of CDCWriteFailure
>
>
> Key: CASSANDRA-13952
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13952
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jaume M
>Assignee: Jaume M
> Fix For: 4.0
>
>
> I've tested this setting setting cdc_total_space_in_mb: 1 and with the
> current python driver master
> {code}
> from cassandra.cluster import Cluster
> cluster = Cluster(protocol_version=5, allow_beta_protocol_version=True)
> session = cluster.connect()
> session.execute("""
> CREATE KEYSPACE IF NOT EXISTS %s
> WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '2'
> }
> """ % "query_per_range")
> session.execute("""
> CREATE TABLE IF NOT EXISTS query_per_range.mytable_int (
> key_one int,
> key_two int,
> col1 text,
> col2 text,
> PRIMARY KEY ((key_one, key_two), col1)
> ) WITH cdc=true;
> """)
> for i in range(1000):
> session.execute("INSERT INTO query_per_range.mytable_int(key_one,
> key_two, col1, col2) VALUES (%s, %s, %s, %s)",
> (i, i, str(i), str(i)))
> {code}
> I'm receiving a {{WriteFailure}} every time but I was expecting a
> {{CDCWriteFailure}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13087) Not enough bytes exception during compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anonymous updated CASSANDRA-13087:
--
Status: In Progress (was: Ready to Commit)
> Not enough bytes exception during compaction
>
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
>Assignee: Cyril Scetbon
> Attachments: CASSANDRA-13087.patch
>
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245
> CassandraDaemon.java:229 - Exception in thread
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length:
> 20275. Buffer size: 12594
> at
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
> ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> ~[guava-16.0.jar:na]
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> ~[guava-16.0.jar:na]
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> ~[guava-16.0.jar:na]
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> ~[guava-16.0.jar:na]
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
> ~[guava-16.0.jar:na]
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> ~[guava-16.0.jar:na]
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> ~[guava-16.0.jar:na]
> at
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
> ~[apache-cassandra-2.1.14.jar:2
> .1.14]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_60]
>
[jira] [Updated] (CASSANDRA-13087) Not enough bytes exception during compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anonymous updated CASSANDRA-13087:
--
Status: Ready to Commit (was: Patch Available)
> Not enough bytes exception during compaction
>
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
>Assignee: Cyril Scetbon
> Attachments: CASSANDRA-13087.patch
>
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245
> CassandraDaemon.java:229 - Exception in thread
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length:
> 20275. Buffer size: 12594
> at
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
> ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> ~[guava-16.0.jar:na]
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> ~[guava-16.0.jar:na]
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> ~[guava-16.0.jar:na]
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> ~[guava-16.0.jar:na]
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
> ~[guava-16.0.jar:na]
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> ~[guava-16.0.jar:na]
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> ~[guava-16.0.jar:na]
> at
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
> ~[apache-cassandra-2.1.14.jar:2
> .1.14]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_60]
>
[jira] [Commented] (CASSANDRA-13442) Support a means of strongly consistent highly available replication with tunable storage requirements
[ https://issues.apache.org/jira/browse/CASSANDRA-13442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215271#comment-16215271 ] Benedict commented on CASSANDRA-13442: -- This is a cool feature idea, so I hope you don’t mind my 2c. I’m generally +1 the idea, if done carefully. Honestly, it makes so much sense that it should probably outright replace the standard approach for multi-DC clusters. It might cause less consternation if we rebranded 3-1 as 2+1. In reality, the “transient” replica is anything but a replica, it’s more of a temporary write log that must be consulted on each read/write that cannot communicate with sufficient actual replicas. If instead of viewing this as “deleting one RF when safe to do so” - which quite reasonably gives people the heebyjeebies - we instead approach it as an availability optimisation to (e.g.) RF=2. Modelled this way, we should be able to make the changes incrementally, optimising each path while maintaining a lower-bound behaviour of RF=2 at CL.ALL This was also a more intuitive mental model for me, but in doing so I seem to have come up against some implied inconsistencies with things that have been said: bq. With cheap quorums read at ONE and write at ALL works as you would expect. What won't work as you would expect is read at ONE and write at something less. I’m confused by this; how does reading at ONE differ in these scenarios? It seems that with or without “cheap” quorum, writing at N-1 or less, and reading at, 1 provides no guarantees. bq. Transient replicas based on incremental repair aren't going to work with 2i and MV because there is no efficient way to anti-compact them since they aren't stored in token order. Is this true? The transient nodes cannot anyway have their own paired MV, since they cannot generate MV deltas. So all that seems necessary is that, on repair of the base table, we generate updates for our paired view - as we must already if I’m not mistaken…? The real problem seems that, given the above, we cannot easily support - in either proposed implementation - the "availability optimisations" to reads to a 2i or MV. At least, not without routing the update through at least one full replica before reaching the transient replica. If (one day) we endorse my ancient proposal to require the coordinator role be fulfilled by an owning replica, for all non-batch queries, we might be able to support this. But I think documenting that the availability optimisation is only applied to the base table is an acceptable behaviour for a first release. bq. RF=3 with 3 full replicas seeking strong consistency with quorum I’m also a little confused by the use of “strong consistency” - which _seems_ to implicitly mean “quorum read/write” - which isn’t strongly consistent… if we mean CAS/LWT in these instances, it would be helpful to clarify bq. Since for CL.ONE queries you would need to only use one of the replicas with all the data on it. bq. Yes, I think there are going to be multiple places where this gets more complicated than it looks at first. This is actually fairly consistent, if viewed through a 2+1 lens: *Every* operation must involve at least one full replica. But any collection of N full local replicas serving at least one non-digest can be safely boosted to N+1 by the inclusion of a data request to a local transient replica. To add some opinions to the implementation debate, it seems to me: * Less work it probably needed for an sstable based approach. If initially we only support Cheap Quorum, we could unconditionally stream sstables from transient replicas to all full local replicas, before immediately dropping them ** No risk of any weird algorithmic problems with this approach, and we could take our time engineering a safe and robust _optional_ full quorum if we decide its useful ** This could then quickly be optimised to e.g. separate by originating node so that reads and repairs only need to consult the data not sent by themselves * As far as the token ring is concerned, the N+M proposal implies not treating the +M nodes as full members of the ring - we should have a secondary/transient/shadow ring, to guarantee we don’t accidentally treat any transient nodes as full replicas > Support a means of strongly consistent highly available replication with > tunable storage requirements > - > > Key: CASSANDRA-13442 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13442 > Project: Cassandra > Issue Type: Improvement > Components: Compaction, Coordination, Distributed Metadata, Local > Write-Read Paths >Reporter: Ariel Weisberg > > Replication factors like RF=2 can't provide strong consistency and > availability because if a single
[jira] [Updated] (CASSANDRA-13849) GossipStage blocks because of race in ActiveRepairService
[
https://issues.apache.org/jira/browse/CASSANDRA-13849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Lapukhov updated CASSANDRA-13849:
Labels: patch (was: )
Status: Patch Available (was: Open)
Idea is to replace intrinsic lock (synchronized) everywhere in
ActiveRepairService with ReentrantLock, and release it before start waiting on
prepareLatch in prepareForRepair.
> GossipStage blocks because of race in ActiveRepairService
> -
>
> Key: CASSANDRA-13849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13849
> Project: Cassandra
> Issue Type: Bug
>Reporter: Tom van der Woerdt
> Labels: patch
> Attachments: CAS-13849.patch
>
>
> Bad luck caused a kernel panic in a cluster, and that took another node with
> it because GossipStage stopped responding.
> I think it's pretty obvious what's happening, here are the relevant excerpts
> from the stack traces :
> {noformat}
> "Thread-24004" #393781 daemon prio=5 os_prio=0 tid=0x7efca9647400
> nid=0xe75c waiting on condition [0x7efaa47fe000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00052b63a7e8> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at
> org.apache.cassandra.service.ActiveRepairService.prepareForRepair(ActiveRepairService.java:332)
> - locked <0x0002e6bc99f0> (a
> org.apache.cassandra.service.ActiveRepairService)
> at
> org.apache.cassandra.repair.RepairRunnable.runMayThrow(RepairRunnable.java:211)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>
> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$3/1498438472.run(Unknown
> Source)
> at java.lang.Thread.run(Thread.java:748)
> "GossipTasks:1" #367 daemon prio=5 os_prio=0 tid=0x7efc5e971000
> nid=0x700b waiting for monitor entry [0x7dfb839fe000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:421)
> - waiting to lock <0x0002e6bc99f0> (a
> org.apache.cassandra.service.ActiveRepairService)
> at
> org.apache.cassandra.service.ActiveRepairService.convict(ActiveRepairService.java:776)
> at
> org.apache.cassandra.gms.FailureDetector.interpret(FailureDetector.java:306)
> at org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:775)
>
> at
> org.apache.cassandra.gms.Gossiper.access$800(Gossiper.java:67)
> at org.apache.cassandra.gms.Gossiper$GossipTask.run(Gossiper.java:187)
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$3/1498438472.run(Unknown
> Source)
> at java.lang.Thread.run(Thread.java:748)
> "GossipStage:1" #320 daemon prio=5 os_prio=0 tid=0x7efc5b9f2c00
> nid=0x6fcd waiting for monitor entry [0x7e260186a000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
>
[jira] [Updated] (CASSANDRA-13849) GossipStage blocks because of race in ActiveRepairService
[
https://issues.apache.org/jira/browse/CASSANDRA-13849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Lapukhov updated CASSANDRA-13849:
Attachment: CAS-13849.patch
Proposed patch for the CAS-13849
> GossipStage blocks because of race in ActiveRepairService
> -
>
> Key: CASSANDRA-13849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13849
> Project: Cassandra
> Issue Type: Bug
>Reporter: Tom van der Woerdt
> Labels: patch
> Attachments: CAS-13849.patch
>
>
> Bad luck caused a kernel panic in a cluster, and that took another node with
> it because GossipStage stopped responding.
> I think it's pretty obvious what's happening, here are the relevant excerpts
> from the stack traces :
> {noformat}
> "Thread-24004" #393781 daemon prio=5 os_prio=0 tid=0x7efca9647400
> nid=0xe75c waiting on condition [0x7efaa47fe000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00052b63a7e8> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at
> org.apache.cassandra.service.ActiveRepairService.prepareForRepair(ActiveRepairService.java:332)
> - locked <0x0002e6bc99f0> (a
> org.apache.cassandra.service.ActiveRepairService)
> at
> org.apache.cassandra.repair.RepairRunnable.runMayThrow(RepairRunnable.java:211)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>
> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$3/1498438472.run(Unknown
> Source)
> at java.lang.Thread.run(Thread.java:748)
> "GossipTasks:1" #367 daemon prio=5 os_prio=0 tid=0x7efc5e971000
> nid=0x700b waiting for monitor entry [0x7dfb839fe000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:421)
> - waiting to lock <0x0002e6bc99f0> (a
> org.apache.cassandra.service.ActiveRepairService)
> at
> org.apache.cassandra.service.ActiveRepairService.convict(ActiveRepairService.java:776)
> at
> org.apache.cassandra.gms.FailureDetector.interpret(FailureDetector.java:306)
> at org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:775)
>
> at
> org.apache.cassandra.gms.Gossiper.access$800(Gossiper.java:67)
> at org.apache.cassandra.gms.Gossiper$GossipTask.run(Gossiper.java:187)
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$3/1498438472.run(Unknown
> Source)
> at java.lang.Thread.run(Thread.java:748)
> "GossipStage:1" #320 daemon prio=5 os_prio=0 tid=0x7efc5b9f2c00
> nid=0x6fcd waiting for monitor entry [0x7e260186a000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:421)
> - waiting to lock <0x0002e6bc99f0> (a
> org.apache.cassandra.service.ActiveRepairService)
>
[jira] [Comment Edited] (CASSANDRA-10857) Allow dropping COMPACT STORAGE flag from tables in 3.X
[
https://issues.apache.org/jira/browse/CASSANDRA-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16214780#comment-16214780
]
Alex Petrov edited comment on CASSANDRA-10857 at 10/23/17 7:56 AM:
---
Thank you for the review. Sorry for such long turnaround.
bq. The first commit has various nits, including some around making language a
bit more consistent.
Great. I did have the same idea; however at time {{Schema}} sounded like a more
logical place for it (mostly because {{trunk}} has this method there). But I
agree it makes the diff smaller, so it's good.
bq. In the parser, I was a bit worried about allowing empty strings in
QUOTED_NAME directly,
Completely agreed. I had the same idea, but my impression was since it's still
possible to programmatically create empty names, it's worth to have validations
anyways, so I went with the current approach. I agree this change might be
safer though...
bq. So I added it in SchemaAlteringStatement directly (and I've let changes on
keyspaces/non compact tables go because no real reason to refuse them). That's
the third commit.
You're right, the way it worked was any alterations that were allowed for
compact tables were still working (there are only few though). But for safety
it's better to disallow them.
I've extended and moved from {{NEWS.txt}} the section describing which columns
get exposed when; now this information is in the CQL appendix as we discussed.
I've changed wording a bit in both news and CQL appendix and got rid of
internal details. The doc changes are only in 3.11 patch as there's no doc
website in 3.0...
|[3.0
patch|https://github.com/apache/cassandra/compare/cassandra-3.0...ifesdjeen:10857-3.0]|[3.11
patch|https://github.com/apache/cassandra/compare/cassandra-3.11...ifesdjeen:10857-3.11]|[Python
Driver
Patch|https://github.com/datastax/python-driver/compare/master...ifesdjeen:10857]|[dtests|https://github.com/apache/cassandra-dtest/compare/master...ifesdjeen:10857]|
was (Author: ifesdjeen):
Thank you for the review. Sorry for such long turnaround.
bq. The first commit has various nits, including some around making language a
bit more consistent.
Great. I did have the same idea; however at time {{Schema}} sounded like a more
logical place for it (mostly because {{trunk}} has this method there). But I
agree it makes the diff smaller, so it's good.
bq. In the parser, I was a bit worried about allowing empty strings in
QUOTED_NAME directly,
Completely agreed. I had the same idea, but my impression was since it's still
possible to programmatically create empty names, it's worth to have validations
anyways, so I went with the current approach. I agree this change might be
safer though...
bq. So I added it in SchemaAlteringStatement directly (and I've let changes on
keyspaces/non compact tables go because no real reason to refuse them). That's
the third commit.
You're right, the way it worked was any alterations that were allowed for
compact tables were still working (there are only few though). But for safety
it's better to disallow them.
I've extended and moved from {{NEWS.txt}} the section describing which columns
get exposed when; now this information is in the CQL appendix as we discussed.
I've changed wording a bit in both news and CQL appendix and got rid of
internal details.
> Allow dropping COMPACT STORAGE flag from tables in 3.X
> --
>
> Key: CASSANDRA-10857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10857
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Blocker
> Fix For: 3.0.x, 3.11.x
>
>
> Thrift allows users to define flexible mixed column families - where certain
> columns would have explicitly pre-defined names, potentially non-default
> validation types, and be indexed.
> Example:
> {code}
> create column family foo
> and default_validation_class = UTF8Type
> and column_metadata = [
> {column_name: bar, validation_class: Int32Type, index_type: KEYS},
> {column_name: baz, validation_class: UUIDType, index_type: KEYS}
> ];
> {code}
> Columns named {{bar}} and {{baz}} will be validated as {{Int32Type}} and
> {{UUIDType}}, respectively, and be indexed. Columns with any other name will
> be validated by {{UTF8Type}} and will not be indexed.
> With CASSANDRA-8099, {{bar}} and {{baz}} would be mapped to static columns
> internally. However, being {{WITH COMPACT STORAGE}}, the table will only
> expose {{bar}} and {{baz}} columns. Accessing any dynamic columns (any column
> not named {{bar}} and {{baz}}) right now requires going through Thrift.
> This is blocking Thrift -> CQL migration for
[jira] [Commented] (CASSANDRA-10857) Allow dropping COMPACT STORAGE flag from tables in 3.X
[
https://issues.apache.org/jira/browse/CASSANDRA-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16214780#comment-16214780
]
Alex Petrov commented on CASSANDRA-10857:
-
Thank you for the review. Sorry for such long turnaround.
bq. The first commit has various nits, including some around making language a
bit more consistent.
Great. I did have the same idea; however at time {{Schema}} sounded like a more
logical place for it (mostly because {{trunk}} has this method there). But I
agree it makes the diff smaller, so it's good.
bq. In the parser, I was a bit worried about allowing empty strings in
QUOTED_NAME directly,
Completely agreed. I had the same idea, but my impression was since it's still
possible to programmatically create empty names, it's worth to have validations
anyways, so I went with the current approach. I agree this change might be
safer though...
bq. So I added it in SchemaAlteringStatement directly (and I've let changes on
keyspaces/non compact tables go because no real reason to refuse them). That's
the third commit.
You're right, the way it worked was any alterations that were allowed for
compact tables were still working (there are only few though). But for safety
it's better to disallow them.
I've extended and moved from {{NEWS.txt}} the section describing which columns
get exposed when; now this information is in the CQL appendix as we discussed.
I've changed wording a bit in both news and CQL appendix and got rid of
internal details.
> Allow dropping COMPACT STORAGE flag from tables in 3.X
> --
>
> Key: CASSANDRA-10857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10857
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL, Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Blocker
> Fix For: 3.0.x, 3.11.x
>
>
> Thrift allows users to define flexible mixed column families - where certain
> columns would have explicitly pre-defined names, potentially non-default
> validation types, and be indexed.
> Example:
> {code}
> create column family foo
> and default_validation_class = UTF8Type
> and column_metadata = [
> {column_name: bar, validation_class: Int32Type, index_type: KEYS},
> {column_name: baz, validation_class: UUIDType, index_type: KEYS}
> ];
> {code}
> Columns named {{bar}} and {{baz}} will be validated as {{Int32Type}} and
> {{UUIDType}}, respectively, and be indexed. Columns with any other name will
> be validated by {{UTF8Type}} and will not be indexed.
> With CASSANDRA-8099, {{bar}} and {{baz}} would be mapped to static columns
> internally. However, being {{WITH COMPACT STORAGE}}, the table will only
> expose {{bar}} and {{baz}} columns. Accessing any dynamic columns (any column
> not named {{bar}} and {{baz}}) right now requires going through Thrift.
> This is blocking Thrift -> CQL migration for users who have mixed
> dynamic/static column families. That said, it *shouldn't* be hard to allow
> users to drop the {{compact}} flag to expose the table as it is internally
> now, and be able to access all columns.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
