cassandra git commit: Make it possible to change neverPurgeTombstones during runtime
Repository: cassandra Updated Branches: refs/heads/trunk d0b34d383 -> 0f58f6c65 Make it possible to change neverPurgeTombstones during runtime Patch by marcuse; reviewed by Chris Lohfink for CASSANDRA-14214 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0f58f6c6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0f58f6c6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0f58f6c6 Branch: refs/heads/trunk Commit: 0f58f6c6501518e57ba021cb959b288fd533f472 Parents: d0b34d3 Author: Marcus ErikssonAuthored: Mon Feb 5 09:27:52 2018 +0100 Committer: Marcus Eriksson Committed: Thu Feb 8 08:41:39 2018 +0100 -- CHANGES.txt | 1 + .../apache/cassandra/db/ColumnFamilyStore.java | 19 + .../cassandra/db/ColumnFamilyStoreMBean.java| 4 + .../compaction/AbstractCompactionStrategy.java | 2 +- .../db/compaction/CompactionController.java | 16 ++-- .../db/compaction/CompactionsCQLTest.java | 83 6 files changed, 119 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f58f6c6/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index cadd4c8..191caaf 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0 + * Make it possible to change neverPurgeTombstones during runtime (CASSANDRA-14214) * Remove GossipDigestSynVerbHandler#doSort() (CASSANDRA-14174) * Add nodetool clientlist (CASSANDRA-13665) * Revert ProtocolVersion changes from CASSANDRA-7544 (CASSANDRA-14211) http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f58f6c6/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index c12b474..8e7b220 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -218,6 +218,8 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean @VisibleForTesting final DiskBoundaryManager diskBoundaryManager = new DiskBoundaryManager(); +private volatile boolean neverPurgeTombstones = false; + public static void shutdownPostFlushExecutor() throws InterruptedException { postFlushExecutor.shutdown(); @@ -2636,4 +2638,21 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean { diskBoundaryManager.invalidate(); } + +@Override +public void setNeverPurgeTombstones(boolean value) +{ +if (neverPurgeTombstones != value) +logger.info("Changing neverPurgeTombstones for {}.{} from {} to {}", keyspace.getName(), getTableName(), neverPurgeTombstones, value); +else +logger.info("Not changing neverPurgeTombstones for {}.{}, it is {}", keyspace.getName(), getTableName(), neverPurgeTombstones); + +neverPurgeTombstones = value; +} + +@Override +public boolean getNeverPurgeTombstones() +{ +return neverPurgeTombstones; +} } http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f58f6c6/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java b/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java index e361ffe..bdb842d 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java @@ -195,4 +195,8 @@ public interface ColumnFamilyStoreMBean Enable/Disable compaction space check */ public void compactionDiskSpaceCheck(boolean enable); + +public void setNeverPurgeTombstones(boolean value); + +public boolean getNeverPurgeTombstones(); } http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f58f6c6/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java index e88524f..a43761f 100644 --- a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java +++ b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java @@ -350,7 +350,7 @@ public abstract class AbstractCompactionStrategy */ protected boolean worthDroppingTombstones(SSTableReader sstable, int gcBefore) { -if
[jira] [Updated] (CASSANDRA-14214) Make it possible to change NEVER_PURGE_TOMBSTONES during runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-14214: Resolution: Fixed Status: Resolved (was: Patch Available) Committed as {{0f58f6c6501518e57ba021cb959b288fd533f472}} - thanks! > Make it possible to change NEVER_PURGE_TOMBSTONES during runtime > > > Key: CASSANDRA-14214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14214 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Major > Fix For: 4.x > > > If a node gets behind on compaction it could be good to be able to flip the > NEVER_PURGE_TOMBSTONES flag to speed up compactions during runtime. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14214) Make it possible to change NEVER_PURGE_TOMBSTONES during runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-14214: Reviewer: Chris Lohfink > Make it possible to change NEVER_PURGE_TOMBSTONES during runtime > > > Key: CASSANDRA-14214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14214 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Major > Fix For: 4.x > > > If a node gets behind on compaction it could be good to be able to flip the > NEVER_PURGE_TOMBSTONES flag to speed up compactions during runtime. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14223) Provide ability to do custom certificate validations (e.g. hostname validation, certificate revocation checks)
Ron Blechman created CASSANDRA-14223: Summary: Provide ability to do custom certificate validations (e.g. hostname validation, certificate revocation checks) Key: CASSANDRA-14223 URL: https://issues.apache.org/jira/browse/CASSANDRA-14223 Project: Cassandra Issue Type: Improvement Components: Configuration Reporter: Ron Blechman Fix For: 3.11.x Cassandra server should be to be able do additional certificate validations, such as hostname validatation and certificate revocation checking against CRLs and/or using OCSP. One approach couild be to have SSLFactory use SSLContext.getDefault() instead of forcing the creation of a new SSLContext using SSLContext.getInstance(). Using the default SSLContext would allow a user to plug in their own custom SSLSocketFactory via the java.security properties file. The custom SSLSocketFactory could create a default SSLContext that was customized to do any extra validation such as certificate revocation, host name validation, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-13973: --- Status: Patch Available (was: Awaiting Feedback) > IllegalArgumentException in upgradesstables compaction > -- > > Key: CASSANDRA-13973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13973 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Dan Kinder >Assignee: Dinesh Joshi >Priority: Major > Fix For: 3.0.x, 3.11.x, 4.x > > > After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try > to run upgradesstables, most of them upgrade fine but I see the exception > below on several nodes, and it doesn't complete. > CASSANDRA-12717 looks similar but the stack trace is not the same, so I > assumed it is not identical. The various nodes this happens on all give the > same trace. > Might be notable that this is an analytics cluster with some large > partitions, in the GB size. > {noformat} > error: Out of range: 7316844981 > -- StackTrace -- > java.lang.IllegalArgumentException: Out of range: 7316844981 > at com.google.common.primitives.Ints.checkedCast(Ints.java:91) > at > org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329) > at > org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157) > at > org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125) > at > org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88) > at > org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109) > at > org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195) > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at > org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89) > at > org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61) > at > org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424) > at > org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at > org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79) > at java.lang.Thread.run(Thread.java:748) > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-13973: --- Reviewer: Jeff Jirsa > IllegalArgumentException in upgradesstables compaction > -- > > Key: CASSANDRA-13973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13973 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Dan Kinder >Assignee: Dinesh Joshi >Priority: Major > Fix For: 3.0.x, 3.11.x, 4.x > > > After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try > to run upgradesstables, most of them upgrade fine but I see the exception > below on several nodes, and it doesn't complete. > CASSANDRA-12717 looks similar but the stack trace is not the same, so I > assumed it is not identical. The various nodes this happens on all give the > same trace. > Might be notable that this is an analytics cluster with some large > partitions, in the GB size. > {noformat} > error: Out of range: 7316844981 > -- StackTrace -- > java.lang.IllegalArgumentException: Out of range: 7316844981 > at com.google.common.primitives.Ints.checkedCast(Ints.java:91) > at > org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329) > at > org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157) > at > org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125) > at > org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88) > at > org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109) > at > org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195) > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at > org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89) > at > org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61) > at > org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424) > at > org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at > org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79) > at java.lang.Thread.run(Thread.java:748) > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa reassigned CASSANDRA-13973: -- Assignee: Dinesh Joshi (was: Jeff Jirsa) > IllegalArgumentException in upgradesstables compaction > -- > > Key: CASSANDRA-13973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13973 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Dan Kinder >Assignee: Dinesh Joshi >Priority: Major > Fix For: 3.0.x, 3.11.x, 4.x > > > After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try > to run upgradesstables, most of them upgrade fine but I see the exception > below on several nodes, and it doesn't complete. > CASSANDRA-12717 looks similar but the stack trace is not the same, so I > assumed it is not identical. The various nodes this happens on all give the > same trace. > Might be notable that this is an analytics cluster with some large > partitions, in the GB size. > {noformat} > error: Out of range: 7316844981 > -- StackTrace -- > java.lang.IllegalArgumentException: Out of range: 7316844981 > at com.google.common.primitives.Ints.checkedCast(Ints.java:91) > at > org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329) > at > org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157) > at > org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125) > at > org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88) > at > org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109) > at > org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195) > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at > org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89) > at > org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61) > at > org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424) > at > org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at > org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79) > at java.lang.Thread.run(Thread.java:748) > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13973) IllegalArgumentException in upgradesstables compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356358#comment-16356358 ] Dinesh Joshi commented on CASSANDRA-13973: -- [~jjirsa] I have a patch that will log a meaningful error and warn the user about this issue. Fixing this issue requires additional effort. Until then this can be a helpful patch. ||13973|| |[branch|https://github.com/dineshjoshi/cassandra/tree/CASSANDRA-13973]| |[utests dtests|https://circleci.com/gh/dineshjoshi/workflows/cassandra/tree/CASSANDRA-13973]| || > IllegalArgumentException in upgradesstables compaction > -- > > Key: CASSANDRA-13973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13973 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Dan Kinder >Assignee: Jeff Jirsa >Priority: Major > Fix For: 3.0.x, 3.11.x, 4.x > > > After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try > to run upgradesstables, most of them upgrade fine but I see the exception > below on several nodes, and it doesn't complete. > CASSANDRA-12717 looks similar but the stack trace is not the same, so I > assumed it is not identical. The various nodes this happens on all give the > same trace. > Might be notable that this is an analytics cluster with some large > partitions, in the GB size. > {noformat} > error: Out of range: 7316844981 > -- StackTrace -- > java.lang.IllegalArgumentException: Out of range: 7316844981 > at com.google.common.primitives.Ints.checkedCast(Ints.java:91) > at > org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329) > at > org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157) > at > org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125) > at > org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88) > at > org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109) > at > org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195) > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at > org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89) > at > org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61) > at > org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424) > at > org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at > org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79) > at java.lang.Thread.run(Thread.java:748) > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14215) Cassandra does not seem to be respecting max hint window
[ https://issues.apache.org/jira/browse/CASSANDRA-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Greaves reassigned CASSANDRA-14215: Assignee: Kurt Greaves > Cassandra does not seem to be respecting max hint window > > > Key: CASSANDRA-14215 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14215 > Project: Cassandra > Issue Type: Bug > Components: Hints, Streaming and Messaging >Reporter: Arijit >Assignee: Kurt Greaves >Priority: Major > > On Cassandra 3.0.9, it was observed that Cassandra continues to write hints > even though a node remains down (and does not come up) for longer than the > default 3 hour window. > > After doing "nodetool setlogginglevel org.apache.cassandra TRACE", we see the > following log line in cassandra (debug) logs: > StorageProxy.java:2625 - Adding hints for [/10.0.100.84] > > One possible code path seems to be: > cas -> commitPaxos(proposal, consistencyForCommit, true); -> submitHint (in > StorageProxy.java) > > The "true" parameter above explicitly states that a hint should be recorded > and ignores the time window calculation performed by the shouldHint method > invoked in other code paths. Is there a reason for this behavior? > > Edit: There are actually two stacks that seem to be producing hints, the > "cas" and "syncWriteBatchedMutations" methods. I have posted them below. > > A third issue seems to be that Cassandra seems to reset the timer which > counts how long a node has been down after a restart. Thus if Cassandra is > restarted on a good node, it continues to accumulate hints for a down node > over the next three hours. > > WARN [SharedPool-Worker-14] 2018-02-06 22:15:51,136 StorageProxy.java:2636 - > Adding hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at > org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) > at > org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) > at > org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2603) > at > org.apache.cassandra.service.StorageProxy.commitPaxos(StorageProxy.java:540) > at org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:282) at > org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:432) > at > org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:407) > at > org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206) > at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237) > at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222) > at > org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115) > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513) > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407) > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) > at > io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32) > at > io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) > at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) at > java.lang.Thread.run(Thread.java:748) WARN > > > [SharedPool-Worker-8] 2018-02-06 22:15:51,153 StorageProxy.java:2636 - Adding > hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at > org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) > at > org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) > at > org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:1247) > at > org.apache.cassandra.service.StorageProxy.syncWriteBatchedMutations(StorageProxy.java:1014) > at > org.apache.cassandra.service.StorageProxy.mutateAtomically(StorageProxy.java:899) > at > org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:834) > at > org.apache.cassandra.cql3.statements.BatchStatement.executeWithoutConditions(BatchStatement.java:365) > at > org.apache.cassandra.cql3.statements.BatchStatement.execute(BatchStatement.java:343) > at > org.apache.cassandra.cql3.statements.BatchStatement.execute(BatchStatement.java:329) > at >
[jira] [Commented] (CASSANDRA-14215) Cassandra does not seem to be respecting max hint window
[ https://issues.apache.org/jira/browse/CASSANDRA-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356324#comment-16356324 ] Kurt Greaves commented on CASSANDRA-14215: -- Huh yeah, that makes sense. I missed that part but definitely also an issue. I've already started working on the patch so I'll take the ticket. > Cassandra does not seem to be respecting max hint window > > > Key: CASSANDRA-14215 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14215 > Project: Cassandra > Issue Type: Bug > Components: Hints, Streaming and Messaging >Reporter: Arijit >Priority: Major > > On Cassandra 3.0.9, it was observed that Cassandra continues to write hints > even though a node remains down (and does not come up) for longer than the > default 3 hour window. > > After doing "nodetool setlogginglevel org.apache.cassandra TRACE", we see the > following log line in cassandra (debug) logs: > StorageProxy.java:2625 - Adding hints for [/10.0.100.84] > > One possible code path seems to be: > cas -> commitPaxos(proposal, consistencyForCommit, true); -> submitHint (in > StorageProxy.java) > > The "true" parameter above explicitly states that a hint should be recorded > and ignores the time window calculation performed by the shouldHint method > invoked in other code paths. Is there a reason for this behavior? > > Edit: There are actually two stacks that seem to be producing hints, the > "cas" and "syncWriteBatchedMutations" methods. I have posted them below. > > A third issue seems to be that Cassandra seems to reset the timer which > counts how long a node has been down after a restart. Thus if Cassandra is > restarted on a good node, it continues to accumulate hints for a down node > over the next three hours. > > WARN [SharedPool-Worker-14] 2018-02-06 22:15:51,136 StorageProxy.java:2636 - > Adding hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at > org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) > at > org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) > at > org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2603) > at > org.apache.cassandra.service.StorageProxy.commitPaxos(StorageProxy.java:540) > at org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:282) at > org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:432) > at > org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:407) > at > org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206) > at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237) > at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222) > at > org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115) > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513) > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407) > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) > at > io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32) > at > io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) > at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) at > java.lang.Thread.run(Thread.java:748) WARN > > > [SharedPool-Worker-8] 2018-02-06 22:15:51,153 StorageProxy.java:2636 - Adding > hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at > org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) > at > org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) > at > org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:1247) > at > org.apache.cassandra.service.StorageProxy.syncWriteBatchedMutations(StorageProxy.java:1014) > at > org.apache.cassandra.service.StorageProxy.mutateAtomically(StorageProxy.java:899) > at > org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:834) > at > org.apache.cassandra.cql3.statements.BatchStatement.executeWithoutConditions(BatchStatement.java:365) > at > org.apache.cassandra.cql3.statements.BatchStatement.execute(BatchStatement.java:343) > at >
[jira] [Comment Edited] (CASSANDRA-14215) Cassandra does not seem to be respecting max hint window
[ https://issues.apache.org/jira/browse/CASSANDRA-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356241#comment-16356241 ] Kurt Greaves edited comment on CASSANDRA-14215 at 2/8/18 1:11 AM: -- My understanding was that we'd keep hints for the first N hours, and then we'd stop storing them. We'd always replay whatever hints are stored and delete them after replaying. Code in {{org.apache.cassandra.service.StorageProxy#shouldHint}} seems to imply the above {code:java} boolean hintWindowExpired = Gossiper.instance.getEndpointDowntime(ep) > DatabaseDescriptor.getMaxHintWindow(); if (hintWindowExpired) { HintsService.instance.metrics.incrPastWindow(ep); Tracing.trace("Not hinting {} which has been down {} ms", ep, Gossiper.instance.getEndpointDowntime(ep)); } return !hintWindowExpired; {code} This sounds like an issue to me, pretty sure we shouldn't be storing anything that's past the hint window. bq. Finally, check out the bit around the node down timer being reset on Cassandra restarts. This does not matter if we are ok with generating hints beyond the max hint window, but I really don't think that we should be... This is also not really intended. As checks are currently only performed on how long the node has been down if it comes up after 3 hours then goes back down straight away we'll effectively store 6 hours of hints for the node. It's probably reasonable to only store a maximum of {{max_hint_window_in_ms}}. We might be able to get away with just looking at the timestamp of the earliest hint for the node and using that if it's prior to the current downtime. I'll have a look at these in the next few days. {quote}There is a deleteAllHintsForEndpoint JMX target that would let you purge hints (manually), but it does perhaps seem like a missing feature that we don't more aggressively clean up hints that are expired. {quote} [~VincentWhite] was looking at this recently while trying to solve a problem where hint replaying would get stuck and continuously replay a single hint if it happened to fail on the receiving side continuously (usually due to receiving node being overloaded and never acknowledging the hint in time). I recall that he also had problems because we have no service to clean up hints in this case. He's on leave this week but would probably have some input around this. was (Author: kurtg): My understanding was that we'd keep hints for the first N hours, and then we'd stop storing them. We'd always replay whatever hints are stored and delete them after replaying. Code in {{org.apache.cassandra.service.StorageProxy#shouldHint}} seems to imply the above {code:java} boolean hintWindowExpired = Gossiper.instance.getEndpointDowntime(ep) > DatabaseDescriptor.getMaxHintWindow(); if (hintWindowExpired) { HintsService.instance.metrics.incrPastWindow(ep); Tracing.trace("Not hinting {} which has been down {} ms", ep, Gossiper.instance.getEndpointDowntime(ep)); } return !hintWindowExpired; {code} This sounds like an issue to me, pretty sure we shouldn't be storing anything that's past the hint window. bq. Finally, check out the bit around the node down timer being reset on Cassandra restarts. This does not matter if we are ok with generating hints beyond the max hint window, but I really don't think that we should be... This is also not really intended. As checks are currently only performed on how long the node has been down if it comes up after 3 hours then goes back down straight away we'll effectively store 6 hours of hints for the node. It's probably reasonable to only store a maximum of {{max_hint_window_in_ms}}. We might be able to get away with just looking at the timestamp of the earliest hint for the node and using that if it's prior to the current downtime. I'll have a look at these in the next few days. {quote}There is a deleteAllHintsForEndpoint JMX target that would let you purge hints (manually), but it does perhaps seem like a missing feature that we don't more aggressively clean up hints that are expired. {quote} [~VincentWhite] was looking at this recently while trying to solve a problem where hint replaying would get stuck and continuously replay a single hint if it happened to fail on the receiving side continuously (usually due to receiving node being overloaded and never acknowledging the hint in time). I recall that he also had problems because we have no service to clean up hints in this case. He's on leave this week but would probably have some input around this. > Cassandra does not seem to be respecting max hint window > > > Key: CASSANDRA-14215 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14215 > Project: Cassandra > Issue Type: Bug > Components: Hints, Streaming and Messaging >
[jira] [Commented] (CASSANDRA-14087) NPE when CAS encounters empty frozen collection
[ https://issues.apache.org/jira/browse/CASSANDRA-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356318#comment-16356318 ] Kurt Greaves commented on CASSANDRA-14087: -- That's worrying, pretty sure it worked when I tested it :S. Thanks, I'll have a look. > NPE when CAS encounters empty frozen collection > --- > > Key: CASSANDRA-14087 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14087 > Project: Cassandra > Issue Type: Bug >Reporter: Jens Bannmann >Assignee: Kurt Greaves >Priority: Major > Fix For: 3.0.16, 3.11.2 > > > When a compare-and-set operation specifying an equality criterion with a > non-{{null}} value encounters an empty collection ({{null}} cell), the server > throws a {{NullPointerException}} and the query fails. > This does not happen for non-frozen collections. > There's a self-contained test case at > [github|https://github.com/incub8/cassandra-npe-in-cas]. > The stack trace for 3.11.0 is: > {code} > ERROR [Native-Transport-Requests-1] 2017-11-27 12:59:26,924 > QueryMessage.java:129 - Unexpected error during query > java.lang.NullPointerException: null > at > org.apache.cassandra.cql3.ColumnCondition$CollectionBound.appliesTo(ColumnCondition.java:546) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.statements.CQL3CasRequest$ColumnsConditions.appliesTo(CQL3CasRequest.java:324) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.statements.CQL3CasRequest.appliesTo(CQL3CasRequest.java:210) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:265) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:441) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:416) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:217) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:248) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:233) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:116) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:517) > [apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:410) > [apache-cassandra-3.11.0.jar:3.11.0] > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) > [netty-all-4.0.44.Final.jar:4.0.44.Final] > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357) > [netty-all-4.0.44.Final.jar:4.0.44.Final] > at > io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35) > [netty-all-4.0.44.Final.jar:4.0.44.Final] > at > io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348) > [netty-all-4.0.44.Final.jar:4.0.44.Final] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_151] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162) > [apache-cassandra-3.11.0.jar:3.11.0] > at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) > [apache-cassandra-3.11.0.jar:3.11.0] > at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151] > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14215) Cassandra does not seem to be respecting max hint window
[ https://issues.apache.org/jira/browse/CASSANDRA-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356275#comment-16356275 ] Arijit commented on CASSANDRA-14215: Thanks [~KurtG]. Yes that's the main issue, we're storing hints beyond the hint window. About the Cassandra restarts, I meant restarts on the *live node* and not on the *dead node* also end up resetting the timer. Though the point you mentioned is valid as well. > Cassandra does not seem to be respecting max hint window > > > Key: CASSANDRA-14215 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14215 > Project: Cassandra > Issue Type: Bug > Components: Hints, Streaming and Messaging >Reporter: Arijit >Priority: Major > > On Cassandra 3.0.9, it was observed that Cassandra continues to write hints > even though a node remains down (and does not come up) for longer than the > default 3 hour window. > > After doing "nodetool setlogginglevel org.apache.cassandra TRACE", we see the > following log line in cassandra (debug) logs: > StorageProxy.java:2625 - Adding hints for [/10.0.100.84] > > One possible code path seems to be: > cas -> commitPaxos(proposal, consistencyForCommit, true); -> submitHint (in > StorageProxy.java) > > The "true" parameter above explicitly states that a hint should be recorded > and ignores the time window calculation performed by the shouldHint method > invoked in other code paths. Is there a reason for this behavior? > > Edit: There are actually two stacks that seem to be producing hints, the > "cas" and "syncWriteBatchedMutations" methods. I have posted them below. > > A third issue seems to be that Cassandra seems to reset the timer which > counts how long a node has been down after a restart. Thus if Cassandra is > restarted on a good node, it continues to accumulate hints for a down node > over the next three hours. > > WARN [SharedPool-Worker-14] 2018-02-06 22:15:51,136 StorageProxy.java:2636 - > Adding hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at > org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) > at > org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) > at > org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2603) > at > org.apache.cassandra.service.StorageProxy.commitPaxos(StorageProxy.java:540) > at org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:282) at > org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:432) > at > org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:407) > at > org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206) > at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237) > at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222) > at > org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115) > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513) > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407) > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) > at > io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32) > at > io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) > at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) at > java.lang.Thread.run(Thread.java:748) WARN > > > [SharedPool-Worker-8] 2018-02-06 22:15:51,153 StorageProxy.java:2636 - Adding > hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at > org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) > at > org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) > at > org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:1247) > at > org.apache.cassandra.service.StorageProxy.syncWriteBatchedMutations(StorageProxy.java:1014) > at > org.apache.cassandra.service.StorageProxy.mutateAtomically(StorageProxy.java:899) > at > org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:834) > at > org.apache.cassandra.cql3.statements.BatchStatement.executeWithoutConditions(BatchStatement.java:365) > at >
[jira] [Commented] (CASSANDRA-14215) Cassandra does not seem to be respecting max hint window
[ https://issues.apache.org/jira/browse/CASSANDRA-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356241#comment-16356241 ] Kurt Greaves commented on CASSANDRA-14215: -- My understanding was that we'd keep hints for the first N hours, and then we'd stop storing them. We'd always replay whatever hints are stored and delete them after replaying. Code in {{org.apache.cassandra.service.StorageProxy#shouldHint}} seems to imply the above {code:java} boolean hintWindowExpired = Gossiper.instance.getEndpointDowntime(ep) > DatabaseDescriptor.getMaxHintWindow(); if (hintWindowExpired) { HintsService.instance.metrics.incrPastWindow(ep); Tracing.trace("Not hinting {} which has been down {} ms", ep, Gossiper.instance.getEndpointDowntime(ep)); } return !hintWindowExpired; {code} This sounds like an issue to me, pretty sure we shouldn't be storing anything that's past the hint window. bq. Finally, check out the bit around the node down timer being reset on Cassandra restarts. This does not matter if we are ok with generating hints beyond the max hint window, but I really don't think that we should be... This is also not really intended. As checks are currently only performed on how long the node has been down if it comes up after 3 hours then goes back down straight away we'll effectively store 6 hours of hints for the node. It's probably reasonable to only store a maximum of {{max_hint_window_in_ms}}. We might be able to get away with just looking at the timestamp of the earliest hint for the node and using that if it's prior to the current downtime. I'll have a look at these in the next few days. {quote}There is a deleteAllHintsForEndpoint JMX target that would let you purge hints (manually), but it does perhaps seem like a missing feature that we don't more aggressively clean up hints that are expired. {quote} [~VincentWhite] was looking at this recently while trying to solve a problem where hint replaying would get stuck and continuously replay a single hint if it happened to fail on the receiving side continuously (usually due to receiving node being overloaded and never acknowledging the hint in time). I recall that he also had problems because we have no service to clean up hints in this case. He's on leave this week but would probably have some input around this. > Cassandra does not seem to be respecting max hint window > > > Key: CASSANDRA-14215 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14215 > Project: Cassandra > Issue Type: Bug > Components: Hints, Streaming and Messaging >Reporter: Arijit >Priority: Major > > On Cassandra 3.0.9, it was observed that Cassandra continues to write hints > even though a node remains down (and does not come up) for longer than the > default 3 hour window. > > After doing "nodetool setlogginglevel org.apache.cassandra TRACE", we see the > following log line in cassandra (debug) logs: > StorageProxy.java:2625 - Adding hints for [/10.0.100.84] > > One possible code path seems to be: > cas -> commitPaxos(proposal, consistencyForCommit, true); -> submitHint (in > StorageProxy.java) > > The "true" parameter above explicitly states that a hint should be recorded > and ignores the time window calculation performed by the shouldHint method > invoked in other code paths. Is there a reason for this behavior? > > Edit: There are actually two stacks that seem to be producing hints, the > "cas" and "syncWriteBatchedMutations" methods. I have posted them below. > > A third issue seems to be that Cassandra seems to reset the timer which > counts how long a node has been down after a restart. Thus if Cassandra is > restarted on a good node, it continues to accumulate hints for a down node > over the next three hours. > > WARN [SharedPool-Worker-14] 2018-02-06 22:15:51,136 StorageProxy.java:2636 - > Adding hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at > org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) > at > org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) > at > org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2603) > at > org.apache.cassandra.service.StorageProxy.commitPaxos(StorageProxy.java:540) > at org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:282) at > org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:432) > at > org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:407) > at > org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206) > at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237) > at
[jira] [Updated] (CASSANDRA-14222) Add hot reloading of SSL Certificates capability to Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-14222: - Status: Patch Available (was: Open) > Add hot reloading of SSL Certificates capability to Cassandra > - > > Key: CASSANDRA-14222 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14222 > Project: Cassandra > Issue Type: New Feature >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Major > Fix For: 4.x > > > Cassandra does not currently hot reload SSL certificates. For ease of > operation it would be useful if we add this capability. This patch would > watch changes to the truststore & keystore and would reload them when they're > changed. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14222) Add hot reloading of SSL Certificates capability to Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356237#comment-16356237 ] Dinesh Joshi commented on CASSANDRA-14222: -- Here's a patch for trunk: ||trunk|| |[branch|https://github.com/dineshjoshi/cassandra/tree/14222-trunk]| |[utests dtests|https://circleci.com/gh/dineshjoshi/workflows/cassandra/tree/14222-trunk]| || > Add hot reloading of SSL Certificates capability to Cassandra > - > > Key: CASSANDRA-14222 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14222 > Project: Cassandra > Issue Type: New Feature >Reporter: Dinesh Joshi >Assignee: Dinesh Joshi >Priority: Major > Fix For: 4.x > > > Cassandra does not currently hot reload SSL certificates. For ease of > operation it would be useful if we add this capability. This patch would > watch changes to the truststore & keystore and would reload them when they're > changed. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14222) Add hot reloading of SSL Certificates capability to Cassandra
Dinesh Joshi created CASSANDRA-14222: Summary: Add hot reloading of SSL Certificates capability to Cassandra Key: CASSANDRA-14222 URL: https://issues.apache.org/jira/browse/CASSANDRA-14222 Project: Cassandra Issue Type: New Feature Reporter: Dinesh Joshi Assignee: Dinesh Joshi Fix For: 4.x Cassandra does not currently hot reload SSL certificates. For ease of operation it would be useful if we add this capability. This patch would watch changes to the truststore & keystore and would reload them when they're changed. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14221) I am running a script to create keyspaces and tables but i am getting schema issues after sometime while it creating some tables.
samba vunnava created CASSANDRA-14221: - Summary: I am running a script to create keyspaces and tables but i am getting schema issues after sometime while it creating some tables. Key: CASSANDRA-14221 URL: https://issues.apache.org/jira/browse/CASSANDRA-14221 Project: Cassandra Issue Type: Bug Components: Configuration Reporter: samba vunnava Fix For: 3.11.1 INFO [main] 2018-02-06 10:01:56,253 c.k.c.f.IidFinderUtils - Creating cache/child tables for LU Acc source table OMS_TBCATALOG_ITEM_1 INFO [main] 2018-02-06 10:02:02,056 c.k.c.f.IidFinderUtils - Creating cache/child tables for LU Acc source table OMS_TBCATALOG_ITEM_2 WARN [cluster1-worker-5] 2018-02-06 10:02:16,289 c.d.d.c.Cluster - No schema agreement from live replicas after 10 s. The schema may not be up to date on some nodes. INFO [main] 2018-02-06 10:02:19,300 c.k.c.f.IidFinderUtils - Creating cache/child tables for LU Acc source table OMS_TBCATALOG_ITEM_3 WARN [cluster1-worker-7] 2018-02-06 10:02:29,471 c.d.d.c.Cluster - No schema agreement from live replicas after 10 s. The schema may not be up to date on some nodes. ERROR [main] 2018-02-06 10:03:00,492 c.k.c.f.IidFinderUtils - Failed to create table for LU [Acc] com.datastax.driver.core.exceptions.OperationTimedOutException: [/135.164.54.234:9042] Timed out waiting for server response at com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:44) at com.datastax.driver.core.exceptions.OperationTimedOutException.copy(OperationTimedOutException.java:26) at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37) at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245) at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68) at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:43) at com.k2view.cdbms.finder.IidFinderUtils.createTable(IidFinderUtils.java:91) at com.k2view.cdbms.finder.IidFinderUtils.createCacheTables(IidFinderUtils.java:76) at com.k2view.cdbms.finder.IidFinderManager.tablesCreationInit(IidFinderManager.java:68) at com.k2view.cdbms.finder.IidFinderManager.main(IidFinderManager.java:43) Caused by: com.datastax.driver.core.exceptions.OperationTimedOutException: [/135.164.54.234:9042] Timed out waiting for server response at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onTimeout(RequestHandler.java:772) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1380) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:625) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:700) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:428) at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144) at java.lang.Thread.run(Thread.java:748) INFO [main] 2018-02-06 10:03:00,493 c.k.c.f.IidFinderUtils - Creating cache/child tables for LU Acc source table OMS_TBCATALOG_ITEM_4 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14087) NPE when CAS encounters empty frozen collection
[ https://issues.apache.org/jira/browse/CASSANDRA-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356057#comment-16356057 ] Blake Eggleston commented on CASSANDRA-14087: - Sorry [~KurtG], I did some test runs and then forgot about them over the holidays. So I'm +1 on the patch for 3.0 and 3.11, however the test case you've added fails against trunk. Could you take a look at that? > NPE when CAS encounters empty frozen collection > --- > > Key: CASSANDRA-14087 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14087 > Project: Cassandra > Issue Type: Bug >Reporter: Jens Bannmann >Assignee: Kurt Greaves >Priority: Major > Fix For: 3.0.16, 3.11.2 > > > When a compare-and-set operation specifying an equality criterion with a > non-{{null}} value encounters an empty collection ({{null}} cell), the server > throws a {{NullPointerException}} and the query fails. > This does not happen for non-frozen collections. > There's a self-contained test case at > [github|https://github.com/incub8/cassandra-npe-in-cas]. > The stack trace for 3.11.0 is: > {code} > ERROR [Native-Transport-Requests-1] 2017-11-27 12:59:26,924 > QueryMessage.java:129 - Unexpected error during query > java.lang.NullPointerException: null > at > org.apache.cassandra.cql3.ColumnCondition$CollectionBound.appliesTo(ColumnCondition.java:546) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.statements.CQL3CasRequest$ColumnsConditions.appliesTo(CQL3CasRequest.java:324) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.statements.CQL3CasRequest.appliesTo(CQL3CasRequest.java:210) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:265) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:441) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:416) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:217) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:248) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:233) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:116) > ~[apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:517) > [apache-cassandra-3.11.0.jar:3.11.0] > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:410) > [apache-cassandra-3.11.0.jar:3.11.0] > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) > [netty-all-4.0.44.Final.jar:4.0.44.Final] > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357) > [netty-all-4.0.44.Final.jar:4.0.44.Final] > at > io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35) > [netty-all-4.0.44.Final.jar:4.0.44.Final] > at > io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348) > [netty-all-4.0.44.Final.jar:4.0.44.Final] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_151] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162) > [apache-cassandra-3.11.0.jar:3.11.0] > at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) > [apache-cassandra-3.11.0.jar:3.11.0] > at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151] > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14214) Make it possible to change NEVER_PURGE_TOMBSTONES during runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356006#comment-16356006 ] Chris Lohfink commented on CASSANDRA-14214: --- looks good to me +1 > Make it possible to change NEVER_PURGE_TOMBSTONES during runtime > > > Key: CASSANDRA-14214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14214 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Major > Fix For: 4.x > > > If a node gets behind on compaction it could be good to be able to flip the > NEVER_PURGE_TOMBSTONES flag to speed up compactions during runtime. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14212) Back port CASSANDRA-13080 to 3.11.2 (Use new token allocation for non bootstrap case as well)
[ https://issues.apache.org/jira/browse/CASSANDRA-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355916#comment-16355916 ] Jon Haddad commented on CASSANDRA-14212: +1 > Back port CASSANDRA-13080 to 3.11.2 (Use new token allocation for non > bootstrap case as well) > - > > Key: CASSANDRA-14212 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14212 > Project: Cassandra > Issue Type: Improvement >Reporter: mck >Assignee: mck >Priority: Major > Fix For: 3.11.x > > > Backport CASSANDRA-13080 to 3.11.x > > The patch applies without conflict to the {{cassandra-3.11}} and equally > concerns to users of Cassandra-3.11.1 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14212) Back port CASSANDRA-13080 to 3.11.2 (Use new token allocation for non bootstrap case as well)
[ https://issues.apache.org/jira/browse/CASSANDRA-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRA-14212: --- Reviewer: Jon Haddad > Back port CASSANDRA-13080 to 3.11.2 (Use new token allocation for non > bootstrap case as well) > - > > Key: CASSANDRA-14212 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14212 > Project: Cassandra > Issue Type: Improvement >Reporter: mck >Assignee: mck >Priority: Major > Fix For: 3.11.x > > > Backport CASSANDRA-13080 to 3.11.x > > The patch applies without conflict to the {{cassandra-3.11}} and equally > concerns to users of Cassandra-3.11.1 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14206) pytest/python 3 dtests breaks tests on 3.0 and 3.11
[ https://issues.apache.org/jira/browse/CASSANDRA-14206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14206: --- Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as [7c05a111a2018649ab4ab06d31952d0abc9b8537|https://github.com/apache/cassandra-dtest/commit/7c05a111a2018649ab4ab06d31952d0abc9b8537] thanks! > pytest/python 3 dtests breaks tests on 3.0 and 3.11 > --- > > Key: CASSANDRA-14206 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14206 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Major > Attachments: loose_version.diff > > > In 2.7 it uses the cmp built-in to compare the list of version components > which accepts comparisons of strings and integers. In python 3 it manually > compares each using <, ==, and > which can fail if the the types don't match. > Switch to using our own comparison function that preserves the old behavior. > Additionally the new thrift bindings don't accept strings where they used to. > The strings need to be provided use utf-8 encoded bytes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[1/3] cassandra-dtest git commit: pytest/python 3 dtests breaks tests on 3.0 and 3.11
Repository: cassandra-dtest Updated Branches: refs/heads/master 39e223fe8 -> 7c05a111a http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/7c05a111/write_failures_test.py -- diff --git a/write_failures_test.py b/write_failures_test.py index 17ce50e..67bb22a 100644 --- a/write_failures_test.py +++ b/write_failures_test.py @@ -222,9 +222,9 @@ class TestWriteFailures(Tester): client.set_keyspace(KEYSPACE) with pytest.raises(self.expected_expt): -client.insert('key1', +client.insert('key1'.encode(), thrift_types.ColumnParent('mytable'), - thrift_types.Column('value', 'Value 1', 0), + thrift_types.Column('value'.encode(), 'Value 1'.enocde(), 0), thrift_types.ConsistencyLevel.ALL) client.transport.close() - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[3/3] cassandra-dtest git commit: pytest/python 3 dtests breaks tests on 3.0 and 3.11
pytest/python 3 dtests breaks tests on 3.0 and 3.11 Patch by Ariel Weisberg; Reviewed by Sam Tunnicliffe for CASSANDRA-14206 Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/7c05a111 Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/7c05a111 Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/7c05a111 Branch: refs/heads/master Commit: 7c05a111a2018649ab4ab06d31952d0abc9b8537 Parents: 39e223f Author: Ariel WeisbergAuthored: Wed Jan 31 12:33:58 2018 -0500 Committer: Ariel Weisberg Committed: Wed Feb 7 13:14:23 2018 -0500 -- conftest.py| 29 +- dtest.py |2 +- offline_tools_test.py |6 +- putget_test.py |4 +- rebuild_test.py|2 +- snitch_test.py |5 +- super_column_cache_test.py | 50 +- super_counter_test.py | 12 +- thrift_hsha_test.py|2 +- thrift_test.py | 1423 --- write_failures_test.py |4 +- 11 files changed, 805 insertions(+), 734 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/7c05a111/conftest.py -- diff --git a/conftest.py b/conftest.py index 650e9c8..04a0eec 100644 --- a/conftest.py +++ b/conftest.py @@ -8,6 +8,7 @@ import platform import copy import inspect import subprocess +from itertools import zip_longest from dtest import running_in_docker, cleanup_docker_environment_before_test_execution @@ -351,10 +352,34 @@ def fixture_dtest_setup(request, parse_dtest_config, fixture_dtest_setup_overrid dtest_setup.cleanup_cluster() +#Based on https://bugs.python.org/file25808/14894.patch +def loose_version_compare(a, b): +for i, j in zip_longest(a.version, b.version, fillvalue=''): +if type(i) != type(j): +i = str(i) +j = str(j) +if i == j: +continue +elif i < j: +return -1 +else: # i > j +return 1 + +#Longer version strings with equal prefixes are equal, but if one version string is longer than it is greater +aLen = len(a.version) +bLen = len(b.version) +if aLen == bLen: +return 0 +elif aLen < bLen: +return -1 +else: +return 1 + + def _skip_msg(current_running_version, since_version, max_version): -if current_running_version < since_version: +if loose_version_compare(current_running_version, since_version) < 0: return "%s < %s" % (current_running_version, since_version) -if max_version and current_running_version > max_version: +if max_version and loose_version_compare(current_running_version, max_version) > 0: return "%s > %s" % (current_running_version, max_version) http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/7c05a111/dtest.py -- diff --git a/dtest.py b/dtest.py index aad9e58..914f2f7 100644 --- a/dtest.py +++ b/dtest.py @@ -268,7 +268,7 @@ class Tester: node.set_install_dir(version=version) else: node.set_install_dir(install_dir=self.dtest_config.cassandra_dir) -os.environ.set('CASSANDRA_DIR', self.dtest_config.cassandra_dir) +os.environ['CASSANDRA_DIR'] = self.dtest_config.cassandra_dir def go(self, func): runner = Runner(func) http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/7c05a111/offline_tools_test.py -- diff --git a/offline_tools_test.py b/offline_tools_test.py index 7c7cc8f..9e3dc73 100644 --- a/offline_tools_test.py +++ b/offline_tools_test.py @@ -378,13 +378,17 @@ class TestOfflineTools(Tester): # change before it's release. if testversion < '4.0': (out, error, rc) = node1.run_sstableupgrade(keyspace='ks', column_family='cf') +out = str(out) +error = str(error) logger.debug(out) logger.debug(error) logger.debug('Upgraded ks.cf sstable: {}'.format(node1.get_sstables(keyspace='ks', column_family='cf'))) -assert 'Found 1 sstables that need upgrading.' in out +assert 'Found 1 sstables that need upgrading.' in str(out) # Check that sstableupgrade finds no upgrade needed on current version. (out, error, rc) = node1.run_sstableupgrade(keyspace='ks', column_family='cf') +out = str(out) +error = str(error) logger.debug(out) logger.debug(error) assert 'Found 0 sstables that need upgrading.' in out
[2/3] cassandra-dtest git commit: pytest/python 3 dtests breaks tests on 3.0 and 3.11
http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/7c05a111/thrift_test.py -- diff --git a/thrift_test.py b/thrift_test.py index 23e3f7b..0990d71 100644 --- a/thrift_test.py +++ b/thrift_test.py @@ -4,6 +4,7 @@ import time import uuid import pytest import logging +import codecs from thrift.protocol import TBinaryProtocol from thrift.Thrift import TApplicationException @@ -31,7 +32,9 @@ from tools.assertions import (assert_all, assert_none, assert_one) since = pytest.mark.since logger = logging.getLogger(__name__) - +utf8encoder = codecs.getencoder('utf-8') +def utf8encode(str): +return utf8encoder(str)[0] def get_thrift_client(host='127.0.0.1', port=9160): socket = TSocket.TSocket(host, port) @@ -66,7 +69,7 @@ class TestThrift(Tester): return dtest_setup_overrides @pytest.fixture(scope='function', autouse=True) -def fixture_set_cluster_settings(self, fixture_dtest_setup): +def fixture_set_cluster_settings(self, fixture_dtest_setup, set_dtest_setup_on_function): fixture_dtest_setup.cluster.populate(1) node1, = fixture_dtest_setup.cluster.nodelist() @@ -75,7 +78,7 @@ class TestThrift(Tester): # automatically. It does not matter what token we set as we only # ever use one node. if not self.dtest_config.use_vnodes: -node1.set_configuration_options(values={'initial_token': "a".encode('hex')}) +node1.set_configuration_options(values={'initial_token': 'abcd'}) fixture_dtest_setup.cluster.start(wait_for_binary_proto=True) fixture_dtest_setup.cluster.nodelist()[0].watch_log_for("Listening for thrift clients") # Wait for the thrift port to open @@ -95,8 +98,8 @@ class TestThrift(Tester): cf_defs=[ Cassandra.CfDef('Keyspace1', 'Standard1'), Cassandra.CfDef('Keyspace1', 'Standard2'), -Cassandra.CfDef('Keyspace1', 'Standard3', column_metadata=[Cassandra.ColumnDef('c1', 'AsciiType'), Cassandra.ColumnDef('c2', 'AsciiType')]), -Cassandra.CfDef('Keyspace1', 'Standard4', column_metadata=[Cassandra.ColumnDef('c1', 'AsciiType')]), +Cassandra.CfDef('Keyspace1', 'Standard3', column_metadata=[Cassandra.ColumnDef(utf8encode('c1'), 'AsciiType'), Cassandra.ColumnDef(utf8encode('c2'), 'AsciiType')]), +Cassandra.CfDef('Keyspace1', 'Standard4', column_metadata=[Cassandra.ColumnDef(utf8encode('c1'), 'AsciiType')]), Cassandra.CfDef('Keyspace1', 'StandardLong1', comparator_type='LongType'), Cassandra.CfDef('Keyspace1', 'StandardInteger1', comparator_type='IntegerType'), Cassandra.CfDef('Keyspace1', 'StandardComposite', comparator_type='CompositeType(AsciiType, AsciiType)'), @@ -105,10 +108,10 @@ class TestThrift(Tester): Cassandra.CfDef('Keyspace1', 'Super3', column_type='Super', comparator_type='LongType', subcomparator_type='UTF8Type'), Cassandra.CfDef('Keyspace1', 'Counter1', default_validation_class='CounterColumnType'), Cassandra.CfDef('Keyspace1', 'SuperCounter1', column_type='Super', default_validation_class='CounterColumnType'), -Cassandra.CfDef('Keyspace1', 'Indexed1', column_metadata=[Cassandra.ColumnDef('birthdate', 'LongType', Cassandra.IndexType.KEYS, 'birthdate_index')]), +Cassandra.CfDef('Keyspace1', 'Indexed1', column_metadata=[Cassandra.ColumnDef(utf8encode('birthdate'), 'LongType', Cassandra.IndexType.KEYS, 'birthdate_index')]), Cassandra.CfDef('Keyspace1', 'Indexed2', comparator_type='TimeUUIDType', column_metadata=[Cassandra.ColumnDef(uuid.UUID('--1000--').bytes, 'LongType', Cassandra.IndexType.KEYS)]), Cassandra.CfDef('Keyspace1', 'Indexed3', comparator_type='TimeUUIDType', column_metadata=[Cassandra.ColumnDef(uuid.UUID('--1000--').bytes, 'UTF8Type', Cassandra.IndexType.KEYS)]), -Cassandra.CfDef('Keyspace1', 'Indexed4', column_metadata=[Cassandra.ColumnDef('a', 'LongType', Cassandra.IndexType.KEYS, 'a_index'), Cassandra.ColumnDef('z', 'UTF8Type')]), +Cassandra.CfDef('Keyspace1', 'Indexed4', column_metadata=[Cassandra.ColumnDef(utf8encode('a'), 'LongType', Cassandra.IndexType.KEYS, 'a_index'), Cassandra.ColumnDef(utf8encode('z'), 'UTF8Type')]), Cassandra.CfDef('Keyspace1', 'Expiring', default_time_to_live=2) ]) @@ -120,7 +123,7 @@ class TestThrift(Tester): Cassandra.CfDef('Keyspace2', 'Super4', column_type='Super', subcomparator_type='TimeUUIDType'), ]) for ks in [keyspace1, keyspace2]: -cls.client.system_add_keyspace(ks) +client.system_add_keyspace(ks) def i64(n): @@ -135,7 +138,12 @@ def i16(n): return _i16(n) -def composite(item1, item2=None,
[jira] [Created] (CASSANDRA-14220) C* returns to the driver NullPointerException sometimes when creating a keyspace
Jaume M created CASSANDRA-14220: --- Summary: C* returns to the driver NullPointerException sometimes when creating a keyspace Key: CASSANDRA-14220 URL: https://issues.apache.org/jira/browse/CASSANDRA-14220 Project: Cassandra Issue Type: Bug Environment: C* version: 3.11.1 Reporter: Jaume M Attachments: node1_system.log, node2_system.log, node3_system.log This happening regular on the python driver CI and it's reasonable to assume that C* is under heavy load when it happens. The CQL command sent is {code} CREATE KEYSPACE keyspace WITH replication " = {'class': 'SimpleStrategy', 'replication_factor': 1} {code} or {code} CREATE TABLE table_name (k INT, i INT, PRIMARY KEY(k, i)) {code} The error the driver gets is {code} {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14206) pytest/python 3 dtests breaks tests on 3.0 and 3.11
[ https://issues.apache.org/jira/browse/CASSANDRA-14206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355793#comment-16355793 ] Ariel Weisberg commented on CASSANDRA-14206: I pushed additional commit(s). I guess I forgot to push it before asking you to review. I think it cleans up the debug statements you are talking about as well as some codec usage. > pytest/python 3 dtests breaks tests on 3.0 and 3.11 > --- > > Key: CASSANDRA-14206 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14206 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Major > Attachments: loose_version.diff > > > In 2.7 it uses the cmp built-in to compare the list of version components > which accepts comparisons of strings and integers. In python 3 it manually > compares each using <, ==, and > which can fail if the the types don't match. > Switch to using our own comparison function that preserves the old behavior. > Additionally the new thrift bindings don't accept strings where they used to. > The strings need to be provided use utf-8 encoded bytes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-7544) Allow storage port to be configurable per node
[ https://issues.apache.org/jira/browse/CASSANDRA-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-7544: -- Status: Patch Available (was: Reopened) RE replicatingNodes this has always been unsafe. Previously it was iterated in {{StringUtils.join}} I'm not sure if we should throw CME if it occurs or just ignore concurrent modifications. This is only used by nodetool for monitoring, but it might be used in an automated fashion. Proposed changes here https://github.com/aweisberg/cassandra/tree/cassandra-7544-fixes I went with iterating it safely, but throwing CME if CME is detected. We need to iterate safely even if we intend to throw CME. > Allow storage port to be configurable per node > -- > > Key: CASSANDRA-7544 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7544 > Project: Cassandra > Issue Type: Improvement >Reporter: Sam Overton >Assignee: Ariel Weisberg >Priority: Major > Fix For: 4.0 > > > Currently storage_port must be configured identically on all nodes in a > cluster and it is assumed that this is the case when connecting to a remote > node. > This prevents running in any environment that requires multiple nodes to be > able to bind to the same network interface, such as with many automatic > provisioning/deployment frameworks. > The current solutions seems to be > * use a separate network interface for each node deployed to the same box. > This puts a big requirement on IP allocation at large scale. > * allow multiple clusters to be provisioned from the same resource pool, but > restrict allocation to a maximum of one node per host from each cluster, > assuming each cluster is running on a different storage port. > It would make operations much simpler in these kind of environments if the > environment provisioning the resources could assign the ports to be used when > bringing up a new node on shared hardware. > The changes required would be at least the following: > 1. configure seeds as IP:port instead of just IP > 2. gossip the storage port as part of a node's ApplicationState > 3. refer internally to nodes by hostID instead of IP, since there will be > multiple nodes with the same IP > (1) & (2) are mostly trivial and I already have a patch for these. The bulk > of the work to enable this is (3), and I would structure this as a separate > pre-requisite patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14206) pytest/python 3 dtests breaks tests on 3.0 and 3.11
[ https://issues.apache.org/jira/browse/CASSANDRA-14206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355784#comment-16355784 ] Sam Tunnicliffe commented on CASSANDRA-14206: - +1 The only failures I see now look legit. The only nit is a debug statement left in conftest.py (line #375). Thanks! > pytest/python 3 dtests breaks tests on 3.0 and 3.11 > --- > > Key: CASSANDRA-14206 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14206 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Major > Attachments: loose_version.diff > > > In 2.7 it uses the cmp built-in to compare the list of version components > which accepts comparisons of strings and integers. In python 3 it manually > compares each using <, ==, and > which can fail if the the types don't match. > Switch to using our own comparison function that preserves the old behavior. > Additionally the new thrift bindings don't accept strings where they used to. > The strings need to be provided use utf-8 encoded bytes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14206) pytest/python 3 dtests breaks tests on 3.0 and 3.11
[ https://issues.apache.org/jira/browse/CASSANDRA-14206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-14206: Status: Ready to Commit (was: Patch Available) > pytest/python 3 dtests breaks tests on 3.0 and 3.11 > --- > > Key: CASSANDRA-14206 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14206 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Major > Attachments: loose_version.diff > > > In 2.7 it uses the cmp built-in to compare the list of version components > which accepts comparisons of strings and integers. In python 3 it manually > compares each using <, ==, and > which can fail if the the types don't match. > Switch to using our own comparison function that preserves the old behavior. > Additionally the new thrift bindings don't accept strings where they used to. > The strings need to be provided use utf-8 encoded bytes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14216) node map does not handle InetAddressAndPort correctly.
[ https://issues.apache.org/jira/browse/CASSANDRA-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Brosius updated CASSANDRA-14216: - Resolution: Fixed Status: Resolved (was: Ready to Commit) pushed to trunk as d0b34d383c20f5add8b8d7d454b4460aace0c939 findbugs/fb-contrib > node map does not handle InetAddressAndPort correctly. > -- > > Key: CASSANDRA-14216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14216 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Dave Brosius >Assignee: Dave Brosius >Priority: Trivial > Fix For: 4.0 > > Attachments: 14216.txt > > > Collection of node information in nodeMap does not use the correct types for > accessing data. Since these maps are keyed by Strings, they are not > metatype-safe, and so i can't be certain what data was meant to be in them. > I'm assuming it was meant that host and port information should be used, but > perhaps it's just host. > > I have created a patch assuming it's host and port info. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra git commit: node map does not handle InetAddressAndPort correctly.
Repository: cassandra Updated Branches: refs/heads/trunk 5c3e33b17 -> d0b34d383 node map does not handle InetAddressAndPort correctly. patch by dbrosius, reviewed by aweisberg for CASSANDRA-14216 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d0b34d38 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d0b34d38 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d0b34d38 Branch: refs/heads/trunk Commit: d0b34d383c20f5add8b8d7d454b4460aace0c939 Parents: 5c3e33b Author: Dave BrosiusAuthored: Wed Feb 7 11:46:47 2018 -0500 Committer: Dave Brosius Committed: Wed Feb 7 11:46:47 2018 -0500 -- src/java/org/apache/cassandra/service/StorageService.java| 8 .../apache/cassandra/tools/nodetool/SetHostStatWithPort.java | 2 +- src/java/org/apache/cassandra/tools/nodetool/Status.java | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0b34d38/src/java/org/apache/cassandra/service/StorageService.java -- diff --git a/src/java/org/apache/cassandra/service/StorageService.java b/src/java/org/apache/cassandra/service/StorageService.java index cda575a..16b6a81 100644 --- a/src/java/org/apache/cassandra/service/StorageService.java +++ b/src/java/org/apache/cassandra/service/StorageService.java @@ -4766,8 +4766,8 @@ public class StorageService extends NotificationBroadcasterSupport implements IE { InetAddressAndPort endpoint = tokenMetadata.getEndpoint(entry.getKey()); Float tokenOwnership = entry.getValue(); -if (nodeMap.containsKey(endpoint)) -nodeMap.put(endpoint.address, nodeMap.get(endpoint) + tokenOwnership); +if (nodeMap.containsKey(endpoint.address)) +nodeMap.put(endpoint.address, nodeMap.get(endpoint.address) + tokenOwnership); else nodeMap.put(endpoint.address, tokenOwnership); } @@ -4784,8 +4784,8 @@ public class StorageService extends NotificationBroadcasterSupport implements IE { InetAddressAndPort endpoint = tokenMetadata.getEndpoint(entry.getKey()); Float tokenOwnership = entry.getValue(); -if (nodeMap.containsKey(endpoint)) -nodeMap.put(endpoint.toString(), nodeMap.get(endpoint) + tokenOwnership); +if (nodeMap.containsKey(endpoint.toString())) +nodeMap.put(endpoint.toString(), nodeMap.get(endpoint.toString()) + tokenOwnership); else nodeMap.put(endpoint.toString(), tokenOwnership); } http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0b34d38/src/java/org/apache/cassandra/tools/nodetool/SetHostStatWithPort.java -- diff --git a/src/java/org/apache/cassandra/tools/nodetool/SetHostStatWithPort.java b/src/java/org/apache/cassandra/tools/nodetool/SetHostStatWithPort.java index 67cd464..6ac0258 100644 --- a/src/java/org/apache/cassandra/tools/nodetool/SetHostStatWithPort.java +++ b/src/java/org/apache/cassandra/tools/nodetool/SetHostStatWithPort.java @@ -50,7 +50,7 @@ public class SetHostStatWithPort implements Iterable public void add(String token, String host, Map ownerships) throws UnknownHostException { InetAddressAndPort endpoint = InetAddressAndPort.getByName(host); -Float owns = ownerships.get(endpoint); +Float owns = ownerships.get(endpoint.toString()); hostStats.add(new HostStatWithPort(token, endpoint, resolveIp, owns)); } } http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0b34d38/src/java/org/apache/cassandra/tools/nodetool/Status.java -- diff --git a/src/java/org/apache/cassandra/tools/nodetool/Status.java b/src/java/org/apache/cassandra/tools/nodetool/Status.java index 49724a5..10c7fdf 100644 --- a/src/java/org/apache/cassandra/tools/nodetool/Status.java +++ b/src/java/org/apache/cassandra/tools/nodetool/Status.java @@ -116,7 +116,7 @@ public class Status extends NodeToolCmd for (InetAddressAndPort endpoint : hostToTokens.keySet()) { -Float owns = ownerships.get(endpoint); +Float owns = ownerships.get(endpoint.toString()); List tokens = hostToTokens.get(endpoint); printNodeWithPort(endpoint.toString(), owns, tokens, hasEffectiveOwns, isTokenPerNode); } - To unsubscribe, e-mail:
[jira] [Commented] (CASSANDRA-14216) node map does not handle InetAddressAndPort correctly.
[ https://issues.apache.org/jira/browse/CASSANDRA-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355638#comment-16355638 ] Ariel Weisberg commented on CASSANDRA-14216: You are correct it's supposed to be the host and port string and we also do need to unwrap the address from the InetAddressAndPort. +1 Thanks for spotting this. Are you finding these things with manual testing, or a linter or some other static analysis tool? > node map does not handle InetAddressAndPort correctly. > -- > > Key: CASSANDRA-14216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14216 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Dave Brosius >Assignee: Dave Brosius >Priority: Trivial > Fix For: 4.0 > > Attachments: 14216.txt > > > Collection of node information in nodeMap does not use the correct types for > accessing data. Since these maps are keyed by Strings, they are not > metatype-safe, and so i can't be certain what data was meant to be in them. > I'm assuming it was meant that host and port information should be used, but > perhaps it's just host. > > I have created a patch assuming it's host and port info. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14216) node map does not handle InetAddressAndPort correctly.
[ https://issues.apache.org/jira/browse/CASSANDRA-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14216: --- Status: Ready to Commit (was: Patch Available) > node map does not handle InetAddressAndPort correctly. > -- > > Key: CASSANDRA-14216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14216 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Dave Brosius >Assignee: Dave Brosius >Priority: Trivial > Fix For: 4.0 > > Attachments: 14216.txt > > > Collection of node information in nodeMap does not use the correct types for > accessing data. Since these maps are keyed by Strings, they are not > metatype-safe, and so i can't be certain what data was meant to be in them. > I'm assuming it was meant that host and port information should be used, but > perhaps it's just host. > > I have created a patch assuming it's host and port info. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14216) node map does not handle InetAddressAndPort correctly.
[ https://issues.apache.org/jira/browse/CASSANDRA-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14216: --- Reviewer: Ariel Weisberg Status: Patch Available (was: Open) > node map does not handle InetAddressAndPort correctly. > -- > > Key: CASSANDRA-14216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14216 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Dave Brosius >Assignee: Dave Brosius >Priority: Trivial > Fix For: 4.0 > > Attachments: 14216.txt > > > Collection of node information in nodeMap does not use the correct types for > accessing data. Since these maps are keyed by Strings, they are not > metatype-safe, and so i can't be certain what data was meant to be in them. > I'm assuming it was meant that host and port information should be used, but > perhaps it's just host. > > I have created a patch assuming it's host and port info. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14194) Chain commit log marker potential performance regression in batch commit mode
[ https://issues.apache.org/jira/browse/CASSANDRA-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355427#comment-16355427 ] Jason Brown commented on CASSANDRA-14194: - [~jjirsa] I'm pretty sure this regression is related to the intersection of the group commit log patch (CASSANDRA-13530) and CASSANDRA-13987, thus only affecting trunk. Doesn't affect 3.0/3.11, so votes on those can proceed when they are ready. > Chain commit log marker potential performance regression in batch commit mode > - > > Key: CASSANDRA-14194 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14194 > Project: Cassandra > Issue Type: Bug > Components: Core, Testing >Reporter: Ariel Weisberg >Assignee: Jason Brown >Priority: Major > Attachments: jstack.txt > > > CASSANDRA-13987 modified how the commit log sync thread works. I noticed that > cql3.ViewTest and ViewBuilderTaskTest have been timing out, but only in > CircleCI. When I jstack in CircleCI what I see is that the commit log writer > thread is parked and the threads trying to append to the commit log are > blocked waiting on it. > I tested the commit before 13987 and it passed in CircleCI and then I tested > with 13987 and it timed out. I suspect this may be a general performance > regression and not just a CircleCI issue. > And this is with write barriers disabled (sync thread doesn't actually sync) > so it wasn't blocked in the filesystem. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14194) Chain commit log marker potential performance regression in batch commit mode
[ https://issues.apache.org/jira/browse/CASSANDRA-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355411#comment-16355411 ] Jason Brown commented on CASSANDRA-14194: - OK, so I've bisected both 3.0 and 3.11 locally, and I can't get them to timeout (or fail in other ways, outside of CASSANDRA-14219). I've also run them in circleci, and they do not timeout. Thus, to quote myself, I'm focusing on trunk for now. (Note: I did bisect trunk, and my initial CASSANDRA-13987 patch introduces the timeout) > Chain commit log marker potential performance regression in batch commit mode > - > > Key: CASSANDRA-14194 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14194 > Project: Cassandra > Issue Type: Bug > Components: Core, Testing >Reporter: Ariel Weisberg >Assignee: Jason Brown >Priority: Major > Attachments: jstack.txt > > > CASSANDRA-13987 modified how the commit log sync thread works. I noticed that > cql3.ViewTest and ViewBuilderTaskTest have been timing out, but only in > CircleCI. When I jstack in CircleCI what I see is that the commit log writer > thread is parked and the threads trying to append to the commit log are > blocked waiting on it. > I tested the commit before 13987 and it passed in CircleCI and then I tested > with 13987 and it timed out. I suspect this may be a general performance > regression and not just a CircleCI issue. > And this is with write barriers disabled (sync thread doesn't actually sync) > so it wasn't blocked in the filesystem. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14219) Change to AlterTableStatement logging breaks MView tests
[ https://issues.apache.org/jira/browse/CASSANDRA-14219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-14219: Description: looks like [~dbrosius]'s ninja commit {{7df36056b12a13b60097b7a9a4f8155a1d02ff62}} to improve the logging of {{AlterTableStatement}} breaks some MView tests that check the exception message. I see about six failed tests from {{ViewComplexTest}} that have messages similar to this: {noformat} junit.framework.AssertionFailedError: Expected error message to contain 'Cannot drop column m on base table with materialized views', but got 'Cannot drop column m on base table table_6 with materialized views.'{noformat} was:looks like [~dbrosius]'s ninja commit {{7df36056b12a13b60097b7a9a4f8155a1d02ff62}} to improve the logging of {{AlterTableStatement}} breaks some MView tests that check the exception message. > Change to AlterTableStatement logging breaks MView tests > > > Key: CASSANDRA-14219 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14219 > Project: Cassandra > Issue Type: Bug >Reporter: Jason Brown >Assignee: Dave Brosius >Priority: Major > Fix For: 3.0.x, 3.11.x, 4.x > > > looks like [~dbrosius]'s ninja commit > {{7df36056b12a13b60097b7a9a4f8155a1d02ff62}} to improve the logging of > {{AlterTableStatement}} breaks some MView tests that check the exception > message. I see about six failed tests from {{ViewComplexTest}} that have > messages similar to this: > {noformat} > junit.framework.AssertionFailedError: Expected error message to contain > 'Cannot drop column m on base table with materialized views', but got 'Cannot > drop column m on base table table_6 with materialized views.'{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14219) Change to AlterTableStatement logging breaks MView tests
Jason Brown created CASSANDRA-14219: --- Summary: Change to AlterTableStatement logging breaks MView tests Key: CASSANDRA-14219 URL: https://issues.apache.org/jira/browse/CASSANDRA-14219 Project: Cassandra Issue Type: Bug Reporter: Jason Brown Assignee: Dave Brosius Fix For: 3.0.x, 3.11.x, 4.x looks like [~dbrosius]'s ninja commit {{7df36056b12a13b60097b7a9a4f8155a1d02ff62}} to improve the logging of {{AlterTableStatement}} breaks some MView tests that check the exception message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13314) Config file based SSL settings
[ https://issues.apache.org/jira/browse/CASSANDRA-13314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355382#comment-16355382 ] Jason Brown commented on CASSANDRA-13314: - [~ronblechman] please open a new Jira for cert validation/revocation stuffs. It's a topic with discussing, and [~djoshi3] and I may have some ideas on that topic, as well. > Config file based SSL settings > -- > > Key: CASSANDRA-13314 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13314 > Project: Cassandra > Issue Type: Improvement > Components: Configuration, Tools >Reporter: Stefan Podkowinski >Assignee: Stefan Podkowinski >Priority: Minor > Fix For: 4.x > > > As follow up of CASSANDRA-13259, I'd like to continue discussing how we can > make SSL less awkward to use and further move SSL related code out of our > code base. Currently we construct our own SSLContext in SSLFactory based on > EncryptionOptions passed by the MessagingService or any individual tool where > we need to offer SSL support. This leads to a situation where the user has > not only to learn how to enable the correct settings in cassandra.yaml, but > these settings must also be reflected in each tool's own command line > options. As argued in CASSANDRA-13259, these settings could be done as well > by setting the appropriate system and security properties > ([overview|http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization]) > and we should just point the user to the right files to do that (jvm.options > and java.security) and make sure that daemon and all affected tools will > source them. > Since giving this a quick try on my WIP branch, I've noticed the following > issues in doing so: > * Keystore passwords will show up in process list > (-Djavax.net.ssl.keyStorePassword=..). We should keep the password setting in > cassandra.yaml and clis and do a System.setProperty() if they have been > provided. > * It's only possible to configure settings for a single default > key-/truststore. Since we currently allow configuring both > ServerEncryptionOptions and ClientEncryptionOptions with different settings, > we'd have to make this a breaking change. I don't really see why you would > want to use different stores for node-to-node and node-to-client, but that > wouldn't be possible anymore. > * This would probably only make sense if we really remove the affected CLI > options, or we'll end up with just another way to configure this stuff. This > will break existing scripts and obsolete existing documentation. > Any opinions? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14218) Deprecate Throwables.propagate usage
[ https://issues.apache.org/jira/browse/CASSANDRA-14218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-14218: Fix Version/s: 4.x > Deprecate Throwables.propagate usage > > > Key: CASSANDRA-14218 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14218 > Project: Cassandra > Issue Type: Improvement > Components: Libraries >Reporter: Romain Hardouin >Priority: Minor > Labels: lhf > Fix For: 4.x > > > Google deciced to deprecate guava {{Throwables.propagate}} method: > * [Why we deprecated > Throwables.propagate|https://github.com/google/guava/wiki/Why-we-deprecated-Throwables.propagate] > * [Documentation > update|https://github.com/google/guava/wiki/ThrowablesExplained/_compare/92190ee7e37d334fa5fcdb6db8d0f43a2fdf02e1...226a3060445716d479981e606f589c99eee517ca] > We have 35 occurences in the trunk: > {code:java} > $ rg -c 'Throwables.propagate' * > src/java/org/apache/cassandra/streaming/StreamReader.java:1 > src/java/org/apache/cassandra/streaming/StreamTransferTask.java:1 > src/java/org/apache/cassandra/db/SnapshotDetailsTabularData.java:1 > src/java/org/apache/cassandra/db/Memtable.java:1 > src/java/org/apache/cassandra/db/ColumnFamilyStore.java:4 > src/java/org/apache/cassandra/cache/ChunkCache.java:2 > src/java/org/apache/cassandra/utils/WrappedRunnable.java:1 > src/java/org/apache/cassandra/hints/Hint.java:1 > src/java/org/apache/cassandra/tools/LoaderOptions.java:1 > src/java/org/apache/cassandra/tools/SSTableOfflineRelevel.java:1 > src/java/org/apache/cassandra/streaming/management/ProgressInfoCompositeData.java:3 > src/java/org/apache/cassandra/streaming/management/StreamStateCompositeData.java:2 > src/java/org/apache/cassandra/streaming/management/StreamSummaryCompositeData.java:2 > src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java:1 > src/java/org/apache/cassandra/db/compaction/Scrubber.java:1 > src/java/org/apache/cassandra/db/compaction/Verifier.java:1 > src/java/org/apache/cassandra/db/compaction/CompactionHistoryTabularData.java:1 > src/java/org/apache/cassandra/db/compaction/Upgrader.java:1 > src/java/org/apache/cassandra/io/compress/CompressionMetadata.java:1 > src/java/org/apache/cassandra/streaming/management/SessionCompleteEventCompositeData.java:2 > src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java:1 > src/java/org/apache/cassandra/io/sstable/ISSTableScanner.java:1 > src/java/org/apache/cassandra/streaming/management/SessionInfoCompositeData.java:3 > src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java:1 > {code} > I don't know if we want to remove all usages but we should at least check > author's intention for each usage and refactor if needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14218) Deprecate Throwables.propagate usage
[ https://issues.apache.org/jira/browse/CASSANDRA-14218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-14218: Labels: lhf (was: ) > Deprecate Throwables.propagate usage > > > Key: CASSANDRA-14218 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14218 > Project: Cassandra > Issue Type: Improvement > Components: Libraries >Reporter: Romain Hardouin >Priority: Minor > Labels: lhf > Fix For: 4.x > > > Google deciced to deprecate guava {{Throwables.propagate}} method: > * [Why we deprecated > Throwables.propagate|https://github.com/google/guava/wiki/Why-we-deprecated-Throwables.propagate] > * [Documentation > update|https://github.com/google/guava/wiki/ThrowablesExplained/_compare/92190ee7e37d334fa5fcdb6db8d0f43a2fdf02e1...226a3060445716d479981e606f589c99eee517ca] > We have 35 occurences in the trunk: > {code:java} > $ rg -c 'Throwables.propagate' * > src/java/org/apache/cassandra/streaming/StreamReader.java:1 > src/java/org/apache/cassandra/streaming/StreamTransferTask.java:1 > src/java/org/apache/cassandra/db/SnapshotDetailsTabularData.java:1 > src/java/org/apache/cassandra/db/Memtable.java:1 > src/java/org/apache/cassandra/db/ColumnFamilyStore.java:4 > src/java/org/apache/cassandra/cache/ChunkCache.java:2 > src/java/org/apache/cassandra/utils/WrappedRunnable.java:1 > src/java/org/apache/cassandra/hints/Hint.java:1 > src/java/org/apache/cassandra/tools/LoaderOptions.java:1 > src/java/org/apache/cassandra/tools/SSTableOfflineRelevel.java:1 > src/java/org/apache/cassandra/streaming/management/ProgressInfoCompositeData.java:3 > src/java/org/apache/cassandra/streaming/management/StreamStateCompositeData.java:2 > src/java/org/apache/cassandra/streaming/management/StreamSummaryCompositeData.java:2 > src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java:1 > src/java/org/apache/cassandra/db/compaction/Scrubber.java:1 > src/java/org/apache/cassandra/db/compaction/Verifier.java:1 > src/java/org/apache/cassandra/db/compaction/CompactionHistoryTabularData.java:1 > src/java/org/apache/cassandra/db/compaction/Upgrader.java:1 > src/java/org/apache/cassandra/io/compress/CompressionMetadata.java:1 > src/java/org/apache/cassandra/streaming/management/SessionCompleteEventCompositeData.java:2 > src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java:1 > src/java/org/apache/cassandra/io/sstable/ISSTableScanner.java:1 > src/java/org/apache/cassandra/streaming/management/SessionInfoCompositeData.java:3 > src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java:1 > {code} > I don't know if we want to remove all usages but we should at least check > author's intention for each usage and refactor if needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14218) Deprecate Throwables.propagate usage
Romain Hardouin created CASSANDRA-14218: --- Summary: Deprecate Throwables.propagate usage Key: CASSANDRA-14218 URL: https://issues.apache.org/jira/browse/CASSANDRA-14218 Project: Cassandra Issue Type: Improvement Components: Libraries Reporter: Romain Hardouin Google deciced to deprecate guava {{Throwables.propagate}} method: * [Why we deprecated Throwables.propagate|https://github.com/google/guava/wiki/Why-we-deprecated-Throwables.propagate] * [Documentation update|https://github.com/google/guava/wiki/ThrowablesExplained/_compare/92190ee7e37d334fa5fcdb6db8d0f43a2fdf02e1...226a3060445716d479981e606f589c99eee517ca] We have 35 occurences in the trunk: {code:java} $ rg -c 'Throwables.propagate' * src/java/org/apache/cassandra/streaming/StreamReader.java:1 src/java/org/apache/cassandra/streaming/StreamTransferTask.java:1 src/java/org/apache/cassandra/db/SnapshotDetailsTabularData.java:1 src/java/org/apache/cassandra/db/Memtable.java:1 src/java/org/apache/cassandra/db/ColumnFamilyStore.java:4 src/java/org/apache/cassandra/cache/ChunkCache.java:2 src/java/org/apache/cassandra/utils/WrappedRunnable.java:1 src/java/org/apache/cassandra/hints/Hint.java:1 src/java/org/apache/cassandra/tools/LoaderOptions.java:1 src/java/org/apache/cassandra/tools/SSTableOfflineRelevel.java:1 src/java/org/apache/cassandra/streaming/management/ProgressInfoCompositeData.java:3 src/java/org/apache/cassandra/streaming/management/StreamStateCompositeData.java:2 src/java/org/apache/cassandra/streaming/management/StreamSummaryCompositeData.java:2 src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java:1 src/java/org/apache/cassandra/db/compaction/Scrubber.java:1 src/java/org/apache/cassandra/db/compaction/Verifier.java:1 src/java/org/apache/cassandra/db/compaction/CompactionHistoryTabularData.java:1 src/java/org/apache/cassandra/db/compaction/Upgrader.java:1 src/java/org/apache/cassandra/io/compress/CompressionMetadata.java:1 src/java/org/apache/cassandra/streaming/management/SessionCompleteEventCompositeData.java:2 src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java:1 src/java/org/apache/cassandra/io/sstable/ISSTableScanner.java:1 src/java/org/apache/cassandra/streaming/management/SessionInfoCompositeData.java:3 src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java:1 {code} I don't know if we want to remove all usages but we should at least check author's intention for each usage and refactor if needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12743) Assertion error while running compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355297#comment-16355297 ] Tom van der Woerdt commented on CASSANDRA-12743: Found one on 3.11.1 as well: {code:java} ERROR [CompactionExecutor:1696] 2018-02-07 09:58:10,510 CassandraDaemon.java:228 - Exception in thread Thread[CompactionExecutor:1696,1,main] java.lang.AssertionError: null at org.apache.cassandra.io.compress.CompressionMetadata$Chunk.(CompressionMetadata.java:474) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:239) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:163) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:73) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:61) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:362) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:290) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:179) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:134) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:98) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:141) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:201) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:85) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61) ~[apache-cassandra-3.11.1.jar:3.11.1] at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:268) ~[apache-cassandra-3.11.1.jar:3.11.1] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.8.0_152] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.8.0_152] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.8.0_152] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.8.0_152] at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81) [apache-cassandra-3.11.1.jar:3.11.1] at java.lang.Thread.run(Unknown Source) ~[na:1.8.0_152] {code} > Assertion error while running compaction > - > > Key: CASSANDRA-12743 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12743 > Project: Cassandra > Issue Type: Bug > Components: Compaction > Environment: unix >Reporter: Jean-Baptiste Le Duigou >Priority: Major > > While running compaction I run into an error sometimes : > {noformat} > nodetool compact > error: null > -- StackTrace -- > java.lang.AssertionError > at > org.apache.cassandra.io.compress.CompressionMetadata$Chunk.(CompressionMetadata.java:463) > at > org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:228) > at > org.apache.cassandra.io.util.CompressedSegmentedFile.createMappedSegments(CompressedSegmentedFile.java:80) > at > org.apache.cassandra.io.util.CompressedPoolingSegmentedFile.(CompressedPoolingSegmentedFile.java:38) > at > org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101) > at > org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:198) > at > org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:315) > at > org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:171) > at > org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:116) > at
[jira] [Updated] (CASSANDRA-13933) Handle mutateRepaired failure in nodetool verify
[ https://issues.apache.org/jira/browse/CASSANDRA-13933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-13933: Resolution: Fixed Fix Version/s: 4.0 3.11.2 3.0.16 Status: Resolved (was: Patch Available) committed as {{7885a703d6dae8c3c6e5a6af632c6a23342593fc}} note that I decided to keep the method signatures in 3.0 and 3.11 to avoid breaking anyone depending on this, the {{throws IOException}} is removed in trunk thanks for the patch! > Handle mutateRepaired failure in nodetool verify > > > Key: CASSANDRA-13933 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13933 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Sumanth Pasupuleti >Priority: Major > Labels: lhf > Fix For: 3.0.16, 3.11.2, 4.0 > > Attachments: CASSANDRA-13933-3.0.txt, CASSANDRA-13933-3.11.txt, > CASSANDRA-13933-trunk.txt > > > See comment here: > https://issues.apache.org/jira/browse/CASSANDRA-13922?focusedCommentId=16189875=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16189875 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[1/6] cassandra git commit: Handle error when mutating repairedAt in nodetool verify
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 7df36056b -> 7885a703d refs/heads/cassandra-3.11 de6c62dd6 -> d36919096 refs/heads/trunk f2fc2e967 -> 5c3e33b17 Handle error when mutating repairedAt in nodetool verify Patch by Sumanth Pasupuleti; reviewed by marcuse for CASSANDRA-13933 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7885a703 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7885a703 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7885a703 Branch: refs/heads/cassandra-3.0 Commit: 7885a703d6dae8c3c6e5a6af632c6a23342593fc Parents: 7df3605 Author: Sumanth PasupuletiAuthored: Wed Feb 7 08:54:53 2018 +0100 Committer: Marcus Eriksson Committed: Wed Feb 7 09:04:45 2018 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/compaction/Verifier.java | 11 ++- 2 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7885a703/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 0eaab6e..4e8f2ac 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.16 + * Handle failure when mutating repaired status in Verifier (CASSANDRA-13933) * Close socket on error during connect on OutboundTcpConnection (CASSANDRA-9630) * Set encoding for javadoc generation (CASSANDRA-14154) * Fix index target computation for dense composite tables with dropped compact storage (CASSANDRA-14104) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7885a703/src/java/org/apache/cassandra/db/compaction/Verifier.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/Verifier.java b/src/java/org/apache/cassandra/db/compaction/Verifier.java index 68088b3..86bc377 100644 --- a/src/java/org/apache/cassandra/db/compaction/Verifier.java +++ b/src/java/org/apache/cassandra/db/compaction/Verifier.java @@ -255,7 +255,16 @@ public class Verifier implements Closeable private void markAndThrow(boolean mutateRepaired) throws IOException { if (mutateRepaired) // if we are able to mutate repaired flag, an incremental repair should be enough - sstable.descriptor.getMetadataSerializer().mutateRepairedAt(sstable.descriptor, ActiveRepairService.UNREPAIRED_SSTABLE); +{ +try +{ + sstable.descriptor.getMetadataSerializer().mutateRepairedAt(sstable.descriptor, ActiveRepairService.UNREPAIRED_SSTABLE); +} +catch(IOException ioe) +{ +outputHandler.output("Error mutating repairedAt for SSTable " + sstable.getFilename() + ", as part of markAndThrow"); +} +} throw new CorruptSSTableException(new Exception(String.format("Invalid SSTable %s, please force %srepair", sstable.getFilename(), mutateRepaired ? "" : "a full ")), sstable.getFilename()); } - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5c3e33b1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5c3e33b1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5c3e33b1 Branch: refs/heads/trunk Commit: 5c3e33b17c59a1f8e4eb8eee60cf4ea59accb6e0 Parents: f2fc2e9 d369190 Author: Marcus ErikssonAuthored: Wed Feb 7 09:10:33 2018 +0100 Committer: Marcus Eriksson Committed: Wed Feb 7 09:10:33 2018 +0100 -- CHANGES.txt| 1 + .../cassandra/db/compaction/CompactionManager.java | 4 ++-- .../apache/cassandra/db/compaction/Verifier.java | 17 + test/unit/org/apache/cassandra/db/VerifyTest.java | 16 4 files changed, 24 insertions(+), 14 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/5c3e33b1/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/5c3e33b1/src/java/org/apache/cassandra/db/compaction/CompactionManager.java -- diff --cc src/java/org/apache/cassandra/db/compaction/CompactionManager.java index 046ae29,e158982..c5e5958 --- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java +++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java @@@ -410,7 -398,7 +410,7 @@@ public class CompactionManager implemen } @Override --public void execute(LifecycleTransaction input) throws IOException ++public void execute(LifecycleTransaction input) { verifyOne(cfs, input.onlyOne(), extendedVerify); } @@@ -985,7 -986,7 +985,7 @@@ } } --private void verifyOne(ColumnFamilyStore cfs, SSTableReader sstable, boolean extendedVerify) throws IOException ++private void verifyOne(ColumnFamilyStore cfs, SSTableReader sstable, boolean extendedVerify) { CompactionInfo.Holder verifyInfo = null; http://git-wip-us.apache.org/repos/asf/cassandra/blob/5c3e33b1/src/java/org/apache/cassandra/db/compaction/Verifier.java -- diff --cc src/java/org/apache/cassandra/db/compaction/Verifier.java index af00915,44a5850..5a2a828 --- a/src/java/org/apache/cassandra/db/compaction/Verifier.java +++ b/src/java/org/apache/cassandra/db/compaction/Verifier.java @@@ -87,7 -86,7 +87,7 @@@ public class Verifier implements Closea this.verifyInfo = new VerifyInfo(dataFile, sstable); } --public void verify(boolean extended) throws IOException ++public void verify(boolean extended) { long rowStart = 0; @@@ -248,15 -248,24 +248,24 @@@ throw (Error) th; } --private void markAndThrow() throws IOException ++private void markAndThrow() { markAndThrow(true); } --private void markAndThrow(boolean mutateRepaired) throws IOException ++private void markAndThrow(boolean mutateRepaired) { if (mutateRepaired) // if we are able to mutate repaired flag, an incremental repair should be enough - sstable.descriptor.getMetadataSerializer().mutateRepaired(sstable.descriptor, ActiveRepairService.UNREPAIRED_SSTABLE, sstable.getSSTableMetadata().pendingRepair); + { + try + { - sstable.descriptor.getMetadataSerializer().mutateRepairedAt(sstable.descriptor, ActiveRepairService.UNREPAIRED_SSTABLE); ++ sstable.descriptor.getMetadataSerializer().mutateRepaired(sstable.descriptor, ActiveRepairService.UNREPAIRED_SSTABLE, sstable.getSSTableMetadata().pendingRepair); + } + catch(IOException ioe) + { + outputHandler.output("Error mutating repairedAt for SSTable " + sstable.getFilename() + ", as part of markAndThrow"); + } + } throw new CorruptSSTableException(new Exception(String.format("Invalid SSTable %s, please force %srepair", sstable.getFilename(), mutateRepaired ? "" : "a full ")), sstable.getFilename()); } http://git-wip-us.apache.org/repos/asf/cassandra/blob/5c3e33b1/test/unit/org/apache/cassandra/db/VerifyTest.java -- diff --cc test/unit/org/apache/cassandra/db/VerifyTest.java index 94fb9ec,6808c47..2eb741c --- a/test/unit/org/apache/cassandra/db/VerifyTest.java +++ b/test/unit/org/apache/cassandra/db/VerifyTest.java @@@ -97,7 -94,7 +97,7 @@@ public class VerifyTes @Test --public void
[2/6] cassandra git commit: Handle error when mutating repairedAt in nodetool verify
Handle error when mutating repairedAt in nodetool verify Patch by Sumanth Pasupuleti; reviewed by marcuse for CASSANDRA-13933 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7885a703 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7885a703 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7885a703 Branch: refs/heads/cassandra-3.11 Commit: 7885a703d6dae8c3c6e5a6af632c6a23342593fc Parents: 7df3605 Author: Sumanth PasupuletiAuthored: Wed Feb 7 08:54:53 2018 +0100 Committer: Marcus Eriksson Committed: Wed Feb 7 09:04:45 2018 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/compaction/Verifier.java | 11 ++- 2 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7885a703/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 0eaab6e..4e8f2ac 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.16 + * Handle failure when mutating repaired status in Verifier (CASSANDRA-13933) * Close socket on error during connect on OutboundTcpConnection (CASSANDRA-9630) * Set encoding for javadoc generation (CASSANDRA-14154) * Fix index target computation for dense composite tables with dropped compact storage (CASSANDRA-14104) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7885a703/src/java/org/apache/cassandra/db/compaction/Verifier.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/Verifier.java b/src/java/org/apache/cassandra/db/compaction/Verifier.java index 68088b3..86bc377 100644 --- a/src/java/org/apache/cassandra/db/compaction/Verifier.java +++ b/src/java/org/apache/cassandra/db/compaction/Verifier.java @@ -255,7 +255,16 @@ public class Verifier implements Closeable private void markAndThrow(boolean mutateRepaired) throws IOException { if (mutateRepaired) // if we are able to mutate repaired flag, an incremental repair should be enough - sstable.descriptor.getMetadataSerializer().mutateRepairedAt(sstable.descriptor, ActiveRepairService.UNREPAIRED_SSTABLE); +{ +try +{ + sstable.descriptor.getMetadataSerializer().mutateRepairedAt(sstable.descriptor, ActiveRepairService.UNREPAIRED_SSTABLE); +} +catch(IOException ioe) +{ +outputHandler.output("Error mutating repairedAt for SSTable " + sstable.getFilename() + ", as part of markAndThrow"); +} +} throw new CorruptSSTableException(new Exception(String.format("Invalid SSTable %s, please force %srepair", sstable.getFilename(), mutateRepaired ? "" : "a full ")), sstable.getFilename()); } - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d3691909 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d3691909 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d3691909 Branch: refs/heads/cassandra-3.11 Commit: d369190964be9a6011f707a7e584446f9ec57a73 Parents: de6c62d 7885a70 Author: Marcus ErikssonAuthored: Wed Feb 7 09:06:19 2018 +0100 Committer: Marcus Eriksson Committed: Wed Feb 7 09:06:19 2018 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/compaction/Verifier.java | 11 ++- 2 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3691909/CHANGES.txt -- diff --cc CHANGES.txt index 30ca8a8,4e8f2ac..8c0d8f0 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,25 -1,6 +1,26 @@@ -3.0.16 - * Handle failure when mutating repaired status in Verifier (CASSANDRA-13933) +3.11.2 + * Remove dependencies on JVM internal classes from JMXServerUtils (CASSANDRA-14173) + * Add DEFAULT, UNSET, MBEAN and MBEANS to `ReservedKeywords` (CASSANDRA-14205) + * Add Unittest for schema migration fix (CASSANDRA-14140) + * Print correct snitch info from nodetool describecluster (CASSANDRA-13528) * Close socket on error during connect on OutboundTcpConnection (CASSANDRA-9630) + * Enable CDC unittest (CASSANDRA-14141) + * Acquire read lock before accessing CompactionStrategyManager fields (CASSANDRA-14139) + * Split CommitLogStressTest to avoid timeout (CASSANDRA-14143) + * Avoid invalidating disk boundaries unnecessarily (CASSANDRA-14083) + * Avoid exposing compaction strategy index externally (CASSANDRA-14082) + * Prevent continuous schema exchange between 3.0 and 3.11 nodes (CASSANDRA-14109) + * Fix imbalanced disks when replacing node with same address with JBOD (CASSANDRA-14084) + * Reload compaction strategies when disk boundaries are invalidated (CASSANDRA-13948) + * Remove OpenJDK log warning (CASSANDRA-13916) + * Prevent compaction strategies from looping indefinitely (CASSANDRA-14079) + * Cache disk boundaries (CASSANDRA-13215) + * Add asm jar to build.xml for maven builds (CASSANDRA-11193) + * Round buffer size to powers of 2 for the chunk cache (CASSANDRA-13897) + * Update jackson JSON jars (CASSANDRA-13949) + * Avoid locks when checking LCS fanout and if we should defrag (CASSANDRA-13930) +Merged from 3.0: ++ * Handle failure when mutating repaired status in Verifier (CASSANDRA-13933) * Set encoding for javadoc generation (CASSANDRA-14154) * Fix index target computation for dense composite tables with dropped compact storage (CASSANDRA-14104) * Improve commit log chain marker updating (CASSANDRA-14108) http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3691909/src/java/org/apache/cassandra/db/compaction/Verifier.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[3/6] cassandra git commit: Handle error when mutating repairedAt in nodetool verify
Handle error when mutating repairedAt in nodetool verify Patch by Sumanth Pasupuleti; reviewed by marcuse for CASSANDRA-13933 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7885a703 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7885a703 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7885a703 Branch: refs/heads/trunk Commit: 7885a703d6dae8c3c6e5a6af632c6a23342593fc Parents: 7df3605 Author: Sumanth PasupuletiAuthored: Wed Feb 7 08:54:53 2018 +0100 Committer: Marcus Eriksson Committed: Wed Feb 7 09:04:45 2018 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/compaction/Verifier.java | 11 ++- 2 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7885a703/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 0eaab6e..4e8f2ac 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.16 + * Handle failure when mutating repaired status in Verifier (CASSANDRA-13933) * Close socket on error during connect on OutboundTcpConnection (CASSANDRA-9630) * Set encoding for javadoc generation (CASSANDRA-14154) * Fix index target computation for dense composite tables with dropped compact storage (CASSANDRA-14104) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7885a703/src/java/org/apache/cassandra/db/compaction/Verifier.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/Verifier.java b/src/java/org/apache/cassandra/db/compaction/Verifier.java index 68088b3..86bc377 100644 --- a/src/java/org/apache/cassandra/db/compaction/Verifier.java +++ b/src/java/org/apache/cassandra/db/compaction/Verifier.java @@ -255,7 +255,16 @@ public class Verifier implements Closeable private void markAndThrow(boolean mutateRepaired) throws IOException { if (mutateRepaired) // if we are able to mutate repaired flag, an incremental repair should be enough - sstable.descriptor.getMetadataSerializer().mutateRepairedAt(sstable.descriptor, ActiveRepairService.UNREPAIRED_SSTABLE); +{ +try +{ + sstable.descriptor.getMetadataSerializer().mutateRepairedAt(sstable.descriptor, ActiveRepairService.UNREPAIRED_SSTABLE); +} +catch(IOException ioe) +{ +outputHandler.output("Error mutating repairedAt for SSTable " + sstable.getFilename() + ", as part of markAndThrow"); +} +} throw new CorruptSSTableException(new Exception(String.format("Invalid SSTable %s, please force %srepair", sstable.getFilename(), mutateRepaired ? "" : "a full ")), sstable.getFilename()); } - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d3691909 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d3691909 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d3691909 Branch: refs/heads/trunk Commit: d369190964be9a6011f707a7e584446f9ec57a73 Parents: de6c62d 7885a70 Author: Marcus ErikssonAuthored: Wed Feb 7 09:06:19 2018 +0100 Committer: Marcus Eriksson Committed: Wed Feb 7 09:06:19 2018 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/compaction/Verifier.java | 11 ++- 2 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3691909/CHANGES.txt -- diff --cc CHANGES.txt index 30ca8a8,4e8f2ac..8c0d8f0 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,25 -1,6 +1,26 @@@ -3.0.16 - * Handle failure when mutating repaired status in Verifier (CASSANDRA-13933) +3.11.2 + * Remove dependencies on JVM internal classes from JMXServerUtils (CASSANDRA-14173) + * Add DEFAULT, UNSET, MBEAN and MBEANS to `ReservedKeywords` (CASSANDRA-14205) + * Add Unittest for schema migration fix (CASSANDRA-14140) + * Print correct snitch info from nodetool describecluster (CASSANDRA-13528) * Close socket on error during connect on OutboundTcpConnection (CASSANDRA-9630) + * Enable CDC unittest (CASSANDRA-14141) + * Acquire read lock before accessing CompactionStrategyManager fields (CASSANDRA-14139) + * Split CommitLogStressTest to avoid timeout (CASSANDRA-14143) + * Avoid invalidating disk boundaries unnecessarily (CASSANDRA-14083) + * Avoid exposing compaction strategy index externally (CASSANDRA-14082) + * Prevent continuous schema exchange between 3.0 and 3.11 nodes (CASSANDRA-14109) + * Fix imbalanced disks when replacing node with same address with JBOD (CASSANDRA-14084) + * Reload compaction strategies when disk boundaries are invalidated (CASSANDRA-13948) + * Remove OpenJDK log warning (CASSANDRA-13916) + * Prevent compaction strategies from looping indefinitely (CASSANDRA-14079) + * Cache disk boundaries (CASSANDRA-13215) + * Add asm jar to build.xml for maven builds (CASSANDRA-11193) + * Round buffer size to powers of 2 for the chunk cache (CASSANDRA-13897) + * Update jackson JSON jars (CASSANDRA-13949) + * Avoid locks when checking LCS fanout and if we should defrag (CASSANDRA-13930) +Merged from 3.0: ++ * Handle failure when mutating repaired status in Verifier (CASSANDRA-13933) * Set encoding for javadoc generation (CASSANDRA-14154) * Fix index target computation for dense composite tables with dropped compact storage (CASSANDRA-14104) * Improve commit log chain marker updating (CASSANDRA-14108) http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3691909/src/java/org/apache/cassandra/db/compaction/Verifier.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org