[jira] [Commented] (CASSANDRA-9633) Add ability to encrypt sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043976#comment-16043976 ] Praveen Agrawal commented on CASSANDRA-9633: Hi Jason. I care about this feature and can contribute. Can you please point to the branch where your current changes are? Thanks. > Add ability to encrypt sstables > --- > > Key: CASSANDRA-9633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9633 > Project: Cassandra > Issue Type: New Feature >Reporter: Jason Brown >Assignee: Jason Brown > Labels: encryption, security, sstable > Fix For: 4.x > > > Add option to allow encrypting of sstables. > I have a version of this functionality built on cassandra 2.0 that > piggy-backs on the existing sstable compression functionality and ICompressor > interface (similar in nature to what DataStax Enterprise does). However, if > we're adding the feature to the main OSS product, I'm not sure if we want to > use the pluggable compression framework or if it's worth investigating a > different path. I think there's a lot of upside in reusing the sstable > compression scheme, but perhaps add a new component in cqlsh for table > encryption and a corresponding field in CFMD. > Encryption configuration in the yaml can use the same mechanism as > CASSANDRA-6018 (which is currently pending internal review). -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12744) Randomness of stress distributions is not good
[ https://issues.apache.org/jira/browse/CASSANDRA-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043871#comment-16043871 ] Ben Slater commented on CASSANDRA-12744: One extra note for future searching: There is a fair chance this fix will change the workload quite substantially in a number of scenarios. So, if you want to compare benchmarks make sure you don't compare results from stress with this fix vs stress without this fix. > Randomness of stress distributions is not good > -- > > Key: CASSANDRA-12744 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12744 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: T Jake Luciani >Assignee: Ben Slater >Priority: Minor > Labels: stress > Fix For: 4.0 > > Attachments: CASSANDRA_12744_SeedManager_changes-trunk.patch > > > The randomness of our distributions is pretty bad. We are using the > JDKRandomGenerator() but in testing of uniform(1..3) we see for 100 > iterations it's only outputting 3. If you bump it to 10k it hits all 3 > values. > I made a change to just use the default commons math random generator and now > see all 3 values for n=10 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13585) NPE in IR cleanup when columnfamily has no sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043834#comment-16043834
]
Jeff Jirsa edited comment on CASSANDRA-13585 at 6/9/17 2:28 AM:
|| [trunk branch|https://github.com/jeffjirsa/cassandra/tree/cassandra-13585] |
[testall|https://circleci.com/gh/jeffjirsa/cassandra/tree/cassandra-13585]
|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/81/]
||
was (Author: jjirsa):
|| [trunk branch|https://github.com/jeffjirsa/cassandra/tree/cassandra-13585] |
[testall|https://circleci.com/gh/jeffjirsa/cassandra/182]
|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/81/]
||
> NPE in IR cleanup when columnfamily has no sstables
> ---
>
> Key: CASSANDRA-13585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13585
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Labels: incremental_repair
> Fix For: 4.x
>
>
> On {{PendingAntiCompaction::abort()}} , we try to release refs to sstables
> and a lifecycle transaction that can be null if there are no sstables in the
> table.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13585) NPE in IR cleanup when columnfamily has no sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-13585:
---
Attachment: (was:
0001-NPE-in-IR-cleanup-when-columnfamily-has-no-sstables.patch)
> NPE in IR cleanup when columnfamily has no sstables
> ---
>
> Key: CASSANDRA-13585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13585
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Labels: incremental_repair
> Fix For: 4.x
>
>
> On {{PendingAntiCompaction::abort()}} , we try to release refs to sstables
> and a lifecycle transaction that can be null if there are no sstables in the
> table.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13585) NPE in IR cleanup when columnfamily has no sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043836#comment-16043836
]
Jeff Jirsa commented on CASSANDRA-13585:
Either [~yukim] , [~krummas] or [~bdeggleston] up for quick review?
> NPE in IR cleanup when columnfamily has no sstables
> ---
>
> Key: CASSANDRA-13585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13585
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Labels: incremental_repair
> Fix For: 4.x
>
> Attachments:
> 0001-NPE-in-IR-cleanup-when-columnfamily-has-no-sstables.patch
>
>
> On {{PendingAntiCompaction::abort()}} , we try to release refs to sstables
> and a lifecycle transaction that can be null if there are no sstables in the
> table.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13585) NPE in IR cleanup when columnfamily has no sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-13585:
---
Attachment: 0001-NPE-in-IR-cleanup-when-columnfamily-has-no-sstables.patch
|| [trunk branch|https://github.com/jeffjirsa/cassandra/tree/cassandra-13585] |
[testall|https://circleci.com/gh/jeffjirsa/cassandra/182]
|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/81/]
||
> NPE in IR cleanup when columnfamily has no sstables
> ---
>
> Key: CASSANDRA-13585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13585
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Labels: incremental_repair
> Fix For: 4.x
>
> Attachments:
> 0001-NPE-in-IR-cleanup-when-columnfamily-has-no-sstables.patch
>
>
> On {{PendingAntiCompaction::abort()}} , we try to release refs to sstables
> and a lifecycle transaction that can be null if there are no sstables in the
> table.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13585) NPE in IR cleanup when columnfamily has no sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-13585:
---
Status: Patch Available (was: Open)
> NPE in IR cleanup when columnfamily has no sstables
> ---
>
> Key: CASSANDRA-13585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13585
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Labels: incremental_repair
> Fix For: 4.x
>
> Attachments:
> 0001-NPE-in-IR-cleanup-when-columnfamily-has-no-sstables.patch
>
>
> On {{PendingAntiCompaction::abort()}} , we try to release refs to sstables
> and a lifecycle transaction that can be null if there are no sstables in the
> table.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13585) NPE in IR cleanup when columnfamily has no sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-13585:
---
Labels: incremental_repair (was: )
> NPE in IR cleanup when columnfamily has no sstables
> ---
>
> Key: CASSANDRA-13585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13585
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Labels: incremental_repair
> Fix For: 4.x
>
>
> On {{PendingAntiCompaction::abort()}} , we try to release refs to sstables
> and a lifecycle transaction that can be null if there are no sstables in the
> table.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13585) NPE in IR cleanup when columnfamily has no sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-13585:
---
Fix Version/s: (was: 3.11.x)
> NPE in IR cleanup when columnfamily has no sstables
> ---
>
> Key: CASSANDRA-13585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13585
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Fix For: 4.x
>
>
> On {{PendingAntiCompaction::abort()}} , we try to release refs to sstables
> and a lifecycle transaction that can be null if there are no sstables in the
> table.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-13585) NPE in IR cleanup when columnfamily has no sstables
Jeff Jirsa created CASSANDRA-13585:
--
Summary: NPE in IR cleanup when columnfamily has no sstables
Key: CASSANDRA-13585
URL: https://issues.apache.org/jira/browse/CASSANDRA-13585
Project: Cassandra
Issue Type: Bug
Reporter: Jeff Jirsa
Assignee: Jeff Jirsa
Fix For: 3.11.x, 4.x
On {{PendingAntiCompaction::abort()}} , we try to release refs to sstables and
a lifecycle transaction that can be null if there are no sstables in the table.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12148) Improve determinism of CDC data availability
[
https://issues.apache.org/jira/browse/CASSANDRA-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043611#comment-16043611
]
Jay Zhuang commented on CASSANDRA-12148:
Hi [~JoshuaMcKenzie], any plan to commit this change to trunk?
> Improve determinism of CDC data availability
>
>
> Key: CASSANDRA-12148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12148
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>
> The latency with which CDC data becomes available has a known limitation due
> to our reliance on CommitLogSegments being discarded to have the data
> available in cdc_raw: if a slowly written table co-habitates a
> CommitLogSegment with CDC data, the CommitLogSegment won't be flushed until
> we hit either memory pressure on memtables or CommitLog limit pressure.
> Ultimately, this leaves a non-deterministic element to when data becomes
> available for CDC consumption unless a consumer parses live CommitLogSegments.
> To work around this limitation and make semi-realtime CDC consumption more
> friendly to end-users, I propose we extend CDC as follows:
> h6. High level:
> * Consumers parse hard links of active CommitLogSegments in cdc_raw instead
> of waiting for flush/discard and file move
> * C* stores an offset of the highest seen CDC mutation in a separate idx file
> per commit log segment in cdc_raw. Clients tail this index file, delta their
> local last parsed offset on change, and parse the corresponding commit log
> segment using their last parsed offset as min
> * C* flags that index file with an offset and DONE when the file is flushed
> so clients know when they can clean up
> h6. Details:
> * On creation of a CommitLogSegment, also hard-link the file in cdc_raw
> * On first write of a CDC-enabled mutation to a segment, we:
> ** Flag it as {{CDCState.CONTAINS}}
> ** Set a long tracking the {{CommitLogPosition}} of the 1st CDC-enabled
> mutation in the log
> ** Set a long in the CommitLogSegment tracking the offset of the end of the
> last written CDC mutation in the segment if higher than the previously known
> highest CDC offset
> * On subsequent writes to the segment, we update the offset of the highest
> known CDC data
> * On CommitLogSegment fsync, we write a file in cdc_raw as
> _cdc.idx containing the min offset and end offset fsynced to
> disk per file
> * On segment discard, if CDCState == {{CDCState.PERMITTED}}, delete both the
> segment in commitlog and in cdc_raw
> * On segment discard, if CDCState == {{CDCState.CONTAINS}}, delete the
> segment in commitlog and update the _cdc.idx file w/end offset
> and a DONE marker
> * On segment replay, store the highest end offset of seen CDC-enabled
> mutations from a segment and write that to _cdc.idx on
> completion of segment replay. This should bridge the potential correctness
> gap of a node writing to a segment and then dying before it can write the
> _cdc.idx file.
> This should allow clients to skip the beginning of a file to the 1st CDC
> mutation, track an offset of how far they've parsed, delta against the
> _cdc.idx file end offset, and use that as a determinant on when to parse new
> CDC data. Any existing clients written to the initial implementation of CDC
> need only add the _cdc.idx logic and checking for DONE marker
> to their code, so the burden on users to update to support this should be
> quite small for the benefit of having data available as soon as it's fsynced
> instead of at a non-deterministic time when potentially unrelated tables are
> flushed.
> Finally, we should look into extending the interface on CommitLogReader to be
> more friendly for realtime parsing, perhaps supporting taking a
> CommitLogDescriptor and RandomAccessReader and resuming readSection calls,
> assuming the reader is at the start of a SyncSegment. Would probably also
> need to rewind to the start of the segment before returning so subsequent
> calls would respect this contract. This would skip needing to deserialize the
> descriptor and all completed SyncSegments to get to the root of the desired
> segment for parsing.
> One alternative we discussed offline - instead of just storing the highest
> seen CDC offset, we could instead store an offset per CDC mutation
> (potentially delta encoded) in the idx file to allow clients to seek and only
> parse the mutations with CDC enabled. My hunch is that the performance delta
> from doing so wouldn't justify the complexity given the SyncSegment
> deserialization and seeking restrictions in the compressed and encrypted
> cases as mentioned above.
> The only complication I can think of with the above design is uncompressed
> mmapped CommitLogSegments on Windows being undeletable, but it'd be pretty
[jira] [Comment Edited] (CASSANDRA-13578) mx4j configuration minor improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043563#comment-16043563
]
Jay Zhuang edited comment on CASSANDRA-13578 at 6/8/17 10:47 PM:
-
The problem is if anyone set {{MX4J_PORT=8081}} in the env instead of
{{MX4J_PORT=-Dmx4jport=8081}}, Cassandra will fail to start:
{noformat}
$ ./bin/cassandra
...
CompilerOracle: dontinline
org/apache/cassandra/io/util/RebufferingInputStream.readPrimitiveSlowly (I)J
CompilerOracle: inline
org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody
(Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds (JJ)V
CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds (JJ)V
CompilerOracle: inline
org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary
(Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I
CompilerOracle: inline
org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan
(Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes
(Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes
(JJIJ[J)V
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare
(Ljava/nio/ByteBuffer;[B)I
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare
([BLjava/nio/ByteBuffer;)I
CompilerOracle: inline
org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned
(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
(Ljava/lang/Object;JILjava/lang/Object;JI)I
CompilerOracle: inline
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
(Ljava/lang/Object;JILjava/nio/ByteBuffer;)I
CompilerOracle: inline
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt
(JI)[B
Error: Could not find or load main class 8081
{noformat}
So I think it worth to have this change in 4.0.
was (Author: jay.zhuang):
The problem is if anyone set {{MX4J_PORT=8081}} in the env instead of
{{MX4J_PORT=-Dmx4jport=8081}}, Cassandra will fail to start:
{{noformat}}
$ ./bin/cassandra
...
CompilerOracle: dontinline
org/apache/cassandra/io/util/RebufferingInputStream.readPrimitiveSlowly (I)J
CompilerOracle: inline
org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody
(Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds (JJ)V
CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds (JJ)V
CompilerOracle: inline
org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary
(Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I
CompilerOracle: inline
org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan
(Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes
(Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes
(JJIJ[J)V
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare
(Ljava/nio/ByteBuffer;[B)I
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare
([BLjava/nio/ByteBuffer;)I
CompilerOracle: inline
org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned
(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
(Ljava/lang/Object;JILjava/lang/Object;JI)I
CompilerOracle: inline
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
(Ljava/lang/Object;JILjava/nio/ByteBuffer;)I
CompilerOracle: inline
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt
(JI)[B
Error: Could not find or load main class 8081
{{noformat}}
So I think it worth to have this change in 4.0.
> mx4j configuration minor improvement
>
>
> Key: CASSANDRA-13578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13578
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.x
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13578) mx4j configuration minor improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043563#comment-16043563
]
Jay Zhuang commented on CASSANDRA-13578:
The problem is if anyone set {{MX4J_PORT=8081}} in the env instead of
{{MX4J_PORT=-Dmx4jport=8081}}, Cassandra will fail to start:
{{noformat}}
$ ./bin/cassandra
...
CompilerOracle: dontinline
org/apache/cassandra/io/util/RebufferingInputStream.readPrimitiveSlowly (I)J
CompilerOracle: inline
org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody
(Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds (JJ)V
CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds (JJ)V
CompilerOracle: inline
org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary
(Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I
CompilerOracle: inline
org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan
(Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes
(Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes
(JJIJ[J)V
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare
(Ljava/nio/ByteBuffer;[B)I
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare
([BLjava/nio/ByteBuffer;)I
CompilerOracle: inline
org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned
(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
(Ljava/lang/Object;JILjava/lang/Object;JI)I
CompilerOracle: inline
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
(Ljava/lang/Object;JILjava/nio/ByteBuffer;)I
CompilerOracle: inline
org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo
(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt
(JI)[B
Error: Could not find or load main class 8081
{{noformat}}
So I think it worth to have this change in 4.0.
> mx4j configuration minor improvement
>
>
> Key: CASSANDRA-13578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13578
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.x
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-8272) 2ndary indexes can return stale data
[ https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042891#comment-16042891 ] Andrés de la Peña edited comment on CASSANDRA-8272 at 6/8/17 3:47 PM: -- It seems that there were some thrift-related dtests failing in the patch for 3.11. While fixing them I have realized that Thrift commands send only the fetched columns, so they may not send the queried columns that are required to apply the index filter in the coordinator side. The indexed column value are actually fetched but filtered [here|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/index/internal/keys/KeysSearcher.java#L182-L193] since [CASSANDRA-11523|https://issues.apache.org/jira/browse/CASSANDRA-11523]. So we could just move this filter to the coordinator-side, probably to [ReadCommand#postReconciliationProcessing|https://github.com/adelapena/cassandra/blob/8272-3.11/src/java/org/apache/cassandra/db/ReadCommand.java#L447-L464]. However, I think that if we do such replica-side change we would end having the same problem with upgrades that prevents us to apply the full solution to 3.x. That is, not-upgraded replicas could send rows without the indexed-but-not-fetched columns to an upgraded coordinator that would reject them. Complementary, upgraded replicas could send rows including the indexed-but-not-fetched columns to not-upgraded coordinators that would return them without applying the row filter. Probably we could also have problematic scenarios during reconciliation. Here is a fixed patch that just skips coordinator-side filtering of index results for Thrift commands: ||[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...adelapena:82b122b1ce5b172e11b4be7f02fdb7581bd28291]|[utests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-3.11-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-3.11-dtest/]| ||[trunk|https://github.com/apache/cassandra/compare/trunk...adelapena:1416d9b082d7f93b187cbf67abd9a917735c4804]|[utests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-trunk-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-trunk-dtest/]| This means that not included index implementations couldn't benefit from our consistency fix when using Thrift. Included index implementations weren't going to do so anyway because we are not going to apply the replica side of the fix in 3.x. No news for trunk. What do you think? Is it acceptable to don't apply the changes to Thrift commands? was (Author: adelapena): It seems that there were some thrift-related dtests failing in the path for 3.11. While fixing them I have realized that Thrift commands send only the fetched columns, so they may not send the queried columns that are required to apply the index filter in the coordinator side. The indexed column value are actually fetched but filtered [here|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/index/internal/keys/KeysSearcher.java#L182-L193] since [CASSANDRA-11523|https://issues.apache.org/jira/browse/CASSANDRA-11523]. So we could just move this filter to the coordinator-side, probably to [ReadCommand#postReconciliationProcessing|https://github.com/adelapena/cassandra/blob/8272-3.11/src/java/org/apache/cassandra/db/ReadCommand.java#L447-L464]. However, I think that if we do such replica-side change we would end having the same problem with upgrades that prevents us to apply the full solution to 3.x. That is, not-upgraded replicas could send rows without the indexed-but-not-fetched columns to an upgraded coordinator that would reject them. Complementary, upgraded replicas could send rows including the indexed-but-not-fetched columns to not-upgraded coordinators that would return them without applying the row filter. Probably we could also have problematic scenarios during reconciliation. Here is a fixed patch that just skips coordinator-side filtering of index results for Thrift commands: ||[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...adelapena:82b122b1ce5b172e11b4be7f02fdb7581bd28291]|[utests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-3.11-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-3.11-dtest/]| ||[trunk|https://github.com/apache/cassandra/compare/trunk...adelapena:1416d9b082d7f93b187cbf67abd9a917735c4804]|[utests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-trunk-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-trunk-dtest/]| This means that not included index implementations couldn't benefit from our consistency fix when using Thrift. Included index implementations weren't going to do so
[jira] [Commented] (CASSANDRA-8272) 2ndary indexes can return stale data
[
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042891#comment-16042891
]
Andrés de la Peña commented on CASSANDRA-8272:
--
It seems that there were some thrift-related dtests failing in the path for
3.11. While fixing them I have realized that Thrift commands send only the
fetched columns, so they may not send the queried columns that are required to
apply the index filter in the coordinator side. The indexed column value are
actually fetched but filtered
[here|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/index/internal/keys/KeysSearcher.java#L182-L193]
since [CASSANDRA-11523|https://issues.apache.org/jira/browse/CASSANDRA-11523].
So we could just move this filter to the coordinator-side, probably to
[ReadCommand#postReconciliationProcessing|https://github.com/adelapena/cassandra/blob/8272-3.11/src/java/org/apache/cassandra/db/ReadCommand.java#L447-L464].
However, I think that if we do such replica-side change we would end having the
same problem with upgrades that prevents us to apply the full solution to 3.x.
That is, not-upgraded replicas could send rows without the
indexed-but-not-fetched columns to an upgraded coordinator that would reject
them. Complementary, upgraded replicas could send rows including the
indexed-but-not-fetched columns to not-upgraded coordinators that would return
them without applying the row filter. Probably we could also have problematic
scenarios during reconciliation.
Here is a fixed patch that just skips coordinator-side filtering of index
results for Thrift commands:
||[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...adelapena:82b122b1ce5b172e11b4be7f02fdb7581bd28291]|[utests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-3.11-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-3.11-dtest/]|
||[trunk|https://github.com/apache/cassandra/compare/trunk...adelapena:1416d9b082d7f93b187cbf67abd9a917735c4804]|[utests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-trunk-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-trunk-dtest/]|
This means that not included index implementations couldn't benefit from our
consistency fix when using Thrift. Included index implementations weren't going
to do so anyway because we are not going to apply the replica side of the fix
in 3.x.
No news for trunk.
What do you think? Is it acceptable to don't apply the changes to Thrift
commands?
> 2ndary indexes can return stale data
>
>
> Key: CASSANDRA-8272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Andrés de la Peña
> Fix For: 3.0.x
>
>
> When replica return 2ndary index results, it's possible for a single replica
> to return a stale result and that result will be sent back to the user,
> potentially failing the CL contract.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v text);
> CREATE INDEX ON test(v);
> INSERT INTO test(k, v) VALUES (0, 'foo');
> {noformat}
> with every replica up to date. Now, suppose that the following queries are
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v = 'bar' WHERE k = 0;
> SELECT * FROM test WHERE v = 'foo';
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before
> having applied the insert, then the now stale result will be returned (since
> C will return it and A or B will return nothing).
> A potential solution would be that when we read a tombstone in the index (and
> provided we make the index inherit the gcGrace of it's parent CF), instead of
> skipping that tombstone, we'd insert in the result a corresponding range
> tombstone.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id
[
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042862#comment-16042862
]
Alex Petrov commented on CASSANDRA-10786:
-
Just to clarify, this is in fact a bugfix. It was somewhat blocked by
introduction of the beta flag and v5 protocol, but it might be considered a
bugfix. Doesn't mean it automatically has to go into 3.11 because of the
protocol change requirement, but in the end that's what the beta flag was for.
> Include hash of result set metadata in prepared statement id
>
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
> Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 4.x
>
>
> *_Initial description:_*
> This is a follow-up to CASSANDRA-7910, which was about invalidating a
> prepared statement when the table is altered, to force clients to update
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host.
> The first client to execute the query after the cache was invalidated will
> receive an UNPREPARED response, re-prepare, and update its local metadata.
> But other clients might miss it entirely (the MD5 hasn't changed), and they
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123,
> clientA and clientB both have a cache of the metadata (columns b and c)
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response,
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been
> populated again, the query succeeds. But clientB still has not updated its
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set
> metadata in the md5. This way the md5 would change at step 3, and any client
> using the old md5 would get an UNPREPARED, regardless of whether another
> client already reprepared.
> -
> *_Resolution (2017/02/13):_*
> The following changes were made to native protocol v5:
> - the PREPARED response includes {{result_metadata_id}}, a hash of the result
> set metadata.
> - every EXECUTE message must provide {{result_metadata_id}} in addition to
> the prepared statement id. If it doesn't match the current one on the server,
> it means the client is operating on a stale schema.
> - to notify the client, the server returns a ROWS response with a new
> {{Metadata_changed}} flag, the new {{result_metadata_id}} and the updated
> result metadata (this overrides the {{No_metadata}} flag, even if the client
> had requested it)
> - the client updates its copy of the result metadata before it decodes the
> results.
> So the scenario above would now look like:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, and
> result set (b, c) that hashes to cde456
> # column a gets added to the table, C* does not invalidate its cache entry,
> but only updates the result set to (a, b, c) which hashes to fff789
> # client sends an EXECUTE request for (statementId=abc123, resultId=cde456)
> and skip_metadata flag
> # cde456!=fff789, so C* responds with ROWS(..., no_metadata=false,
> metadata_changed=true, new_metadata_id=fff789,col specs for (a,b,c))
> # client updates its column specifications, and will send the next execute
> queries with (statementId=abc123, resultId=fff789)
> This works the same with multiple clients.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10765) add RangeIterator interface and QueryPlan for SI
[ https://issues.apache.org/jira/browse/CASSANDRA-10765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042791#comment-16042791 ] Corentin Chary commented on CASSANDRA-10765: Note: https://github.com/iksaif/cassandra/commit/edbc0a0572b47ef5d5f25d56bd43587eb136170a was an attempt at improving that, which work very well in the cases where multiple indexes are queried and some of them intersect but all of them do not. > add RangeIterator interface and QueryPlan for SI > > > Key: CASSANDRA-10765 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10765 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich > Fix For: 4.x > > > Currently built-in indexes have only one way of handling > intersections/unions: pick the highest selectivity predicate and filter on > other index expressions. This is not always the most efficient approach. > Dynamic query planning based on the different index characteristics would be > more optimal. Query Plan should be able to choose how to do intersections, > unions based on the metadata provided by indexes (returned by RangeIterator) > and RangeIterator would became a base for cross index interactions and should > have information such as min/max token, estimate number of wrapped tokens etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13580) Readonly datacenter support
[ https://issues.apache.org/jira/browse/CASSANDRA-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042773#comment-16042773 ] Marcus Eriksson commented on CASSANDRA-13580: - bq. Nodes in a read only DC will still receive and persist data from the rest of the clusters and get counted against the CL, right? good point, this wont work the way it is implemented now, and I doubt the use case is common enough to warrant a more complicated solution. > Readonly datacenter support > --- > > Key: CASSANDRA-13580 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13580 > Project: Cassandra > Issue Type: New Feature >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 4.x > > > Some setups include datacenters where only reads are performed (example could > be datacenter dedicated for taking backups). > We could use this information during repair to make sure that we never stream > out of a read only dc. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13580) Readonly datacenter support
[ https://issues.apache.org/jira/browse/CASSANDRA-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-13580: Status: Open (was: Patch Available) > Readonly datacenter support > --- > > Key: CASSANDRA-13580 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13580 > Project: Cassandra > Issue Type: New Feature >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 4.x > > > Some setups include datacenters where only reads are performed (example could > be datacenter dedicated for taking backups). > We could use this information during repair to make sure that we never stream > out of a read only dc. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13580) Readonly datacenter support
[ https://issues.apache.org/jira/browse/CASSANDRA-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Podkowinski updated CASSANDRA-13580: --- Issue Type: New Feature (was: Bug) > Readonly datacenter support > --- > > Key: CASSANDRA-13580 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13580 > Project: Cassandra > Issue Type: New Feature >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 4.x > > > Some setups include datacenters where only reads are performed (example could > be datacenter dedicated for taking backups). > We could use this information during repair to make sure that we never stream > out of a read only dc. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13580) Readonly datacenter support
[
https://issues.apache.org/jira/browse/CASSANDRA-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042756#comment-16042756
]
Stefan Podkowinski commented on CASSANDRA-13580:
If the scope was to simply exclude DCs from repairs, wouldn't {{repair -dc}}
have the same effect?
I'm also still trying to wrap my head around the help text of the new repair
option:
"Use --ignore_readonly_dcs to do a full sync on repair - even stream out data
from read only dcs."
What's being ignored here? The read only nodes from the repair or the read only
semantics that would usually be in place ignoring those nodes by default during
repairs? Does this option imply -full?
I'd personally avoid naming this "readonly datacenter", as people likely have
different expectations about the actual meaning. E.g. nodes in such a DC might
still be able to effect the cluster through actions like gossip, schema, or
read repairs. Nodes in a read only DC will still receive and persist data from
the rest of the clusters and get counted against the CL, right?
> Readonly datacenter support
> ---
>
> Key: CASSANDRA-13580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13580
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> Some setups include datacenters where only reads are performed (example could
> be datacenter dedicated for taking backups).
> We could use this information during repair to make sure that we never stream
> out of a read only dc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13403) nodetool repair breaks SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042602#comment-16042602
]
Alex Petrov commented on CASSANDRA-13403:
-
Sorry until now I haven't gotten to do it. If anyone has time meanwhile -
please feel free to reassign the issue.
> nodetool repair breaks SASI index
> -
>
> Key: CASSANDRA-13403
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13403
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: 3.10
>Reporter: Igor Novgorodov
>Assignee: Alex Petrov
>
> I've got table:
> {code}
> CREATE TABLE cservice.bulks_recipients (
> recipient text,
> bulk_id uuid,
> datetime_final timestamp,
> datetime_sent timestamp,
> request_id uuid,
> status int,
> PRIMARY KEY (recipient, bulk_id)
> ) WITH CLUSTERING ORDER BY (bulk_id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX bulk_recipients_bulk_id ON cservice.bulks_recipients
> (bulk_id) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> There are 11 rows in it:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> Let's query by index (all rows have the same *bulk_id*):
> {code}
> > select * from bulks_recipients where bulk_id =
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;
> >
> ...
> (11 rows)
> {code}
> Ok, everything is fine.
> Now i'm doing *nodetool repair --partitioner-range --job-threads 4 --full* on
> each node in cluster sequentially.
> After it finished:
> {code}
> > select * from bulks_recipients where bulk_id =
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;
> ...
> (2 rows)
> {code}
> Only two rows.
> While the rows are actually there:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> If i issue an incremental repair on a random node, i can get like 7 rows
> after index query.
> Dropping index and recreating it fixes the issue. Is it a bug or am i doing
> the repair the wrong way?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13501) Upgrade some dependencies.
[ https://issues.apache.org/jira/browse/CASSANDRA-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042549#comment-16042549 ] vincent royer commented on CASSANDRA-13501: --- Assuming there is no regression, we can switch to the latest version that you mention. For snowball-stemmer ( https://github.com/rholder/snowball-stemmer/tree/master/src/main/java/org/tartarus/snowball ), it's a bit messy, because same classes in a different version have been included in lucene-analyzers-common-5.5.4.jar (see https://github.com/apache/lucene-solr/tree/branch_5_4/lucene/analysis/common/src/java/org/tartarus/snowball ). So, depending on class loading order, you could have various behavior. I guess that at least on project (rholder/snowball-stemmer or lucene) should rename this package to avoid such a version conflict ? (The stratio lucene plugin has the same issue in version 3.10, because it also include lucene-analyzers-common-5.5.4.jar). > Upgrade some dependencies. > -- > > Key: CASSANDRA-13501 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13501 > Project: Cassandra > Issue Type: Improvement > Components: Libraries >Reporter: vincent royer >Priority: Minor > Fix For: 3.0.x, 3.11.x, 4.x > > > Upgrade some java libraries to be able to run elasticsearch as a cassandra > plugin (an elasticsearch jar dropped in lib), see CASSANDRA-13270. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13579) Request to change the accessibility of `RowCacheSerializer` needed for CAPi-Flash Enablement.
[ https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042519#comment-16042519 ] Amitkumar Ghatwal commented on CASSANDRA-13579: --- PR raised : https://github.com/apache/cassandra/pull/117/commits/148a0bc5de03b4c47c620a4199c2ae56071742dc with the correct comments as suggested - [~jjirsa]. > Request to change the accessibility of `RowCacheSerializer` needed for > CAPi-Flash Enablement. > - > > Key: CASSANDRA-13579 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13579 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Amitkumar Ghatwal >Assignee: Amitkumar Ghatwal > Labels: patch > Fix For: 4.x > > Attachments: cassandra_change_RowCacheSerializer_accessibility.patch > > > Hi All, > [~ReiOdaira] , and I are working on creating a plugin for vendor specific > (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are > now working on the plugin approach .And for creating the same we would need a > change in the accessibility of `RowCacheSerializer` . > Please refer to jira ticket/dev mails for reference : > 1) https://issues.apache.org/jira/browse/CASSANDRA-13486 > 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html > [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the > accessibility. Let me know once the change will be merged in the mainline. > PR raised : https://github.com/apache/cassandra/pull/117 > Thanks !!! -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13581) Adding plugins support to Cassandra's webpage
[ https://issues.apache.org/jira/browse/CASSANDRA-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042400#comment-16042400 ] Amitkumar Ghatwal commented on CASSANDRA-13581: --- [~jjirsa] - Any comments on the above ? > Adding plugins support to Cassandra's webpage > - > > Key: CASSANDRA-13581 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13581 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Amitkumar Ghatwal > Labels: documentation > Fix For: 4.x > > > Hi [~spo...@gmail.com], > As was suggested here : > http://www.mail-archive.com/dev@cassandra.apache.org/msg11183.html . Have > created the necessary *.rst file to create "plugins" link here : > https://cassandra.apache.org/doc/latest/. > Have followed the steps here : > https://cassandra.apache.org/doc/latest/development/documentation.html and > raised a PR : https://github.com/apache/cassandra/pull/118 for introducing > plugins support on Cassandra's Webpage. > Let me know your review comments and if i have not done things correctly to > make changes to cassandra's website i can rectify the same. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-12201) Burst Hour Compaction Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-12201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pedro Gordo updated CASSANDRA-12201: Comment: was deleted (was: Apologies for the lack of progress on this issue but to a bigger workload from my job and other factors, I had to halt work on this ticket for six months. I'm now resuming this piece of work. I studied on the data structure for Cassandra 2.0 but from what I know, there were significant changes to 3.0, so I'll need to get up to speed on the new data storage on 3.0. I was considering to implement this just on 2.2, but from what I see here: http://cassandra.apache.org/doc/latest/development/patches.html no contributions for previous versions are being accepted, so I'll need to go straight to 3.x.) > Burst Hour Compaction Strategy > -- > > Key: CASSANDRA-12201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12201 > Project: Cassandra > Issue Type: New Feature > Components: Compaction >Reporter: Pedro Gordo > Original Estimate: 1,008h > Remaining Estimate: 1,008h > > This strategy motivation revolves around taking advantage of periods of the > day where there's less I/O on the cluster. This time of the day will be > called “Burst Hour” (BH), and hence the strategy will be named “Burst Hour > Compaction Strategy” (BHCS). > The following process would be fired during BH: > 1. Read all the SSTables and detect which partition keys are present in more > than the compaction minimum threshold value. > 2. Gather all the tables that have keys present in other tables, with a > minimum of replicas equal to the minimum compaction threshold. > 3. Repeat step 2 until the bucket for gathered SSTables reaches the maximum > compaction threshold (32 by default), or until we've searched all the keys. > 4. The compaction per se will be done through by MaxSSTableSizeWriter. The > compacted tables will have a maximum size equal to the configurable value of > max_sstable_size (100MB by default). > The maximum compaction task (nodetool compact command), does exactly the same > operation as the background compaction task, but differing in that it can be > triggered outside of the Burst Hour. > This strategy tries to address three issues of the existing compaction > strategies: > - Due to max_sstable_size_limit, there's no need to reserve disc space for a > huge compaction. > - The number of SSTables that we need to read from to reply to a read query > will be consistently maintained at a low level and controllable through the > referenced_sstable_limit property. > - It removes the dependency of a continuous high I/O. > Possible future improvements: > - Continuously evaluate how many pending compactions we have and I/O status, > and then based on that, we start (or not) the compaction. > - If during the day, the size for all the SSTables in a family set reaches a > certain maximum, then background compaction can occur anyway. This maximum > should be elevated due to the high CPU usage of BHCS. > - Make it possible to set several compaction times intervals, instead of just > one. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13579) Request to change the accessibility of `RowCacheSerializer` needed for CAPi-Flash Enablement.
[ https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042368#comment-16042368 ] Amitkumar Ghatwal commented on CASSANDRA-13579: --- Agreed : [~jjirsa] . let me know when will it be merged ? > Request to change the accessibility of `RowCacheSerializer` needed for > CAPi-Flash Enablement. > - > > Key: CASSANDRA-13579 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13579 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Amitkumar Ghatwal >Assignee: Amitkumar Ghatwal > Labels: patch > Fix For: 4.x > > Attachments: cassandra_change_RowCacheSerializer_accessibility.patch > > > Hi All, > [~ReiOdaira] , and I are working on creating a plugin for vendor specific > (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are > now working on the plugin approach .And for creating the same we would need a > change in the accessibility of `RowCacheSerializer` . > Please refer to jira ticket/dev mails for reference : > 1) https://issues.apache.org/jira/browse/CASSANDRA-13486 > 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html > [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the > accessibility. Let me know once the change will be merged in the mainline. > PR raised : https://github.com/apache/cassandra/pull/117 > Thanks !!! -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-12201) Burst Hour Compaction Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-12201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pedro Gordo updated CASSANDRA-12201: Description: This strategy motivation revolves around taking advantage of periods of the day where there's less I/O on the cluster. This time of the day will be called “Burst Hour” (BH), and hence the strategy will be named “Burst Hour Compaction Strategy” (BHCS). The following process would be fired during BH: 1. Read all the SSTables and detect which partition keys are present in more than the compaction minimum threshold value. 2. Gather all the tables that have keys present in other tables, with a minimum of replicas equal to the minimum compaction threshold. 3. Repeat step 2 until the bucket for gathered SSTables reaches the maximum compaction threshold (32 by default), or until we've searched all the keys. 4. The compaction per se will be done through by MaxSSTableSizeWriter. The compacted tables will have a maximum size equal to the configurable value of max_sstable_size (100MB by default). The maximum compaction task (nodetool compact command), does exactly the same operation as the background compaction task, but differing in that it can be triggered outside of the Burst Hour. This strategy tries to address three issues of the existing compaction strategies: - Due to max_sstable_size_limit, there's no need to reserve disc space for a huge compaction. - The number of SSTables that we need to read from to reply to a read query will be consistently maintained at a low level and controllable through the referenced_sstable_limit property. - It removes the dependency of a continuous high I/O. Possible future improvements: - Continuously evaluate how many pending compactions we have and I/O status, and then based on that, we start (or not) the compaction. - If during the day, the size for all the SSTables in a family set reaches a certain maximum, then background compaction can occur anyway. This maximum should be elevated due to the high CPU usage of BHCS. - Make it possible to set several compaction times intervals, instead of just one. was: This strategy motivation revolves around taking advantage of periods of the day where there's less I/O on the cluster. This time of the day will be called “Burst Hour” (BH), and hence the strategy will be named “Burst Hour Compaction Strategy” (BHCS). The following process would be fired during BH: 1. Read all the SSTables and detect which partition keys are present in more than the compaction minimum threshold value. 2. Gather all the tables that have keys present in other tables, with a minimum of replicas equal to the minimum compaction threshold. 3. Repeat step 2 until the bucket for gathered SSTables reaches the maximum compaction threshold (32 by default), or until we've searched all the keys. 4. The compaction per se will be done through by MaxSSTableSizeWriter. The compacted tables will have a maximum size equal to the configurable value of max_sstable_size. The maximum compaction task (nodetool compact command), does exactly the same operation as the background compaction task, but differing in that it can be triggered outside of the Burst Hour. This strategy tries to address three issues of the existing compaction strategies: - Due to max_sstable_size_limit, there's no need to reserve disc space for a huge compaction. - The number of SSTables that we need to read from to reply to a read query will be consistently maintained at a low level and controllable through the referenced_sstable_limit property. - It removes the dependency of a continuous high I/O. Possible future improvements: - Continuously evaluate how many pending compactions we have and I/O status, and then based on that, we start (or not) the compaction. - If during the day, the size for all the SSTables in a family set reaches a certain maximum, then background compaction can occur anyway. This maximum should be elevated due to the high CPU usage of BHCS. - Make it possible to set several compaction times intervals, instead of just one. > Burst Hour Compaction Strategy > -- > > Key: CASSANDRA-12201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12201 > Project: Cassandra > Issue Type: New Feature > Components: Compaction >Reporter: Pedro Gordo > Original Estimate: 1,008h > Remaining Estimate: 1,008h > > This strategy motivation revolves around taking advantage of periods of the > day where there's less I/O on the cluster. This time of the day will be > called “Burst Hour” (BH), and hence the strategy will be named “Burst Hour > Compaction Strategy” (BHCS). > The following process would be fired during BH: > 1. Read all the SSTables and detect which partition keys are present in more > than the compaction
[jira] [Updated] (CASSANDRA-12201) Burst Hour Compaction Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-12201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pedro Gordo updated CASSANDRA-12201: Description: This strategy motivation revolves around taking advantage of periods of the day where there's less I/O on the cluster. This time of the day will be called “Burst Hour” (BH), and hence the strategy will be named “Burst Hour Compaction Strategy” (BHCS). The following process would be fired during BH: 1. Read all the SSTables and detect which partition keys are present in more than the compaction minimum threshold value. 2. Gather all the tables that have keys present in other tables, with a minimum of replicas equal to the minimum compaction threshold. 3. Repeat step 2 until the bucket for gathered SSTables reaches the maximum compaction threshold (32 by default), or until we've searched all the keys. 4. The compaction per se will be done through by MaxSSTableSizeWriter. The compacted tables will have a maximum size equal to the configurable value of max_sstable_size. The maximum compaction task (nodetool compact command), does exactly the same operation as the background compaction task, but differing in that it can be triggered outside of the Burst Hour. This strategy tries to address three issues of the existing compaction strategies: - Due to max_sstable_size_limit, there's no need to reserve disc space for a huge compaction. - The number of SSTables that we need to read from to reply to a read query will be consistently maintained at a low level and controllable through the referenced_sstable_limit property. - It removes the dependency of a continuous high I/O. Possible future improvements: - Continuously evaluate how many pending compactions we have and I/O status, and then based on that, we start (or not) the compaction. - If during the day, the size for all the SSTables in a family set reaches a certain maximum, then background compaction can occur anyway. This maximum should be elevated due to the high CPU usage of BHCS. - Make it possible to set several compaction times intervals, instead of just one. was: This strategy motivation revolves around taking advantage of periods of the day where there's less I/O on the cluster. This time of the day will be called “Burst Hour” (BH), and hence the strategy will be named “Burst Hour Compaction Strategy” (BHCS). The following process would be fired during BH: 1. Read all the SSTables and detect which partition keys are present in more than the compaction minimum threshold value. 2. Gather all the tables that have keys present in other tables, with a minimum of replicas equal to the minimum compaction threshold. 3. Repeat step 2 until the bucket for gathered SSTables reaches the maximum compaction threshold (32 by default), or until we've searched all the keys. 4. The compaction per se will be done through by MaxSSTableSizeWriter. The compacted tables will have a maximum size equal to the configurable value of max_sstable_size. The maximum compaction task (nodetool compact command), does exactly the same operation as the background compaction task, but differing in that it can be triggered outside of the Burst Hour. This strategy tries to address three issues of the existing compaction strategies: - Due to max_sstable_size_limit, there's no need to reserve disc space for a huge compaction. - The number of SSTables that we need to read from to reply to a read query will be consistently maintained at a low level and controllable through the referenced_sstable_limit property. - It removes the dependency of a continuous high I/O. Possible future improvements: - Continuously evaluate how many pending compactions we have and I/O status, and then based on that, we start (or not) the compaction. - If during the day, the size for all the SSTables in a family set reaches a certain maximum, then background compaction can occur anyway. This maximum should be elevated due to the high CPU usage of BHCS. > Burst Hour Compaction Strategy > -- > > Key: CASSANDRA-12201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12201 > Project: Cassandra > Issue Type: New Feature > Components: Compaction >Reporter: Pedro Gordo > Original Estimate: 1,008h > Remaining Estimate: 1,008h > > This strategy motivation revolves around taking advantage of periods of the > day where there's less I/O on the cluster. This time of the day will be > called “Burst Hour” (BH), and hence the strategy will be named “Burst Hour > Compaction Strategy” (BHCS). > The following process would be fired during BH: > 1. Read all the SSTables and detect which partition keys are present in more > than the compaction minimum threshold value. > 2. Gather all the tables that have keys present in other tables, with a >
[jira] [Updated] (CASSANDRA-12201) Burst Hour Compaction Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-12201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pedro Gordo updated CASSANDRA-12201: Description: This strategy motivation revolves around taking advantage of periods of the day where there's less I/O on the cluster. This time of the day will be called “Burst Hour” (BH), and hence the strategy will be named “Burst Hour Compaction Strategy” (BHCS). The following process would be fired during BH: 1. Read all the SSTables and detect which partition keys are present in more than the compaction minimum threshold value. 2. Gather all the tables that have keys present in other tables, with a minimum of replicas equal to the minimum compaction threshold. 3. Repeat step 2 until the bucket for gathered SSTables reaches the maximum compaction threshold (32 by default), or until we've searched all the keys. 4. The compaction per se will be done through by MaxSSTableSizeWriter. The compacted tables will have a maximum size equal to the configurable value of max_sstable_size. The maximum compaction task (nodetool compact command), does exactly the same operation as the background compaction task, but differing in that it can be triggered outside of the Burst Hour. This strategy tries to address three issues of the existing compaction strategies: - Due to max_sstable_size_limit, there's no need to reserve disc space for a huge compaction. - The number of SSTables that we need to read from to reply to a read query will be consistently maintained at a low level and controllable through the referenced_sstable_limit property. - It removes the dependency of a continuous high I/O. Possible future improvements: - Continuously evaluate how many pending compactions we have and I/O status, and then based on that, we start (or not) the compaction. - If during the day, the size for all the SSTables in a family set reaches a certain maximum, then background compaction can occur anyway. This maximum should be elevated due to the high CPU usage of BHCS. was: Although it may be subject to changes, for the moment I plan to create a strategy that will revolve around taking advantage of periods of the day where there's less I/O on the cluster. This time of the day will be called “Burst Hour” (BH), and hence the strategy will be named “Burst Hour Compaction Strategy” (BHCS). The following process would be fired during BH: 1. Read all the SSTables and detect which partition keys are present in more than a configurable value which I'll call referenced_sstable_limit. This value will be three by default. 2. Group all the repeated keys with a reference to the SSTables containing them. 3. Calculate the total size of the SSTables which will be merged for the first partition key on the list created in step 2. If the size calculated is bigger than property which I'll call max_sstable_size (also configurable), more than one table will be created in step 4. 4. During the merge, the data will be streamed from SSTables up to a point when we have a size close to max_sstable_size. After we reach this point, the stream is paused, and the new SSTable will be closed, becoming immutable. Repeat the streaming process until we've merged all tables for the partition key that we're iterating. 5. Cycle through the rest of the collection created in step 2 and remove any SSTables which don't exist anymore because they were merged in step 5. An alternative course of action here would be to, instead of removing the SSTable from the collection, to change its reference to the SSTable(s) which was created in step 5. 6. Repeat from step 3 to step 6 until we traversed the entirety of the collection created in step 2. This strategy addresses three issues of the existing compaction strategies: - Due to max_sstable_size_limit, there's no need to reserve disc space for a huge compaction, as it can happen on STCS. - The number of SSTables that we need to read from to reply to a read query will be consistently maintained at a low level and controllable through the referenced_sstable_limit property. This addresses the scenario of STCS when we might have to read from a lot of SSTables. - It removes the dependency of a continuous high I/O of LCS. > Burst Hour Compaction Strategy > -- > > Key: CASSANDRA-12201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12201 > Project: Cassandra > Issue Type: New Feature > Components: Compaction >Reporter: Pedro Gordo > Original Estimate: 1,008h > Remaining Estimate: 1,008h > > This strategy motivation revolves around taking advantage of periods of the > day where there's less I/O on the cluster. This time of the day will be > called “Burst Hour” (BH), and hence the strategy will be named “Burst Hour > Compaction Strategy” (BHCS). > The
