[jira] [Comment Edited] (CASSANDRA-14910) Don't skip entire sstables when reading backwards with mixed clustering column order
[
https://issues.apache.org/jira/browse/CASSANDRA-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697107#comment-16697107
]
Aleksey Yeschenko edited comment on CASSANDRA-14910 at 11/23/18 2:14 PM:
-
Code [here|https://github.com/iamaleksey/cassandra/commits/14910-2.2], CI
[here|https://circleci.com/workflow-run/08fbce09-e965-4909-9fcc-74c0f3893230].
was (Author: iamaleksey):
Code [here|https://github.com/iamaleksey/cassandra/commits/14910-2.2], CI
[here|https://circleci.com/workflow-run/fed811d6-03d4-411d-9091-ff1e2b89d56f].
> Don't skip entire sstables when reading backwards with mixed clustering
> column order
>
>
> Key: CASSANDRA-14910
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14910
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 2.2.x
>
>
> In 2.x, if a table has clustering columns in {{DESC}} order, any SSTable that
> doesn’t have any static rows in it will be skipped altogether when iterated
> in reverse.
> This occurs due to the logic in {{ColumnSlice.compare()}} errorneusly sorting
> any empty {{ByteBuffer}} after non-empty values due to the way
> {{ReversedType}} operates. In case that empty {{ByteBuffer}} is coming from a
> static {{Composite}}, however, the logic breaks down. Static {{Composite}}
> components must *always* sort before any non-empty value, no matter the
> table’s comparator.
> 2.0, 2.1, and 2.2 are all affected. 3.0 isn’t, but only because we skip slice
> intersection logic entirely if static rows are present in the filter.
> Introduced by CASSANDRA-8502.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13917) COMPACT STORAGE queries on dense static tables accept hidden column1 and value columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-13917:
Summary: COMPACT STORAGE queries on dense static tables accept hidden
column1 and value columns (was: COMPACT STORAGE inserts on tables without
clusterings accept hidden column1 and value columns)
> COMPACT STORAGE queries on dense static tables accept hidden column1 and
> value columns
> --
>
> Key: CASSANDRA-13917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13917
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Alex Petrov
>Assignee: Aleksandr Sorokoumov
>Priority: Minor
> Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13917-3.0.png, 13917-3.11.png
>
>
> Test for the issue:
> {code}
> @Test
> public void testCompactStorage() throws Throwable
> {
> createTable("CREATE TABLE %s (a int PRIMARY KEY, b int, c int) WITH
> COMPACT STORAGE");
> assertInvalid("INSERT INTO %s (a, b, c, column1) VALUES (?, ?, ?,
> ?)", 1, 1, 1, ByteBufferUtil.bytes('a'));
> // This one fails with Some clustering keys are missing: column1,
> which is still wrong
> assertInvalid("INSERT INTO %s (a, b, c, value) VALUES (?, ?, ?, ?)",
> 1, 1, 1, ByteBufferUtil.bytes('a'));
> assertInvalid("INSERT INTO %s (a, b, c, column1, value) VALUES (?, ?,
> ?, ?, ?)", 1, 1, 1, ByteBufferUtil.bytes('a'), ByteBufferUtil.bytes('b'));
> assertEmpty(execute("SELECT * FROM %s"));
> }
> {code}
> Gladly, these writes are no-op, even though they succeed.
> {{value}} and {{column1}} should be completely hidden. Fixing this one should
> be as easy as just adding validations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14910) Don't skip entire sstables when reading backwards with mixed clustering column order
[
https://issues.apache.org/jira/browse/CASSANDRA-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14910:
--
Description:
In 2.x, if a table has clustering columns in {{DESC}} order, any SSTable that
doesn’t have any static rows in it will be skipped altogether when iterated in
reverse.
This occurs due to the logic in {{ColumnSlice.compare()}} errorneusly sorting
any empty {{ByteBuffer}} after non-empty values due to the way {{ReversedType}}
operates. In case that empty {{ByteBuffer}} is coming from a static
{{Composite}}, however, the logic breaks down. Static {{Composite}} components
must *always* sort before any non-empty value, no matter the table’s comparator.
2.0, 2.1, and 2.2 are all affected. 3.0 isn’t, but only because we skip slice
intersection logic entirely if static rows are present in the filter.
Introduced by CASSANDRA-8502.
was:
In 2.x, if a table has clustering columns in {{DESC}} order, any SSTable that
doesn’t have any static rows in it will be skipped altogether when iterated in
reverse.
This occurs due to the logic in {{ColumnSlice.compare()}} errorneusly sorting
any empty {{ByteBuffer}} after non-empty values due to the way {{ReversedType}}
operates. In case that empty {{ByteBuffer}} is coming from a static
{{Composite}}, however, the logic breaks down. Static {{Composite}} components
must *always* sort before any non-empty value, no matter the table’s comparator.
2.0, 2.1, and 2.2 are all affected. 3.0 isn’t, but only because we skip slice
intersection logic entirely if static rows are present in the filter.
> Don't skip entire sstables when reading backwards with mixed clustering
> column order
>
>
> Key: CASSANDRA-14910
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14910
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 2.2.x
>
>
> In 2.x, if a table has clustering columns in {{DESC}} order, any SSTable that
> doesn’t have any static rows in it will be skipped altogether when iterated
> in reverse.
> This occurs due to the logic in {{ColumnSlice.compare()}} errorneusly sorting
> any empty {{ByteBuffer}} after non-empty values due to the way
> {{ReversedType}} operates. In case that empty {{ByteBuffer}} is coming from a
> static {{Composite}}, however, the logic breaks down. Static {{Composite}}
> components must *always* sort before any non-empty value, no matter the
> table’s comparator.
> 2.0, 2.1, and 2.2 are all affected. 3.0 isn’t, but only because we skip slice
> intersection logic entirely if static rows are present in the filter.
> Introduced by CASSANDRA-8502.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14910) Don't skip entire sstables when reading backwards with mixed clustering column order
Aleksey Yeschenko created CASSANDRA-14910:
-
Summary: Don't skip entire sstables when reading backwards with
mixed clustering column order
Key: CASSANDRA-14910
URL: https://issues.apache.org/jira/browse/CASSANDRA-14910
Project: Cassandra
Issue Type: Bug
Components: Local Write-Read Paths
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Fix For: 2.2.x
In 2.x, if a table has clustering columns in {{DESC}} order, any SSTable that
doesn’t have any static rows in it will be skipped altogether when iterated in
reverse.
This occurs due to the logic in {{ColumnSlice.compare()}} errorneusly sorting
any empty {{ByteBuffer}} after non-empty values due to the way {{ReversedType}}
operates. In case that empty {{ByteBuffer}} is coming from a static
{{Composite}}, however, the logic breaks down. Static {{Composite}} components
must *always* sort before any non-empty value, no matter the table’s comparator.
2.0, 2.1, and 2.2 are all affected. 3.0 isn’t, but only because we skip slice
intersection logic entirely if static rows are present in the filter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14908) Deadlock occurs when executing a file selection in levelcompact
[
https://issues.apache.org/jira/browse/CASSANDRA-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas updated CASSANDRA-14908:
-
Component/s: Compaction
> Deadlock occurs when executing a file selection in levelcompact
> ---
>
> Key: CASSANDRA-14908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14908
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: version : cassandra 2.1.15
> jdk: 8
> os:suse
>Reporter: wu taiyin
>Priority: Major
> Attachments: deadlock stack.txt
>
>
> detailed exception stack as follows:
> "CompactionExecutor:33616" #142049 daemon prio=1 os_prio=4
> tid=0x7f73244cc000 nid=0x1919a waiting for monitor entry
> [0x7fa94e13b000]
> java.lang.Thread.State: {color:#FF}BLOCKED{color} (on object monitor)
> at
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:265)
> - waiting to lock <0x7faa776209b0> (a
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy)
> at
> org.apache.cassandra.db.DataTracker.notifySSTablesChanged(DataTracker.java:517)
> at org.apache.cassandra.db.DataTracker.replaceReaders(DataTracker.java:408)
> at
> org.apache.cassandra.db.DataTracker.replaceWithNewInstances(DataTracker.java:305)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.moveStarts(SSTableRewriter.java:337)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:187)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "CompactionExecutor:33615" #142048 daemon prio=1 os_prio=4
> tid=0x7f7324451800 nid=0x19199 runnable [0x7fa7a096f000]
> java.lang.Thread.State: RUNNABLE
> at java.util.HashMap.hash(HashMap.java:338)
> at java.util.HashMap.put(HashMap.java:611)
> at java.util.HashSet.add(HashSet.java:219)
> at java.util.AbstractCollection.addAll(AbstractCollection.java:344)
> at java.util.HashSet.(HashSet.java:119)
> at com.google.common.collect.Sets.newHashSet(Sets.java:218)
> at
> {color:#FF}org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:307){color}
> {color:#FF} - locked <0x7faa77620e70> (a
> org.apache.cassandra.db.compaction.LeveledManifest){color}
> {color:#FF} at{color}
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getMaximalTask(LeveledCompactionStrategy.java:101)
> at
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:90)
> - locked <0x7faa77620e30> (a
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy)
> at
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:84)
> - locked <0x7faa776209b0> (a
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:258)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "MemtableFlushWriter:22715" #142123 daemon prio=5 os_prio=0
> tid=0x7f5a3d4ba000 nid=0x24063 waiting for monitor entry
> [0x7fa8337bd000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:265)
> - waiting to lock <0x7faa776209b0> (a
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy)
> at
[jira] [Updated] (CASSANDRA-14910) Don't skip entire sstables when reading backwards with mixed clustering column order
[
https://issues.apache.org/jira/browse/CASSANDRA-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14910:
--
Reproduced In: 2.2.13, 2.1.20 (was: 2.1.20, 2.2.13)
Status: Patch Available (was: Open)
> Don't skip entire sstables when reading backwards with mixed clustering
> column order
>
>
> Key: CASSANDRA-14910
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14910
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 2.2.x
>
>
> In 2.x, if a table has clustering columns in {{DESC}} order, any SSTable that
> doesn’t have any static rows in it will be skipped altogether when iterated
> in reverse.
> This occurs due to the logic in {{ColumnSlice.compare()}} errorneusly sorting
> any empty {{ByteBuffer}} after non-empty values due to the way
> {{ReversedType}} operates. In case that empty {{ByteBuffer}} is coming from a
> static {{Composite}}, however, the logic breaks down. Static {{Composite}}
> components must *always* sort before any non-empty value, no matter the
> table’s comparator.
> 2.0, 2.1, and 2.2 are all affected. 3.0 isn’t, but only because we skip slice
> intersection logic entirely if static rows are present in the filter.
> Introduced by CASSANDRA-8502.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14910) Don't skip entire sstables when reading backwards with mixed clustering column order
[
https://issues.apache.org/jira/browse/CASSANDRA-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697107#comment-16697107
]
Aleksey Yeschenko commented on CASSANDRA-14910:
---
Code [here|https://github.com/iamaleksey/cassandra/commits/14910-2.2], CI
[here|https://circleci.com/workflow-run/fed811d6-03d4-411d-9091-ff1e2b89d56f].
> Don't skip entire sstables when reading backwards with mixed clustering
> column order
>
>
> Key: CASSANDRA-14910
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14910
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 2.2.x
>
>
> In 2.x, if a table has clustering columns in {{DESC}} order, any SSTable that
> doesn’t have any static rows in it will be skipped altogether when iterated
> in reverse.
> This occurs due to the logic in {{ColumnSlice.compare()}} errorneusly sorting
> any empty {{ByteBuffer}} after non-empty values due to the way
> {{ReversedType}} operates. In case that empty {{ByteBuffer}} is coming from a
> static {{Composite}}, however, the logic breaks down. Static {{Composite}}
> components must *always* sort before any non-empty value, no matter the
> table’s comparator.
> 2.0, 2.1, and 2.2 are all affected. 3.0 isn’t, but only because we skip slice
> intersection logic entirely if static rows are present in the filter.
> Introduced by CASSANDRA-8502.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14909) Netty IOExceptions caused by unclean client disconnects being logged at INFO instead of TRACE
[
https://issues.apache.org/jira/browse/CASSANDRA-14909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi reassigned CASSANDRA-14909:
Assignee: Sumanth Pasupuleti
> Netty IOExceptions caused by unclean client disconnects being logged at INFO
> instead of TRACE
> -
>
> Key: CASSANDRA-14909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14909
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Minor
> Fix For: 4.0
>
>
> Observed spam logs on 3.0.17 cluster with redundant Netty IOExceptions caused
> due to client-side disconnections.
> {code:java}
> INFO [epollEventLoopGroup-2-28] 2018-11-20 23:23:04,386 Message.java:619 -
> Unexpected exception during request; channel = [id: 0x12995bc1,
> L:/xxx.xx.xxx.xxx:7104 - R:/xxx.xx.xxx.xxx:33754]
> io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed:
> Connection reset by peer
> at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown
> Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> {code}
> {code:java}
> INFO [epollEventLoopGroup-2-23] 2018-11-20 13:16:33,263 Message.java:619 -
> Unexpected exception during request; channel = [id: 0x98bd7c0e,
> L:/xxx.xx.xxx.xxx:7104 - R:/xxx.xx.xx.xx:33350]
> io.netty.channel.unix.Errors$NativeIoException: readAddress() failed:
> Connection timed out
> at io.netty.channel.unix.Errors.newIOException(Errors.java:117)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at io.netty.channel.unix.Errors.ioResult(Errors.java:138)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.unix.FileDescriptor.readAddress(FileDescriptor.java:175)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.epoll.AbstractEpollChannel.doReadBytes(AbstractEpollChannel.java:238)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:926)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:397)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:302)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> {code}
> [CASSANDRA-7849|https://issues.apache.org/jira/browse/CASSANDRA-7849]
> addresses this for JAVA IO Exception like "java.io.IOException: Connection
> reset by peer", but not for Netty IOException since the exception message in
> Netty includes method name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13917) COMPACT STORAGE inserts on tables without clusterings accept hidden column1 and value columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696675#comment-16696675
]
Alex Petrov edited comment on CASSANDRA-13917 at 11/23/18 11:24 AM:
After looking a bit more, it might be that the issue is not fully fixed:
{code}
# Still throws 'Range deletions are not supported for specific columns'
DELETE value FROM %s WHERE a = 1
# Still throws 'Invalid identifier column1 for deletion (should not be a
PRIMARY KEY part)'
DELETE column1 FROM %s WHERE a = 1
# Still works:
DELETE FROM %s WHERE a = 1 and column1 = 'b'
# Still works:
SELECT value, column1 FROM %s
{code}
You're right it's not supercolumn cf, so we have only [compact
value|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/config/CFMetaData.java#L124]
that we can use.
After thinking a bit more, I'd try and explore more places where
{{getColumnDefinition}} is used in order to understand better the surface of
the problem and maybe even try looking
[here|https://github.com/apache/cassandra/blob/cassandra-3.11/test/unit/org/apache/cassandra/cql3/validation/operations/DropCompactStorageThriftTest.java]
to learn more about compact storage.
If you have any questions about compact storage, I'm happy to help either on
IRC or here.
was (Author: ifesdjeen):
After looking a bit more, I do not think the issue is fully fixed:
{code}
# Still throws 'Range deletions are not supported for specific columns'
DELETE value FROM %s WHERE a = 1
# Still throws 'Invalid identifier column1 for deletion (should not be a
PRIMARY KEY part)'
DELETE column1 FROM %s WHERE a = 1
# Still works:
DELETE FROM %s WHERE a = 1 and column1 = 'b'
# Still works:
SELECT value, column1 FROM %s
{code}
You're right it's not supercolumn cf, so we have only [compact
value|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/config/CFMetaData.java#L124]
that we can use.
After thinking a bit more, I'd try and explore more places where
{{getColumnDefinition}} is used in order to understand better the surface of
the problem and maybe even try looking
[here|https://github.com/apache/cassandra/blob/cassandra-3.11/test/unit/org/apache/cassandra/cql3/validation/operations/DropCompactStorageThriftTest.java]
to learn more about compact storage.
If you have any questions about compact storage, I'm happy to help either on
IRC or here.
> COMPACT STORAGE inserts on tables without clusterings accept hidden column1
> and value columns
> -
>
> Key: CASSANDRA-13917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13917
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Alex Petrov
>Assignee: Aleksandr Sorokoumov
>Priority: Minor
> Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13917-3.0.png, 13917-3.11.png
>
>
> Test for the issue:
> {code}
> @Test
> public void testCompactStorage() throws Throwable
> {
> createTable("CREATE TABLE %s (a int PRIMARY KEY, b int, c int) WITH
> COMPACT STORAGE");
> assertInvalid("INSERT INTO %s (a, b, c, column1) VALUES (?, ?, ?,
> ?)", 1, 1, 1, ByteBufferUtil.bytes('a'));
> // This one fails with Some clustering keys are missing: column1,
> which is still wrong
> assertInvalid("INSERT INTO %s (a, b, c, value) VALUES (?, ?, ?, ?)",
> 1, 1, 1, ByteBufferUtil.bytes('a'));
> assertInvalid("INSERT INTO %s (a, b, c, column1, value) VALUES (?, ?,
> ?, ?, ?)", 1, 1, 1, ByteBufferUtil.bytes('a'), ByteBufferUtil.bytes('b'));
> assertEmpty(execute("SELECT * FROM %s"));
> }
> {code}
> Gladly, these writes are no-op, even though they succeed.
> {{value}} and {{column1}} should be completely hidden. Fixing this one should
> be as easy as just adding validations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13917) COMPACT STORAGE inserts on tables without clusterings accept hidden column1 and value columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696675#comment-16696675
]
Alex Petrov commented on CASSANDRA-13917:
-
After looking a bit more, I do not think the issue is fully fixed:
{code}
# Still throws 'Range deletions are not supported for specific columns'
DELETE value FROM %s WHERE a = 1
# Still throws 'Invalid identifier column1 for deletion (should not be a
PRIMARY KEY part)'
DELETE column1 FROM %s WHERE a = 1
# Still works:
DELETE FROM %s WHERE a = 1 and column1 = 'b'
# Still works:
SELECT value, column1 FROM %s
{code}
You're right it's not supercolumn cf, so we have only [compact
value|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/config/CFMetaData.java#L124]
that we can use.
After thinking a bit more, I'd try and explore more places where
{{getColumnDefinition}} is used in order to understand better the surface of
the problem and maybe even try looking
[here|https://github.com/apache/cassandra/blob/cassandra-3.11/test/unit/org/apache/cassandra/cql3/validation/operations/DropCompactStorageThriftTest.java]
to learn more about compact storage.
If you have any questions about compact storage, I'm happy to help either on
IRC or here.
> COMPACT STORAGE inserts on tables without clusterings accept hidden column1
> and value columns
> -
>
> Key: CASSANDRA-13917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13917
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Alex Petrov
>Assignee: Aleksandr Sorokoumov
>Priority: Minor
> Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13917-3.0.png, 13917-3.11.png
>
>
> Test for the issue:
> {code}
> @Test
> public void testCompactStorage() throws Throwable
> {
> createTable("CREATE TABLE %s (a int PRIMARY KEY, b int, c int) WITH
> COMPACT STORAGE");
> assertInvalid("INSERT INTO %s (a, b, c, column1) VALUES (?, ?, ?,
> ?)", 1, 1, 1, ByteBufferUtil.bytes('a'));
> // This one fails with Some clustering keys are missing: column1,
> which is still wrong
> assertInvalid("INSERT INTO %s (a, b, c, value) VALUES (?, ?, ?, ?)",
> 1, 1, 1, ByteBufferUtil.bytes('a'));
> assertInvalid("INSERT INTO %s (a, b, c, column1, value) VALUES (?, ?,
> ?, ?, ?)", 1, 1, 1, ByteBufferUtil.bytes('a'), ByteBufferUtil.bytes('b'));
> assertEmpty(execute("SELECT * FROM %s"));
> }
> {code}
> Gladly, these writes are no-op, even though they succeed.
> {{value}} and {{column1}} should be completely hidden. Fixing this one should
> be as easy as just adding validations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14909) Netty IOExceptions caused by unclean client disconnects being logged at INFO instead of TRACE
[
https://issues.apache.org/jira/browse/CASSANDRA-14909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-14909:
-
Fix Version/s: 3.0.x
> Netty IOExceptions caused by unclean client disconnects being logged at INFO
> instead of TRACE
> -
>
> Key: CASSANDRA-14909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14909
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Minor
> Fix For: 4.0, 3.0.x
>
>
> Observed spam logs on 3.0.17 cluster with redundant Netty IOExceptions caused
> due to client-side disconnections.
> {code:java}
> INFO [epollEventLoopGroup-2-28] 2018-11-20 23:23:04,386 Message.java:619 -
> Unexpected exception during request; channel = [id: 0x12995bc1,
> L:/xxx.xx.xxx.xxx:7104 - R:/xxx.xx.xxx.xxx:33754]
> io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed:
> Connection reset by peer
> at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown
> Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> {code}
> {code:java}
> INFO [epollEventLoopGroup-2-23] 2018-11-20 13:16:33,263 Message.java:619 -
> Unexpected exception during request; channel = [id: 0x98bd7c0e,
> L:/xxx.xx.xxx.xxx:7104 - R:/xxx.xx.xx.xx:33350]
> io.netty.channel.unix.Errors$NativeIoException: readAddress() failed:
> Connection timed out
> at io.netty.channel.unix.Errors.newIOException(Errors.java:117)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at io.netty.channel.unix.Errors.ioResult(Errors.java:138)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.unix.FileDescriptor.readAddress(FileDescriptor.java:175)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.epoll.AbstractEpollChannel.doReadBytes(AbstractEpollChannel.java:238)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:926)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:397)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:302)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> {code}
> [CASSANDRA-7849|https://issues.apache.org/jira/browse/CASSANDRA-7849]
> addresses this for JAVA IO Exception like "java.io.IOException: Connection
> reset by peer", but not for Netty IOException since the exception message in
> Netty includes method name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14909) Netty IOExceptions caused by unclean client disconnects being logged at INFO instead of TRACE
[
https://issues.apache.org/jira/browse/CASSANDRA-14909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-14909:
-
Component/s: Streaming and Messaging
> Netty IOExceptions caused by unclean client disconnects being logged at INFO
> instead of TRACE
> -
>
> Key: CASSANDRA-14909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14909
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Minor
> Fix For: 4.0, 3.0.x
>
>
> Observed spam logs on 3.0.17 cluster with redundant Netty IOExceptions caused
> due to client-side disconnections.
> {code:java}
> INFO [epollEventLoopGroup-2-28] 2018-11-20 23:23:04,386 Message.java:619 -
> Unexpected exception during request; channel = [id: 0x12995bc1,
> L:/xxx.xx.xxx.xxx:7104 - R:/xxx.xx.xxx.xxx:33754]
> io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed:
> Connection reset by peer
> at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown
> Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> {code}
> {code:java}
> INFO [epollEventLoopGroup-2-23] 2018-11-20 13:16:33,263 Message.java:619 -
> Unexpected exception during request; channel = [id: 0x98bd7c0e,
> L:/xxx.xx.xxx.xxx:7104 - R:/xxx.xx.xx.xx:33350]
> io.netty.channel.unix.Errors$NativeIoException: readAddress() failed:
> Connection timed out
> at io.netty.channel.unix.Errors.newIOException(Errors.java:117)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at io.netty.channel.unix.Errors.ioResult(Errors.java:138)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.unix.FileDescriptor.readAddress(FileDescriptor.java:175)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.epoll.AbstractEpollChannel.doReadBytes(AbstractEpollChannel.java:238)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:926)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:397)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:302)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> {code}
> [CASSANDRA-7849|https://issues.apache.org/jira/browse/CASSANDRA-7849]
> addresses this for JAVA IO Exception like "java.io.IOException: Connection
> reset by peer", but not for Netty IOException since the exception message in
> Netty includes method name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14303) NetworkTopologyStrategy could have a "default replication" option
[
https://issues.apache.org/jira/browse/CASSANDRA-14303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-14303:
---
Reviewer: Jon Haddad
> NetworkTopologyStrategy could have a "default replication" option
> -
>
> Key: CASSANDRA-14303
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14303
> Project: Cassandra
> Issue Type: Improvement
> Components: Configuration
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
>
> Right now when creating a keyspace with {{NetworkTopologyStrategy}} the user
> has to manually specify the datacenters they want their data replicated to
> with parameters, e.g.:
> {noformat}
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'dc1': 3, 'dc2': 3}{noformat}
> This is a poor user interface because it requires the creator of the keyspace
> (typically a developer) to know the layout of the Cassandra cluster (which
> may or may not be controlled by them). Also, at least in my experience, folks
> typo the datacenters _all_ the time. To work around this I see a number of
> users creating automation around this where the automation describes the
> Cassandra cluster and automatically expands out to all the dcs that Cassandra
> knows about. Why can't Cassandra just do this for us, re-using the previously
> forbidden {{replication_factor}} option (for backwards compatibility):
> {noformat}
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'replication_factor': 3}{noformat}
> This would automatically replicate this Keyspace to all datacenters that are
> present in the cluster. If you need to _override_ the default you could
> supply a datacenter name, e.g.:
> {noformat}
> > CREATE KEYSPACE test WITH replication = {'class':
> > 'NetworkTopologyStrategy', 'replication_factor': 3, 'dc1': 2}
> > DESCRIBE KEYSPACE test
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'dc1': '2', 'dc2': 3} AND durable_writes = true;
> {noformat}
> On the implementation side I think this may be reasonably straightforward to
> do an auto-expansion at the time of keyspace creation (or alter), where the
> above would automatically expand to list out the datacenters. We could allow
> this to be recomputed whenever an AlterKeyspaceStatement runs so that to add
> datacenters you would just run:
> {noformat}
> ALTER KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'replication_factor': 3}{noformat}
> and this would check that if the dc's in the current schema are different you
> add in the new ones (_for safety reasons we'd never remove non explicitly
> supplied zero dcs when auto-generating dcs_). Removing a datacenter becomes
> an alter that includes an override for the dc you want to remove (or of
> course you can always not use the auto-expansion and just use the old way):
> {noformat}
> // Tell it explicitly not to replicate to dc2
> > ALTER KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> > 'replication_factor': 3, 'dc2': 0}
> > DESCRIBE KEYSPACE test
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'dc1': '3'} AND durable_writes = true;{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14303) NetworkTopologyStrategy could have a "default replication" option
[
https://issues.apache.org/jira/browse/CASSANDRA-14303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697456#comment-16697456
]
Jon Haddad commented on CASSANDRA-14303:
The only thing I can think of that might cause an issue is if someone tried to
name their DC {{default_datacenter_replication}}. I don't _think_ that's
something that's likely to happen. I don't want to hold up this patch for a
corner case that's likely to never happen in the real world, so I'll open a bug
for it as a follow up after I merge this in.
> NetworkTopologyStrategy could have a "default replication" option
> -
>
> Key: CASSANDRA-14303
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14303
> Project: Cassandra
> Issue Type: Improvement
> Components: Configuration
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
>
> Right now when creating a keyspace with {{NetworkTopologyStrategy}} the user
> has to manually specify the datacenters they want their data replicated to
> with parameters, e.g.:
> {noformat}
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'dc1': 3, 'dc2': 3}{noformat}
> This is a poor user interface because it requires the creator of the keyspace
> (typically a developer) to know the layout of the Cassandra cluster (which
> may or may not be controlled by them). Also, at least in my experience, folks
> typo the datacenters _all_ the time. To work around this I see a number of
> users creating automation around this where the automation describes the
> Cassandra cluster and automatically expands out to all the dcs that Cassandra
> knows about. Why can't Cassandra just do this for us, re-using the previously
> forbidden {{replication_factor}} option (for backwards compatibility):
> {noformat}
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'replication_factor': 3}{noformat}
> This would automatically replicate this Keyspace to all datacenters that are
> present in the cluster. If you need to _override_ the default you could
> supply a datacenter name, e.g.:
> {noformat}
> > CREATE KEYSPACE test WITH replication = {'class':
> > 'NetworkTopologyStrategy', 'replication_factor': 3, 'dc1': 2}
> > DESCRIBE KEYSPACE test
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'dc1': '2', 'dc2': 3} AND durable_writes = true;
> {noformat}
> On the implementation side I think this may be reasonably straightforward to
> do an auto-expansion at the time of keyspace creation (or alter), where the
> above would automatically expand to list out the datacenters. We could allow
> this to be recomputed whenever an AlterKeyspaceStatement runs so that to add
> datacenters you would just run:
> {noformat}
> ALTER KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'replication_factor': 3}{noformat}
> and this would check that if the dc's in the current schema are different you
> add in the new ones (_for safety reasons we'd never remove non explicitly
> supplied zero dcs when auto-generating dcs_). Removing a datacenter becomes
> an alter that includes an override for the dc you want to remove (or of
> course you can always not use the auto-expansion and just use the old way):
> {noformat}
> // Tell it explicitly not to replicate to dc2
> > ALTER KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> > 'replication_factor': 3, 'dc2': 0}
> > DESCRIBE KEYSPACE test
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'dc1': '3'} AND durable_writes = true;{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
