[jira] [Commented] (CASSANDRA-7743) Possible C* OOM issue during long running test
[
https://issues.apache.org/jira/browse/CASSANDRA-7743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096619#comment-14096619
]
Benedict commented on CASSANDRA-7743:
-
Hmm. So, looking at this a little more closely, I think this may effectively be
a netty bug after all. It looks like no matter what pool/thread a pooled
bytebuf is allocated on, it gets returned to the pool of the thread that
_releases_ it. This means it simply accumulates indefinitely (up to the pool
limit, which defaults to 32Mb) in the SEPWorkers, since they never themselves
_allocate_, only release.
[~norman] is that analysis correct? If so, it looks like this behaviour is
somewhat unexpected and not ideal. However we can work around it for now.
Possible C* OOM issue during long running test
--
Key: CASSANDRA-7743
URL: https://issues.apache.org/jira/browse/CASSANDRA-7743
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Google Compute Engine, n1-standard-1
Reporter: Pierre Laporte
Assignee: Benedict
Fix For: 2.1.0
During a long running test, we ended up with a lot of
java.lang.OutOfMemoryError: Direct buffer memory errors on the Cassandra
instances.
Here is an example of stacktrace from system.log :
{code}
ERROR [SharedPool-Worker-1] 2014-08-11 11:09:34,610 ErrorMessage.java:218 -
Unexpected exception during request
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658) ~[na:1.7.0_25]
at java.nio.DirectByteBuffer.init(DirectByteBuffer.java:123)
~[na:1.7.0_25]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
~[na:1.7.0_25]
at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:434)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:168)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:98)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:251)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:107)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:112)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
{code}
The test consisted of a 3-nodes cluster of n1-standard-1 GCE instances (1
vCPU, 3.75 GB RAM) running cassandra-2.1.0-rc5, and a n1-standard-2 instance
running the test.
After ~2.5 days, several requests start to fail and we see the previous
stacktraces in the system.log file.
The output from linux ‘free’ and ‘meminfo’ suggest that there is still memory
available.
{code}
$ free -m
total used free sharedbuffers cached
Mem: 3702 3532169 0161854
-/+ buffers/cache: 2516 1185
Swap:0 0 0
$ head -n 4 /proc/meminfo
MemTotal:3791292 kB
MemFree: 173568 kB
Buffers: 165608 kB
Cached: 874752 kB
{code}
These errors do not affect all the queries we run. The cluster is still
responsive but is unable to display tracing information using cqlsh :
{code}
$ ./bin/nodetool --host 10.240.137.253 status duration_test
[jira] [Commented] (CASSANDRA-7743) Possible C* OOM issue during long running test
[
https://issues.apache.org/jira/browse/CASSANDRA-7743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096625#comment-14096625
]
Norman Maurer commented on CASSANDRA-7743:
--
[~benedict] hmm.. it should always get returned to the pool that it was
allocated from. Could you provide me with an easy way to reproduce ?
Possible C* OOM issue during long running test
--
Key: CASSANDRA-7743
URL: https://issues.apache.org/jira/browse/CASSANDRA-7743
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Google Compute Engine, n1-standard-1
Reporter: Pierre Laporte
Assignee: Benedict
Fix For: 2.1.0
During a long running test, we ended up with a lot of
java.lang.OutOfMemoryError: Direct buffer memory errors on the Cassandra
instances.
Here is an example of stacktrace from system.log :
{code}
ERROR [SharedPool-Worker-1] 2014-08-11 11:09:34,610 ErrorMessage.java:218 -
Unexpected exception during request
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658) ~[na:1.7.0_25]
at java.nio.DirectByteBuffer.init(DirectByteBuffer.java:123)
~[na:1.7.0_25]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
~[na:1.7.0_25]
at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:434)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:168)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:98)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:251)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:107)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:112)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
{code}
The test consisted of a 3-nodes cluster of n1-standard-1 GCE instances (1
vCPU, 3.75 GB RAM) running cassandra-2.1.0-rc5, and a n1-standard-2 instance
running the test.
After ~2.5 days, several requests start to fail and we see the previous
stacktraces in the system.log file.
The output from linux ‘free’ and ‘meminfo’ suggest that there is still memory
available.
{code}
$ free -m
total used free sharedbuffers cached
Mem: 3702 3532169 0161854
-/+ buffers/cache: 2516 1185
Swap:0 0 0
$ head -n 4 /proc/meminfo
MemTotal:3791292 kB
MemFree: 173568 kB
Buffers: 165608 kB
Cached: 874752 kB
{code}
These errors do not affect all the queries we run. The cluster is still
responsive but is unable to display tracing information using cqlsh :
{code}
$ ./bin/nodetool --host 10.240.137.253 status duration_test
Datacenter: DC1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID
Rack
UN 10.240.98.27925.17 KB 256 100.0%
41314169-eff5-465f-85ea-d501fd8f9c5e RAC1
UN 10.240.137.253 1.1 MB 256 100.0%
c706f5f9-c5f3-4d5e-95e9-a8903823827e RAC1
UN
[jira] [Commented] (CASSANDRA-7743) Possible C* OOM issue during long running test
[
https://issues.apache.org/jira/browse/CASSANDRA-7743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096631#comment-14096631
]
Benedict commented on CASSANDRA-7743:
-
I haven't got to that stage yet, I'm just analysing the code right now. It's
why I asked for your input, was hoping you could disabuse me if I'm completely
wrong. I don't 100% understand the control flow, as it doesn't make much sense
(to me) to be adding it to a different cache. However if you look in
PooledByteBuf.deallocate(), it calls PoolArena.free() to release the memory,
which in turn calls parent.threadCache.get().add() to cache its memory;
obviously the threadCache.get() is grabbing the threadlocal cache for the
thread releasing, not the source PoolThreadCache.
Also worth noting I'm not convinced that, even if I'm correct, this fully
explains the behaviour. We should only release on a different thread if an
exception occurs during processing anyway, so I'm still digging for a more
satisfactory full explanation of the behaviour.
Possible C* OOM issue during long running test
--
Key: CASSANDRA-7743
URL: https://issues.apache.org/jira/browse/CASSANDRA-7743
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Google Compute Engine, n1-standard-1
Reporter: Pierre Laporte
Assignee: Benedict
Fix For: 2.1.0
During a long running test, we ended up with a lot of
java.lang.OutOfMemoryError: Direct buffer memory errors on the Cassandra
instances.
Here is an example of stacktrace from system.log :
{code}
ERROR [SharedPool-Worker-1] 2014-08-11 11:09:34,610 ErrorMessage.java:218 -
Unexpected exception during request
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658) ~[na:1.7.0_25]
at java.nio.DirectByteBuffer.init(DirectByteBuffer.java:123)
~[na:1.7.0_25]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
~[na:1.7.0_25]
at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:434)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:168)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:98)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:251)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:107)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:112)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
{code}
The test consisted of a 3-nodes cluster of n1-standard-1 GCE instances (1
vCPU, 3.75 GB RAM) running cassandra-2.1.0-rc5, and a n1-standard-2 instance
running the test.
After ~2.5 days, several requests start to fail and we see the previous
stacktraces in the system.log file.
The output from linux ‘free’ and ‘meminfo’ suggest that there is still memory
available.
{code}
$ free -m
total used free sharedbuffers cached
Mem: 3702 3532169 0161854
-/+ buffers/cache: 2516 1185
Swap:0 0 0
$ head -n 4 /proc/meminfo
MemTotal:3791292 kB
MemFree:
[jira] [Commented] (CASSANDRA-7743) Possible C* OOM issue during long running test
[
https://issues.apache.org/jira/browse/CASSANDRA-7743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096634#comment-14096634
]
Norman Maurer commented on CASSANDRA-7743:
--
[~benedict] Yeah it add to the cache of the releasing thread that is right..
I thought you talk about return to pool.
Possible C* OOM issue during long running test
--
Key: CASSANDRA-7743
URL: https://issues.apache.org/jira/browse/CASSANDRA-7743
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Google Compute Engine, n1-standard-1
Reporter: Pierre Laporte
Assignee: Benedict
Fix For: 2.1.0
During a long running test, we ended up with a lot of
java.lang.OutOfMemoryError: Direct buffer memory errors on the Cassandra
instances.
Here is an example of stacktrace from system.log :
{code}
ERROR [SharedPool-Worker-1] 2014-08-11 11:09:34,610 ErrorMessage.java:218 -
Unexpected exception during request
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658) ~[na:1.7.0_25]
at java.nio.DirectByteBuffer.init(DirectByteBuffer.java:123)
~[na:1.7.0_25]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
~[na:1.7.0_25]
at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:434)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:168)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:98)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:251)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:107)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:112)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
{code}
The test consisted of a 3-nodes cluster of n1-standard-1 GCE instances (1
vCPU, 3.75 GB RAM) running cassandra-2.1.0-rc5, and a n1-standard-2 instance
running the test.
After ~2.5 days, several requests start to fail and we see the previous
stacktraces in the system.log file.
The output from linux ‘free’ and ‘meminfo’ suggest that there is still memory
available.
{code}
$ free -m
total used free sharedbuffers cached
Mem: 3702 3532169 0161854
-/+ buffers/cache: 2516 1185
Swap:0 0 0
$ head -n 4 /proc/meminfo
MemTotal:3791292 kB
MemFree: 173568 kB
Buffers: 165608 kB
Cached: 874752 kB
{code}
These errors do not affect all the queries we run. The cluster is still
responsive but is unable to display tracing information using cqlsh :
{code}
$ ./bin/nodetool --host 10.240.137.253 status duration_test
Datacenter: DC1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID
Rack
UN 10.240.98.27925.17 KB 256 100.0%
41314169-eff5-465f-85ea-d501fd8f9c5e RAC1
UN 10.240.137.253 1.1 MB 256 100.0%
c706f5f9-c5f3-4d5e-95e9-a8903823827e RAC1
UN 10.240.72.183 896.57 KB
[jira] [Commented] (CASSANDRA-7743) Possible C* OOM issue during long running test
[
https://issues.apache.org/jira/browse/CASSANDRA-7743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096638#comment-14096638
]
Benedict commented on CASSANDRA-7743:
-
We're conflating two pools maybe :)
I mean the pool of memory the thread can allocate from. So, to confirm I have
this right, if you have two threads A and B, A only allocating and B only
releasing, you would get memory accumulating up to max pool size in B, and A
always allocating new memory?
Possible C* OOM issue during long running test
--
Key: CASSANDRA-7743
URL: https://issues.apache.org/jira/browse/CASSANDRA-7743
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Google Compute Engine, n1-standard-1
Reporter: Pierre Laporte
Assignee: Benedict
Fix For: 2.1.0
During a long running test, we ended up with a lot of
java.lang.OutOfMemoryError: Direct buffer memory errors on the Cassandra
instances.
Here is an example of stacktrace from system.log :
{code}
ERROR [SharedPool-Worker-1] 2014-08-11 11:09:34,610 ErrorMessage.java:218 -
Unexpected exception during request
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658) ~[na:1.7.0_25]
at java.nio.DirectByteBuffer.init(DirectByteBuffer.java:123)
~[na:1.7.0_25]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
~[na:1.7.0_25]
at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:434)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:168)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:98)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:251)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:107)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:112)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
{code}
The test consisted of a 3-nodes cluster of n1-standard-1 GCE instances (1
vCPU, 3.75 GB RAM) running cassandra-2.1.0-rc5, and a n1-standard-2 instance
running the test.
After ~2.5 days, several requests start to fail and we see the previous
stacktraces in the system.log file.
The output from linux ‘free’ and ‘meminfo’ suggest that there is still memory
available.
{code}
$ free -m
total used free sharedbuffers cached
Mem: 3702 3532169 0161854
-/+ buffers/cache: 2516 1185
Swap:0 0 0
$ head -n 4 /proc/meminfo
MemTotal:3791292 kB
MemFree: 173568 kB
Buffers: 165608 kB
Cached: 874752 kB
{code}
These errors do not affect all the queries we run. The cluster is still
responsive but is unable to display tracing information using cqlsh :
{code}
$ ./bin/nodetool --host 10.240.137.253 status duration_test
Datacenter: DC1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID
Rack
UN 10.240.98.27925.17 KB 256 100.0%
[jira] [Commented] (CASSANDRA-7731) Get max values for live/tombstone cells per slice
[ https://issues.apache.org/jira/browse/CASSANDRA-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096640#comment-14096640 ] Cyril Scetbon commented on CASSANDRA-7731: -- Cool ! Can you tell me why you add Deprecated annotation to all of those new methods ? Get max values for live/tombstone cells per slice - Key: CASSANDRA-7731 URL: https://issues.apache.org/jira/browse/CASSANDRA-7731 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Cyril Scetbon Assignee: Robert Stupp Priority: Minor Fix For: 2.0.11, 2.1.1 Attachments: 7731-2.0.txt, 7731-2.1.txt I think you should not say that slice statistics are valid for the [last five minutes |https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java#L955-L956] in CFSTATS command of nodetool. I've read the documentation from yammer for Histograms and there is no way to force values to expire after x minutes except by [clearing|http://grepcode.com/file/repo1.maven.org/maven2/com.yammer.metrics/metrics-core/2.1.2/com/yammer/metrics/core/Histogram.java#96] it . The only thing I can see is that the last snapshot used to provide the median (or whatever you'd used instead) value is based on 1028 values. I think we should also be able to detect that some requests are accessing a lot of live/tombstone cells per query and that's not possible for now without activating DEBUG for SliceQueryFilter for example and by tweaking the threshold. Currently as nodetool cfstats returns the median if a low part of the queries are scanning a lot of live/tombstone cells we miss it ! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7743) Possible C* OOM issue during long running test
[
https://issues.apache.org/jira/browse/CASSANDRA-7743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096647#comment-14096647
]
Norman Maurer commented on CASSANDRA-7743:
--
[~benedict] well it will be released after a while if not used. But I think for
your use-case it would be best to disable the cache which can be done via the
PooledByteBufAllocator constructor just pass in 0 for int tinyCacheSize, int
smallCacheSize, int normalCacheSize.
Possible C* OOM issue during long running test
--
Key: CASSANDRA-7743
URL: https://issues.apache.org/jira/browse/CASSANDRA-7743
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Google Compute Engine, n1-standard-1
Reporter: Pierre Laporte
Assignee: Benedict
Fix For: 2.1.0
During a long running test, we ended up with a lot of
java.lang.OutOfMemoryError: Direct buffer memory errors on the Cassandra
instances.
Here is an example of stacktrace from system.log :
{code}
ERROR [SharedPool-Worker-1] 2014-08-11 11:09:34,610 ErrorMessage.java:218 -
Unexpected exception during request
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658) ~[na:1.7.0_25]
at java.nio.DirectByteBuffer.init(DirectByteBuffer.java:123)
~[na:1.7.0_25]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
~[na:1.7.0_25]
at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:434)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:168)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:98)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:251)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:107)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:112)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
{code}
The test consisted of a 3-nodes cluster of n1-standard-1 GCE instances (1
vCPU, 3.75 GB RAM) running cassandra-2.1.0-rc5, and a n1-standard-2 instance
running the test.
After ~2.5 days, several requests start to fail and we see the previous
stacktraces in the system.log file.
The output from linux ‘free’ and ‘meminfo’ suggest that there is still memory
available.
{code}
$ free -m
total used free sharedbuffers cached
Mem: 3702 3532169 0161854
-/+ buffers/cache: 2516 1185
Swap:0 0 0
$ head -n 4 /proc/meminfo
MemTotal:3791292 kB
MemFree: 173568 kB
Buffers: 165608 kB
Cached: 874752 kB
{code}
These errors do not affect all the queries we run. The cluster is still
responsive but is unable to display tracing information using cqlsh :
{code}
$ ./bin/nodetool --host 10.240.137.253 status duration_test
Datacenter: DC1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID
Rack
UN 10.240.98.27925.17 KB 256 100.0%
[jira] [Comment Edited] (CASSANDRA-7731) Get max values for live/tombstone cells per slice
[
https://issues.apache.org/jira/browse/CASSANDRA-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096640#comment-14096640
]
Cyril Scetbon edited comment on CASSANDRA-7731 at 8/14/14 6:35 AM:
---
{quote}
In fact these do not reset the backing timers and counters but use the previous
timer+counter values as an offset. (Hope it's clear that way)
{quote}
If I understand you well, launching the command each day or 1 day a week, at
the end of the week we would get same values, right ?
Cool to see that the patch is really short ! Can you tell me why you add
Deprecated annotation to all of those new methods ?
was (Author: cscetbon):
Cool ! Can you tell me why you add Deprecated annotation to all of those new
methods ?
Get max values for live/tombstone cells per slice
-
Key: CASSANDRA-7731
URL: https://issues.apache.org/jira/browse/CASSANDRA-7731
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Cyril Scetbon
Assignee: Robert Stupp
Priority: Minor
Fix For: 2.0.11, 2.1.1
Attachments: 7731-2.0.txt, 7731-2.1.txt
I think you should not say that slice statistics are valid for the [last five
minutes
|https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java#L955-L956]
in CFSTATS command of nodetool. I've read the documentation from yammer for
Histograms and there is no way to force values to expire after x minutes
except by
[clearing|http://grepcode.com/file/repo1.maven.org/maven2/com.yammer.metrics/metrics-core/2.1.2/com/yammer/metrics/core/Histogram.java#96]
it . The only thing I can see is that the last snapshot used to provide the
median (or whatever you'd used instead) value is based on 1028 values.
I think we should also be able to detect that some requests are accessing a
lot of live/tombstone cells per query and that's not possible for now without
activating DEBUG for SliceQueryFilter for example and by tweaking the
threshold. Currently as nodetool cfstats returns the median if a low part of
the queries are scanning a lot of live/tombstone cells we miss it !
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7743) Possible C* OOM issue during long running test
[
https://issues.apache.org/jira/browse/CASSANDRA-7743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096651#comment-14096651
]
Benedict commented on CASSANDRA-7743:
-
bq. well it will be released after a while if not used.
how long? it shouldn't ever be used, and it looks like it accumulates gigabytes
in total over the course of a few days (around 16-32Mb per thread)
bq. just pass in 0 for int tinyCacheSize, int smallCacheSize, int
normalCacheSize.
Won't that obviate most of the benefit of the pooled buffers?
I plan to simply prevent our deallocating on the other threads.
Possible C* OOM issue during long running test
--
Key: CASSANDRA-7743
URL: https://issues.apache.org/jira/browse/CASSANDRA-7743
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Google Compute Engine, n1-standard-1
Reporter: Pierre Laporte
Assignee: Benedict
Fix For: 2.1.0
During a long running test, we ended up with a lot of
java.lang.OutOfMemoryError: Direct buffer memory errors on the Cassandra
instances.
Here is an example of stacktrace from system.log :
{code}
ERROR [SharedPool-Worker-1] 2014-08-11 11:09:34,610 ErrorMessage.java:218 -
Unexpected exception during request
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658) ~[na:1.7.0_25]
at java.nio.DirectByteBuffer.init(DirectByteBuffer.java:123)
~[na:1.7.0_25]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
~[na:1.7.0_25]
at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:434)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:168)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:98)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:251)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:107)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:112)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
~[netty-all-4.0.20.Final.jar:4.0.20.Final]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
{code}
The test consisted of a 3-nodes cluster of n1-standard-1 GCE instances (1
vCPU, 3.75 GB RAM) running cassandra-2.1.0-rc5, and a n1-standard-2 instance
running the test.
After ~2.5 days, several requests start to fail and we see the previous
stacktraces in the system.log file.
The output from linux ‘free’ and ‘meminfo’ suggest that there is still memory
available.
{code}
$ free -m
total used free sharedbuffers cached
Mem: 3702 3532169 0161854
-/+ buffers/cache: 2516 1185
Swap:0 0 0
$ head -n 4 /proc/meminfo
MemTotal:3791292 kB
MemFree: 173568 kB
Buffers: 165608 kB
Cached: 874752 kB
{code}
These errors do not affect all the queries we run. The cluster is still
responsive but is unable to display tracing information using cqlsh :
{code}
$ ./bin/nodetool --host 10.240.137.253 status duration_test
Datacenter: DC1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns
[jira] [Commented] (CASSANDRA-7731) Get max values for live/tombstone cells per slice
[ https://issues.apache.org/jira/browse/CASSANDRA-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1409#comment-1409 ] Robert Stupp commented on CASSANDRA-7731: - bq. Can you tell me why you add Deprecated annotations For the same reason there are two patches :) The mbean code has been refactored in 2.1 and has been deprecated in 2.0 (or even earlier) to prepare that. So the 2.1 patch is just very small but does the same as the 2.0 patch. Get max values for live/tombstone cells per slice - Key: CASSANDRA-7731 URL: https://issues.apache.org/jira/browse/CASSANDRA-7731 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Cyril Scetbon Assignee: Robert Stupp Priority: Minor Fix For: 2.0.11, 2.1.1 Attachments: 7731-2.0.txt, 7731-2.1.txt I think you should not say that slice statistics are valid for the [last five minutes |https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java#L955-L956] in CFSTATS command of nodetool. I've read the documentation from yammer for Histograms and there is no way to force values to expire after x minutes except by [clearing|http://grepcode.com/file/repo1.maven.org/maven2/com.yammer.metrics/metrics-core/2.1.2/com/yammer/metrics/core/Histogram.java#96] it . The only thing I can see is that the last snapshot used to provide the median (or whatever you'd used instead) value is based on 1028 values. I think we should also be able to detect that some requests are accessing a lot of live/tombstone cells per query and that's not possible for now without activating DEBUG for SliceQueryFilter for example and by tweaking the threshold. Currently as nodetool cfstats returns the median if a low part of the queries are scanning a lot of live/tombstone cells we miss it ! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7731) Get max values for live/tombstone cells per slice
[ https://issues.apache.org/jira/browse/CASSANDRA-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096674#comment-14096674 ] Cyril Scetbon commented on CASSANDRA-7731: -- Ok, I usually use Deprecated only when there's already in the current code another way to get the same thing done with a new code version :) Get max values for live/tombstone cells per slice - Key: CASSANDRA-7731 URL: https://issues.apache.org/jira/browse/CASSANDRA-7731 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Cyril Scetbon Assignee: Robert Stupp Priority: Minor Fix For: 2.0.11, 2.1.1 Attachments: 7731-2.0.txt, 7731-2.1.txt I think you should not say that slice statistics are valid for the [last five minutes |https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java#L955-L956] in CFSTATS command of nodetool. I've read the documentation from yammer for Histograms and there is no way to force values to expire after x minutes except by [clearing|http://grepcode.com/file/repo1.maven.org/maven2/com.yammer.metrics/metrics-core/2.1.2/com/yammer/metrics/core/Histogram.java#96] it . The only thing I can see is that the last snapshot used to provide the median (or whatever you'd used instead) value is based on 1028 values. I think we should also be able to detect that some requests are accessing a lot of live/tombstone cells per query and that's not possible for now without activating DEBUG for SliceQueryFilter for example and by tweaking the threshold. Currently as nodetool cfstats returns the median if a low part of the queries are scanning a lot of live/tombstone cells we miss it ! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7731) Get max values for live/tombstone cells per slice
[ https://issues.apache.org/jira/browse/CASSANDRA-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096681#comment-14096681 ] Robert Stupp commented on CASSANDRA-7731: - Yea - please don't nail me on the exact versions (it's actually deprecated a long time) - but it has been refactored :) Get max values for live/tombstone cells per slice - Key: CASSANDRA-7731 URL: https://issues.apache.org/jira/browse/CASSANDRA-7731 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Cyril Scetbon Assignee: Robert Stupp Priority: Minor Fix For: 2.0.11, 2.1.1 Attachments: 7731-2.0.txt, 7731-2.1.txt I think you should not say that slice statistics are valid for the [last five minutes |https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java#L955-L956] in CFSTATS command of nodetool. I've read the documentation from yammer for Histograms and there is no way to force values to expire after x minutes except by [clearing|http://grepcode.com/file/repo1.maven.org/maven2/com.yammer.metrics/metrics-core/2.1.2/com/yammer/metrics/core/Histogram.java#96] it . The only thing I can see is that the last snapshot used to provide the median (or whatever you'd used instead) value is based on 1028 values. I think we should also be able to detect that some requests are accessing a lot of live/tombstone cells per query and that's not possible for now without activating DEBUG for SliceQueryFilter for example and by tweaking the threshold. Currently as nodetool cfstats returns the median if a low part of the queries are scanning a lot of live/tombstone cells we miss it ! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7762) range query on secondary index times out (1 row, 1 node)
[
https://issues.apache.org/jira/browse/CASSANDRA-7762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7762:
Assignee: Tyler Hobbs
range query on secondary index times out (1 row, 1 node)
Key: CASSANDRA-7762
URL: https://issues.apache.org/jira/browse/CASSANDRA-7762
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Mac OSX 10.7.5
Fresh install of Cassandra 2.0.9
Reporter: Ian Rose
Assignee: Tyler Hobbs
Attachments: query-trace.txt
Setup:
- single cassandra node (on localhost)
- in cassandra.yaml, listen_address: 127.0.0.1 and seeds: 127.0.0.1
Here is my test schema:
CREATE TABLE foo (
foo_name ascii,
foo_shard bigint,
int_val bigint,
PRIMARY KEY ((foo_name, foo_shard))
) WITH read_repair_chance=0.1;
CREATE INDEX ON foo (int_val);
CREATE INDEX ON foo (foo_name);
I have inserted just a single row into this table:
insert into foo(foo_name, foo_shard, int_val) values('dave', 27, 100);
This query works fine:
select * from foo where foo_name='dave';
But when I run this query, I get an RPC timeout:
select * from foo where foo_name='dave' and int_val 0 allow filtering;
Request did not complete within rpc_timeout.
With tracing enabled, here is the trace output (also attached):
http://pastebin.com/raw.php?i=6XMEVUcQ
(In short, everything looks fine to my untrained eye until 10s elapsed, at
which time the following event is logged: Timed out; received 0 of 1
responses for range 257 of 257)
Also note that if I *remove* the index on int_val (leaving just the index on
foo_name), then the query works fine:
select * from foo where foo_name='dave' and int_val 0 allow filtering;
foo_name | foo_shard | int_val
--+---+-
dave |27 | 100
(1 rows)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7762) range query on secondary index times out (1 row, 1 node)
[
https://issues.apache.org/jira/browse/CASSANDRA-7762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7762:
Assignee: (was: Tyler Hobbs)
range query on secondary index times out (1 row, 1 node)
Key: CASSANDRA-7762
URL: https://issues.apache.org/jira/browse/CASSANDRA-7762
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Mac OSX 10.7.5
Fresh install of Cassandra 2.0.9
Reporter: Ian Rose
Attachments: query-trace.txt
Setup:
- single cassandra node (on localhost)
- in cassandra.yaml, listen_address: 127.0.0.1 and seeds: 127.0.0.1
Here is my test schema:
CREATE TABLE foo (
foo_name ascii,
foo_shard bigint,
int_val bigint,
PRIMARY KEY ((foo_name, foo_shard))
) WITH read_repair_chance=0.1;
CREATE INDEX ON foo (int_val);
CREATE INDEX ON foo (foo_name);
I have inserted just a single row into this table:
insert into foo(foo_name, foo_shard, int_val) values('dave', 27, 100);
This query works fine:
select * from foo where foo_name='dave';
But when I run this query, I get an RPC timeout:
select * from foo where foo_name='dave' and int_val 0 allow filtering;
Request did not complete within rpc_timeout.
With tracing enabled, here is the trace output (also attached):
http://pastebin.com/raw.php?i=6XMEVUcQ
(In short, everything looks fine to my untrained eye until 10s elapsed, at
which time the following event is logged: Timed out; received 0 of 1
responses for range 257 of 257)
Also note that if I *remove* the index on int_val (leaving just the index on
foo_name), then the query works fine:
select * from foo where foo_name='dave' and int_val 0 allow filtering;
foo_name | foo_shard | int_val
--+---+-
dave |27 | 100
(1 rows)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (CASSANDRA-7762) range query on secondary index times out (1 row, 1 node)
[
https://issues.apache.org/jira/browse/CASSANDRA-7762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne resolved CASSANDRA-7762.
-
Resolution: Duplicate
This is a duplicate of CASSANDRA-6612. The same AssertionError shows in the
server log in particular (and the trace confirms that the no index is used
while it should have). I've confirmed that this does work properly on the
current cassandra-2.0 branch.
range query on secondary index times out (1 row, 1 node)
Key: CASSANDRA-7762
URL: https://issues.apache.org/jira/browse/CASSANDRA-7762
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Mac OSX 10.7.5
Fresh install of Cassandra 2.0.9
Reporter: Ian Rose
Attachments: query-trace.txt
Setup:
- single cassandra node (on localhost)
- in cassandra.yaml, listen_address: 127.0.0.1 and seeds: 127.0.0.1
Here is my test schema:
CREATE TABLE foo (
foo_name ascii,
foo_shard bigint,
int_val bigint,
PRIMARY KEY ((foo_name, foo_shard))
) WITH read_repair_chance=0.1;
CREATE INDEX ON foo (int_val);
CREATE INDEX ON foo (foo_name);
I have inserted just a single row into this table:
insert into foo(foo_name, foo_shard, int_val) values('dave', 27, 100);
This query works fine:
select * from foo where foo_name='dave';
But when I run this query, I get an RPC timeout:
select * from foo where foo_name='dave' and int_val 0 allow filtering;
Request did not complete within rpc_timeout.
With tracing enabled, here is the trace output (also attached):
http://pastebin.com/raw.php?i=6XMEVUcQ
(In short, everything looks fine to my untrained eye until 10s elapsed, at
which time the following event is logged: Timed out; received 0 of 1
responses for range 257 of 257)
Also note that if I *remove* the index on int_val (leaving just the index on
foo_name), then the query works fine:
select * from foo where foo_name='dave' and int_val 0 allow filtering;
foo_name | foo_shard | int_val
--+---+-
dave |27 | 100
(1 rows)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7499) Unable to update list element by index using CAS condition
[
https://issues.apache.org/jira/browse/CASSANDRA-7499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7499:
Attachment: 7499-2.0.txt
Attaching simple throw IRE patch for 2.0 too.
bq. The cas_and_list_index_test is failing on 2.1
Right, that's the test for this issue and this hasn't been committed yet. The
test shouldn't have been run, it's my fault for commenting out the
'@require(7499)' on the test. I've re-enabled it (I'll re-remove it once it's
committed).
Unable to update list element by index using CAS condition
--
Key: CASSANDRA-7499
URL: https://issues.apache.org/jira/browse/CASSANDRA-7499
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassandra *2.0.9*, Java Driver *2.0.2* *2.0.3*
Client: cqlsh *3.1.8*, CQL spec *3.1.0*, Thrift protocol *19.39.0*
Reporter: DOAN DuyHai
Assignee: Sylvain Lebresne
Fix For: 2.0.10
Attachments: 7499-2.0.txt, 7499-2.1.txt, 7499.txt
While running IT tests for *Achilles*, I ran into a strange bug:
*With CQLSH*
{code:sql}
cqlsh:test CREATE TABLE cas_update(id int PRIMARY KEY,name text,friends
listtext);
cqlsh:test INSERT INTO cas_update (id, name , friends ) VALUES (
10,'John',['Paul','George']);
cqlsh:test SELECT * FROM cas_update WHERE id=10;
id | friends| name
++--
10 | ['Paul', 'George'] | John
cqlsh:test UPDATE cas_update SET friends[0]='Helen' WHERE id=10 IF
name='John';
Bad Request: List index 0 out of bound, list has size 0
cqlsh:test UPDATE cas_update SET friends[0]='Helen' WHERE id=10;
cqlsh:test SELECT * FROM cas_update WHERE id=10;
id | friends | name
+-+--
10 | ['Helen', 'George'] | John
{code}
It seems that we cannot update list element by index with a CAS condition.
*With Java driver 2.0.2 or 2.0.3*
{code:java}
ACHILLES_DML_STATEMENT@:writeDMLStatementLog Prepared statement : [INSERT
INTO CompleteBean(id,followers,friends,name,preferences) VALUES
(:id,:followers,:friends,:name,:preferences) USING TTL :ttl;] with
CONSISTENCY LEVEL [ONE]
ACHILLES_DML_STATEMENT@:writeDMLStatementLogbound values :
[621309709026375591, [], [Paul, Andrew], John, {}, 0]
ACHILLES_DML_STATEMENT@:writeDMLStartBatch
ACHILLES_DML_STATEMENT@:writeDMLStartBatch
ACHILLES_DML_STATEMENT@:writeDMLStartBatch ** BATCH UNLOGGED START
**
ACHILLES_DML_STATEMENT@:writeDMLStartBatch
ACHILLES_DML_STATEMENT@:writeDMLStatementLog Parameterized statement :
[UPDATE CompleteBean USING TTL 100 SET friends[0]=? WHERE
id=621309709026375591 IF name=?;] with CONSISTENCY LEVEL [ONE]
ACHILLES_DML_STATEMENT@:writeDMLStatementLogbound values : [100, 0,
Helen, 621309709026375591, John]
ACHILLES_DML_STATEMENT@:writeDMLStatementLog Parameterized statement :
[UPDATE CompleteBean USING TTL 100 SET friends[1]=null WHERE
id=621309709026375591 IF name=?;] with CONSISTENCY LEVEL [ONE]
ACHILLES_DML_STATEMENT@:writeDMLStatementLogbound values : [100, 1,
null, 621309709026375591, John]
ACHILLES_DML_STATEMENT@:writeDMLEndBatch
ACHILLES_DML_STATEMENT@:writeDMLEndBatch ** BATCH UNLOGGED END with
CONSISTENCY LEVEL [DEFAULT] **
ACHILLES_DML_STATEMENT@:writeDMLEndBatch
ACHILLES_DML_STATEMENT@:writeDMLEndBatch
ACHILLES_DML_STATEMENT@:truncateTable Simple query : [TRUNCATE
entity_with_enum] with CONSISTENCY LEVEL [ALL]
ACHILLES_DML_STATEMENT@:truncateTable Simple query : [TRUNCATE
CompleteBean] with CONSISTENCY LEVEL [ALL]
com.datastax.driver.core.exceptions.InvalidQueryException: List index 0 out
of bound, list has size 0
at
com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:35)
at
com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:256)
at
com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:172)
at
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
{code}
With Cassandra *2.0.8* and Java Driver 2.0.2 or 2.0.3, *the test passed* so
it seems that there is a regression somewhere in the CAS update code
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7766) Secondary index not working after a while
Fabrice Larcher created CASSANDRA-7766: -- Summary: Secondary index not working after a while Key: CASSANDRA-7766 URL: https://issues.apache.org/jira/browse/CASSANDRA-7766 Project: Cassandra Issue Type: Bug Environment: C* 2.1.0-rc5 with small clusters (one or two nodes) Reporter: Fabrice Larcher Since 2.1.0-rc2, it appears that the secondary indexes are not always working. Immediately after the INSERT of a row, the index seems to be there. But after a while (I do not know when or why), SELECT statements based on any secondary index do not return the corresponding row(s) anymore. I noticed that a restart of C* may have an impact (the data inserted before the restart may be seen through the index, even if it was not returned before the restart). Here is a use-case example (in order to clarify my request) : CREATE TABLE IF NOT EXISTS ks.cf ( k int PRIMARY KEY, ind ascii, value text); CREATE INDEX IF NOT EXISTS ks_cf_index ON ks.cf(ind); INSERT INTO ks.cf (k, ind, value) VALUES (1, 'toto', 'Hello'); SELECT * FROM ks.cf WHERE ind = 'toto'; // Returns no result after a while The last SELECT statement may or may not return a row depending on the instant of the request. I experienced that with 2.1.0-rc5 through CQLSH with clusters of one and two nodes. Since it depends on the instant of the request, I am not able to deliver any way to reproduce that systematically (It appears to be linked with some scheduled job inside C*). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7766) Secondary index not working after a while
[
https://issues.apache.org/jira/browse/CASSANDRA-7766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabrice Larcher updated CASSANDRA-7766:
---
Description:
Since 2.1.0-rc2, it appears that the secondary indexes are not always working.
Immediately after the INSERT of a row, the index seems to be there. But after a
while (I do not know when or why), SELECT statements based on any secondary
index do not return the corresponding row(s) anymore. I noticed that a restart
of C* may have an impact (the data inserted before the restart may be seen
through the index, even if it was not returned before the restart).
Here is a use-case example (in order to clarify my request) :
{code}
CREATE TABLE IF NOT EXISTS ks.cf ( k int PRIMARY KEY, ind ascii, value text);
CREATE INDEX IF NOT EXISTS ks_cf_index ON ks.cf(ind);
INSERT INTO ks.cf (k, ind, value) VALUES (1, 'toto', 'Hello');
SELECT * FROM ks.cf WHERE ind = 'toto'; // Returns no result after a while
{code}
The last SELECT statement may or may not return a row depending on the instant
of the request. I experienced that with 2.1.0-rc5 through CQLSH with clusters
of one and two nodes. Since it depends on the instant of the request, I am not
able to deliver any way to reproduce that systematically (It appears to be
linked with some scheduled job inside C*).
was:
Since 2.1.0-rc2, it appears that the secondary indexes are not always working.
Immediately after the INSERT of a row, the index seems to be there. But after a
while (I do not know when or why), SELECT statements based on any secondary
index do not return the corresponding row(s) anymore. I noticed that a restart
of C* may have an impact (the data inserted before the restart may be seen
through the index, even if it was not returned before the restart).
Here is a use-case example (in order to clarify my request) :
CREATE TABLE IF NOT EXISTS ks.cf ( k int PRIMARY KEY, ind ascii, value text);
CREATE INDEX IF NOT EXISTS ks_cf_index ON ks.cf(ind);
INSERT INTO ks.cf (k, ind, value) VALUES (1, 'toto', 'Hello');
SELECT * FROM ks.cf WHERE ind = 'toto'; // Returns no result after a while
The last SELECT statement may or may not return a row depending on the instant
of the request. I experienced that with 2.1.0-rc5 through CQLSH with clusters
of one and two nodes. Since it depends on the instant of the request, I am not
able to deliver any way to reproduce that systematically (It appears to be
linked with some scheduled job inside C*).
Attachment: result-success.txt
result-failure.txt
You will find some traces attached ; one file the success case and one for the
failure case. Here are some information about them ;
*CQL structure :*
A table {{user}} with an index {{user_email}} on column {{email}} of type
{{ascii}}.
*CQLSH commands :*
{code}
use main;
tracing on;
consistency all;
select * from user where email = 'fabrice.larc...@level5.fr';
{code}
I hope it could help
Secondary index not working after a while
-
Key: CASSANDRA-7766
URL: https://issues.apache.org/jira/browse/CASSANDRA-7766
Project: Cassandra
Issue Type: Bug
Environment: C* 2.1.0-rc5 with small clusters (one or two nodes)
Reporter: Fabrice Larcher
Attachments: result-failure.txt, result-success.txt
Since 2.1.0-rc2, it appears that the secondary indexes are not always
working. Immediately after the INSERT of a row, the index seems to be there.
But after a while (I do not know when or why), SELECT statements based on any
secondary index do not return the corresponding row(s) anymore. I noticed
that a restart of C* may have an impact (the data inserted before the restart
may be seen through the index, even if it was not returned before the
restart).
Here is a use-case example (in order to clarify my request) :
{code}
CREATE TABLE IF NOT EXISTS ks.cf ( k int PRIMARY KEY, ind ascii, value text);
CREATE INDEX IF NOT EXISTS ks_cf_index ON ks.cf(ind);
INSERT INTO ks.cf (k, ind, value) VALUES (1, 'toto', 'Hello');
SELECT * FROM ks.cf WHERE ind = 'toto'; // Returns no result after a while
{code}
The last SELECT statement may or may not return a row depending on the
instant of the request. I experienced that with 2.1.0-rc5 through CQLSH with
clusters of one and two nodes. Since it depends on the instant of the
request, I am not able to deliver any way to reproduce that systematically
(It appears to be linked with some scheduled job inside C*).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7704) FileNotFoundException during STREAM-OUT triggers 100% CPU usage
[ https://issues.apache.org/jira/browse/CASSANDRA-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096779#comment-14096779 ] Benedict commented on CASSANDRA-7704: - Not cleaning up resources is really not ideal in my book, however there is absolutely no reason we need to cancel with interruption - note this does *not* always result in a cancelled state if it ran, only if it was in the middle of running at the time (but still completed), and this can be fixed by not permitting it to be interrupted. However this is not the problem - in the test it will often be the case that the task was _genuinely_ successfully cancelled. In my opinion the test is broken, since previously there was _no_ guarantee that all cancellations would run (although the cancellation in the test case will); after the last task completes successfully the scheduled tasks were all removed from the queue (but *not* cancelled), so the behaviour of the future in this case would be to never return, which is much more surprising and inconsistent in my book. I'm not entirely sure what the offending line is intended to test, anyway? FileNotFoundException during STREAM-OUT triggers 100% CPU usage --- Key: CASSANDRA-7704 URL: https://issues.apache.org/jira/browse/CASSANDRA-7704 Project: Cassandra Issue Type: Bug Reporter: Rick Branson Assignee: Benedict Attachments: 7704.20.v2.txt, 7704.txt, backtrace.txt, other-errors.txt See attached backtrace which was what triggered this. This stream failed and then ~12 seconds later it emitted that exception. At that point, all CPUs went to 100%. A thread dump shows all the ReadStage threads stuck inside IntervalTree.searchInternal inside of CFS.markReferenced(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6726) Recycle CompressedRandomAccessReader/RandomAccessReader buffers independently of their owners, and move them off-heap when possible
[ https://issues.apache.org/jira/browse/CASSANDRA-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6726: Assignee: Branimir Lambov Recycle CompressedRandomAccessReader/RandomAccessReader buffers independently of their owners, and move them off-heap when possible --- Key: CASSANDRA-6726 URL: https://issues.apache.org/jira/browse/CASSANDRA-6726 Project: Cassandra Issue Type: Improvement Reporter: Benedict Assignee: Branimir Lambov Priority: Minor Labels: performance Fix For: 3.0 Whilst CRAR and RAR are pooled, we could and probably should pool the buffers independently, so that they are not tied to a specific sstable. It may be possible to move the RAR buffer off-heap, and the CRAR sometimes (e.g. Snappy may possibly support off-heap buffers) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-5902) Dealing with hints after a topology change
[ https://issues.apache.org/jira/browse/CASSANDRA-5902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-5902: Assignee: Branimir Lambov Dealing with hints after a topology change -- Key: CASSANDRA-5902 URL: https://issues.apache.org/jira/browse/CASSANDRA-5902 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Branimir Lambov Priority: Minor Hints are stored and delivered by destination node id. This allows them to survive IP changes in the target, while making scan all the hints for a given destination an efficient operation. However, we do not detect and handle new node assuming responsibility for the hinted row via bootstrap before it can be delivered. I think we have to take a performance hit in this case -- we need to deliver such a hint to *all* replicas, since we don't know which is the new one. This happens infrequently enough, however -- requiring first the target node to be down to create the hint, then the hint owner to be down long enough for the target to both recover and stream to a new node -- that this should be okay. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7039) DirectByteBuffer compatible LZ4 methods
[ https://issues.apache.org/jira/browse/CASSANDRA-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-7039: Assignee: Branimir Lambov (was: Lyuben Todorov) DirectByteBuffer compatible LZ4 methods --- Key: CASSANDRA-7039 URL: https://issues.apache.org/jira/browse/CASSANDRA-7039 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Branimir Lambov Priority: Minor Labels: performance Fix For: 3.0 As we move more things off-heap, it's becoming more and more essential to be able to use DirectByteBuffer (or native pointers) in various places. Unfortunately LZ4 doesn't currently support this operation, despite being JNI based - this means we both have to perform unnecessary copies to de/compress data from DBB, but also we can stall GC as any JNI method operating over a java array using the GetPrimitiveArrayCritical enters a critical section that prevents GC for its duration. This means STWs will be at least as long any running compression/decompression (and no GC will happen until they complete, so it's additive). We should temporarily fork (and then resubmit upstream) jpountz-lz4 to support operating over a native pointer, so that we can pass a DBB or a raw pointer we have allocated ourselves. This will help improve performance when flushing the new offheap memtables, as well as enable us to implement CASSANDRA-6726 and finish CASSANDRA-4338. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-7546: Fix Version/s: 2.1.1 2.0.11 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory - Key: CASSANDRA-7546 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 Project: Cassandra Issue Type: Bug Components: Core Reporter: graham sanderson Assignee: graham sanderson Fix For: 2.0.11, 2.1.1 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, 7546.20_alt.txt, suggestion1.txt, suggestion1_21.txt In order to preserve atomicity, this code attempts to read, clone/update, then CAS the state of the partition. Under heavy contention for updating a single partition this can cause some fairly staggering memory growth (the more cores on your machine the worst it gets). Whilst many usage patterns don't do highly concurrent updates to the same partition, hinting today, does, and in this case wild (order(s) of magnitude more than expected) memory allocation rates can be seen (especially when the updates being hinted are small updates to different partitions which can happen very fast on their own) - see CASSANDRA-7545 It would be best to eliminate/reduce/limit the spinning memory allocation whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7561) On DROP we should invalidate CounterKeyCache as well as Key/Row cache
[ https://issues.apache.org/jira/browse/CASSANDRA-7561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-7561: Fix Version/s: 2.1.0 On DROP we should invalidate CounterKeyCache as well as Key/Row cache - Key: CASSANDRA-7561 URL: https://issues.apache.org/jira/browse/CASSANDRA-7561 Project: Cassandra Issue Type: Bug Reporter: Benedict Assignee: Aleksey Yeschenko Priority: Minor Fix For: 2.1.0 We should also probably ensure we don't attempt to auto save _any_ of the caches while they are in an inconsistent state (i.e. there are keys present to be saved that should not be restored, or that would throw exceptions when we save (e.g. CounterCacheKey)) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7704) FileNotFoundException during STREAM-OUT triggers 100% CPU usage
[ https://issues.apache.org/jira/browse/CASSANDRA-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-7704: Fix Version/s: 2.1.0 2.0.10 FileNotFoundException during STREAM-OUT triggers 100% CPU usage --- Key: CASSANDRA-7704 URL: https://issues.apache.org/jira/browse/CASSANDRA-7704 Project: Cassandra Issue Type: Bug Reporter: Rick Branson Assignee: Benedict Fix For: 2.0.10, 2.1.0 Attachments: 7704.20.v2.txt, 7704.txt, backtrace.txt, other-errors.txt See attached backtrace which was what triggered this. This stream failed and then ~12 seconds later it emitted that exception. At that point, all CPUs went to 100%. A thread dump shows all the ReadStage threads stuck inside IntervalTree.searchInternal inside of CFS.markReferenced(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-3852) use LIFO queueing policy when queue size exceeds thresholds
[
https://issues.apache.org/jira/browse/CASSANDRA-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict updated CASSANDRA-3852:
Fix Version/s: 3.0
use LIFO queueing policy when queue size exceeds thresholds
---
Key: CASSANDRA-3852
URL: https://issues.apache.org/jira/browse/CASSANDRA-3852
Project: Cassandra
Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Labels: performance
Fix For: 3.0
A strict FIFO policy for queueing (between stages) is detrimental to latency
and forward progress. Whenever a node is saturated beyond incoming request
rate, *all* requests become slow. If it is consistently saturated, you start
effectively timing out on *all* requests.
A much better strategy from the point of view of latency is to serve a subset
requests quickly, and letting some time out, rather than letting all either
time out or be slow.
Care must be taken such that:
* We still guarantee that requests are processed reasonably timely (we
couldn't go strict LIFO for example as that would result in requests getting
stuck potentially forever on a loaded node).
* Maybe, depending on the previous point's solution, ensure that some
requests bypass the policy and get prioritized (e.g., schema migrations, or
anything internal to a node).
A possible implementation is to go LIFO whenever there are requests in the
queue that are older than N milliseconds (or a certain queue size, etc).
Benefits:
* All cases where the client is directly, or is indirectly affecting through
other layers, a system which has limited concurrency (e.g., thread pool size
of X to serve some incoming request rate), it is *much* better for a few
requests to time out while most are serviced quickly, than for all requests
to become slow, as it doesn't explode concurrency. Think any random
non-super-advanced php app, ruby web app, java servlet based app, etc.
Essentially, it optimizes very heavily for improved average latencies.
* Systems with strict p95/p99/p999 requirements on latencies should greatly
benefit from such a policy. For example, suppose you have a system at 85% of
capacity, and it takes a write spike (or has a hiccup like GC pause, blocking
on a commit log write, etc). Suppose the hiccup racks up 500 ms worth of
requests. At 15% margin at steady state, that takes 500ms * 100/15 = 3.2
seconds to recover. Instead of *all* requests for an entire 3.2 second window
being slow, we'd serve requests quickly for 2.7 of those seconds, with the
incoming requests during that 500 ms interval being the ones primarily
affected. The flip side though is that once you're at the point where more
than N percent of requests end up having to wait for others to take LIFO
priority, the p(100-N) latencies will actually be *worse* than without this
change (but at this point you have to consider what the root reason for those
pXX requirements are).
* In the case of complete saturation, it allows forward progress. Suppose
you're taking 25% more traffic than you are able to handle. Instead of
getting backed up and ending up essentially timing out *every single
request*, you will succeed in processing up to 75% of them (I say up to
because it depends; for example on a {{QUORUM}} request you need at least two
of the requests from the co-ordinator to succeed so the percentage is brought
down) and allowing clients to make forward progress and get work done, rather
than being stuck.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7542) Reduce CAS contention
[ https://issues.apache.org/jira/browse/CASSANDRA-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096805#comment-14096805 ] Benedict commented on CASSANDRA-7542: - [~kohlisankalp] any news? Reduce CAS contention - Key: CASSANDRA-7542 URL: https://issues.apache.org/jira/browse/CASSANDRA-7542 Project: Cassandra Issue Type: Improvement Reporter: sankalp kohli Assignee: Benedict Fix For: 2.0.10 CAS updates on same CQL partition can lead to heavy contention inside C*. I am looking for simple ways(no algorithmic changes) to reduce contention as the penalty of it is high in terms of latency, specially for reads. We can put some sort of synchronization on CQL partition at StorageProxy level. This will reduce contention at least for all requests landing on one box for same partition. Here is an example of why it will help: 1) Say 1 write and 2 read CAS requests for the same partition key is send to C* in parallel. 2) Since client is token-aware, it sends these 3 request to the same C* instance A. (Lets assume that all 3 requests goto same instance A) 3) In this C* instance A, all 3 CAS requests will contend with each other in Paxos. (This is bad) To improve contention in 3), what I am proposing is to add a lock on partition key similar to what we do in PaxosState.java to serialize these 3 requests. This will remove the contention and improve performance as these 3 requests will not collide with each other. Another improvement we can do in client is to pick a deterministic live replica for a given partition doing CAS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-6780) Memtable OffHeap GC Statistics
[ https://issues.apache.org/jira/browse/CASSANDRA-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict resolved CASSANDRA-6780. - Resolution: Later Since offheap GC has been postponed indefinitely, this ticket should also be closed to revisit later. Memtable OffHeap GC Statistics -- Key: CASSANDRA-6780 URL: https://issues.apache.org/jira/browse/CASSANDRA-6780 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor As mentioned in CASSANDRA-6689, it would be nice to expose via JMX some statistics on GC behaviour, instead of just optionally debug logging it (and maybe expand to cover some more things): - Time spent in GC - Amount of memory reclaimed - Number of collections (per CFS?), and average reclaimed per collection -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp reassigned CASSANDRA-7577: --- Assignee: Robert Stupp cqlsh: CTRL-R history search not working on OSX --- Key: CASSANDRA-7577 URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 Project: Cassandra Issue Type: Bug Environment: OSX - plain Terminal program C* 2.0.x, 2.1, trunk Reporter: Robert Stupp Assignee: Robert Stupp Priority: Minor _recursive-history-search_ via ctrl-R does not work in cqlsh. The history itself works via cursor up/down. It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-6709) Changes to KeyCache
[ https://issues.apache.org/jira/browse/CASSANDRA-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict resolved CASSANDRA-6709. - Resolution: Later Closing as the new sstable format most likely makes this unnecessary by eliminating the need for a separate key cache, although we *may* want to revisit this at some point afterwards since a separate cache could still be beneficial by improving memory occupancy rate, so closing as later instead of duplicate. Changes to KeyCache --- Key: CASSANDRA-6709 URL: https://issues.apache.org/jira/browse/CASSANDRA-6709 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor It seems to me that KeyCache can be improved in a number of ways, but first let's state the basic goal of KeyCache: to reduce the average query response time by providing an exact seek position in a file for a given key. As it stands, KeyCache is both 100% accurate, but requires a lot of overhead per entry. I propose to make KeyCache *mostly* accurate (say 99.%), by which I means it will always fail accurately, but may rarely return an incorrect address, and code the end users of it to be able to retry to confirm they seeked to the correct position in the file, and to retry without the cache if they did not. The advantage of this is that we can both take the cache off-heap easily, and pack a lot more items into the cache. If we permit collisions across files and simply use the (full 128-bit) murmur hash of the key + cfId + file generation, we should get enough uniqueness to rarely have an erroneuous collision, however we will be using only 20 bytes per key, instead of the current 100 + key length bytes. This should allow us to answer far more queries from the key cache than before, so the positive improvement to performance should be greater than the negative drain. For the structure I propose an associative cache, where a single contiguous address space is broken up into regions of, say, 8 entries, plus one counter. The counter tracks the recency of access of each of the entries, so that on write the least recently accessed/written can be replaced. A linear probe within the region is used to determine if the entry we're looking for is present. This should be very quick, as the entire region should fit into one or two lines of L1. Advantage: we may see 5x bump in cache hit-rate, or even more for large keys. -- This message was sent by Atlassian JIRA (v6.2#6252)
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure on builder cassandra-trunk while building libcloud. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/646 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: lares_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] 23d4355a1c86cefdeb04936265a820723a3f5c15 Blamelist: Tomaz Muraus to...@apache.org BUILD FAILED: failed sincerely, -The Buildbot
[jira] [Resolved] (CASSANDRA-6802) Row cache improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict resolved CASSANDRA-6802. - Resolution: Later Since offheap GC has been postponed indefinitely, this ticket should also be closed to revisit later. Row cache improvements -- Key: CASSANDRA-6802 URL: https://issues.apache.org/jira/browse/CASSANDRA-6802 Project: Cassandra Issue Type: Improvement Reporter: Marcus Eriksson Labels: performance Fix For: 3.0 There are a few things we could do; * Start using the native memory constructs from CASSANDRA-6694 to avoid serialization/deserialization costs and to minimize the on-heap overhead * Stop invalidating cached rows on writes (update on write instead). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7029) Investigate alternative transport protocols for both client and inter-server communications
[ https://issues.apache.org/jira/browse/CASSANDRA-7029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096814#comment-14096814 ] Robert Stupp commented on CASSANDRA-7029: - mTCP seems to be a nice thing - although not directly related to C* but to Java Runtime: http://shader.kaist.edu/mtcp/ https://github.com/eunyoung14/mtcp Give that a try? Means: original OpenJDK vs. OpenJDK with mTCP. Patching mTCP into OpenJDK seems to pretty straightforward ( s/foo/bar/g ). Investigate alternative transport protocols for both client and inter-server communications --- Key: CASSANDRA-7029 URL: https://issues.apache.org/jira/browse/CASSANDRA-7029 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Labels: performance Fix For: 3.0 There are a number of reasons to think we can do better than TCP for our communications: 1) We can actually tolerate sporadic small message losses, so guaranteed delivery isn't essential (although for larger messages it probably is) 2) As shown in \[1\] and \[2\], Linux can behave quite suboptimally with regard to TCP message delivery when the system is under load. Judging from the theoretical description, this is likely to apply even when the system-load is not high, but the number of processes to schedule is high. Cassandra generally has a lot of threads to schedule, so this is quite pertinent for us. UDP performs substantially better here. 3) Even when the system is not under load, UDP has a lower CPU burden, and that burden is constant regardless of the number of connections it processes. 4) On a simple benchmark on my local PC, using non-blocking IO for UDP and busy spinning on IO I can actually push 20-40% more throughput through loopback (where TCP should be optimal, as no latency), even for very small messages. Since we can see networking taking multiple CPUs' worth of time during a stress test, using a busy-spin for ~100micros after last message receipt is almost certainly acceptable, especially as we can (ultimately) process inter-server and client communications on the same thread/socket in this model. 5) We can optimise the threading model heavily: since we generally process very small messages (200 bytes not at all implausible), the thread signalling costs on the processing thread can actually dramatically impede throughput. In general it costs ~10micros to signal (and passing the message to another thread for processing in the current model requires signalling). For 200-byte messages this caps our throughput at 20MB/s. I propose to knock up a highly naive UDP-based connection protocol with super-trivial congestion control over the course of a few days, with the only initial goal being maximum possible performance (not fairness, reliability, or anything else), and trial it in Netty (possibly making some changes to Netty to mitigate thread signalling costs). The reason for knocking up our own here is to get a ceiling on what the absolute limit of potential for this approach is. Assuming this pans out with performance gains in C* proper, we then look to contributing to/forking the udt-java project and see how easy it is to bring performance in line with what we can get with our naive approach (I don't suggest starting here, as the project is using blocking old-IO, and modifying it with latency in mind may be challenging, and we won't know for sure what the best case scenario is). \[1\] http://test-docdb.fnal.gov/0016/001648/002/Potential%20Performance%20Bottleneck%20in%20Linux%20TCP.PDF \[2\] http://cd-docdb.fnal.gov/cgi-bin/RetrieveFile?docid=1968;filename=Performance%20Analysis%20of%20Linux%20Networking%20-%20Packet%20Receiving%20(Official).pdf;version=2 Further related reading: http://public.dhe.ibm.com/software/commerce/doc/mft/cdunix/41/UDTWhitepaper.pdf https://mospace.umsystem.edu/xmlui/bitstream/handle/10355/14482/ChoiUndPerTcp.pdf?sequence=1 https://access.redhat.com/site/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.153.3762rep=rep1type=pdf -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-5019) Still too much object allocation on reads
[ https://issues.apache.org/jira/browse/CASSANDRA-5019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict resolved CASSANDRA-5019. - Resolution: Duplicate Since the read path will be rewritten as part of efforts to introduce CASSANDRA-7447 (both regards internal APIs, and the implementation details for the new format), this ticket should be addressed by doing things right here. This may mean the legacy format continues to be somewhat inefficient, but this may or may not eventually be retired entirely, so there is probably not much point spending a lot of time optimising it, esp. when the impact is unknown and probably not dramatic in relation to the other costs associated with this format. Still too much object allocation on reads - Key: CASSANDRA-5019 URL: https://issues.apache.org/jira/browse/CASSANDRA-5019 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Labels: performance Fix For: 3.0 ArrayBackedSortedColumns was a step in the right direction but it's still relatively heavyweight thanks to allocating individual Columns. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7029) Investigate alternative transport protocols for both client and inter-server communications
[ https://issues.apache.org/jira/browse/CASSANDRA-7029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096817#comment-14096817 ] Benedict commented on CASSANDRA-7029: - mTCP is not stable enough, nor universal enough, to be useful to us. It requires very specific linux kernel versions, and very specific network interfaces, in order to work. If it matures it will be worth revisiting. Investigate alternative transport protocols for both client and inter-server communications --- Key: CASSANDRA-7029 URL: https://issues.apache.org/jira/browse/CASSANDRA-7029 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Labels: performance Fix For: 3.0 There are a number of reasons to think we can do better than TCP for our communications: 1) We can actually tolerate sporadic small message losses, so guaranteed delivery isn't essential (although for larger messages it probably is) 2) As shown in \[1\] and \[2\], Linux can behave quite suboptimally with regard to TCP message delivery when the system is under load. Judging from the theoretical description, this is likely to apply even when the system-load is not high, but the number of processes to schedule is high. Cassandra generally has a lot of threads to schedule, so this is quite pertinent for us. UDP performs substantially better here. 3) Even when the system is not under load, UDP has a lower CPU burden, and that burden is constant regardless of the number of connections it processes. 4) On a simple benchmark on my local PC, using non-blocking IO for UDP and busy spinning on IO I can actually push 20-40% more throughput through loopback (where TCP should be optimal, as no latency), even for very small messages. Since we can see networking taking multiple CPUs' worth of time during a stress test, using a busy-spin for ~100micros after last message receipt is almost certainly acceptable, especially as we can (ultimately) process inter-server and client communications on the same thread/socket in this model. 5) We can optimise the threading model heavily: since we generally process very small messages (200 bytes not at all implausible), the thread signalling costs on the processing thread can actually dramatically impede throughput. In general it costs ~10micros to signal (and passing the message to another thread for processing in the current model requires signalling). For 200-byte messages this caps our throughput at 20MB/s. I propose to knock up a highly naive UDP-based connection protocol with super-trivial congestion control over the course of a few days, with the only initial goal being maximum possible performance (not fairness, reliability, or anything else), and trial it in Netty (possibly making some changes to Netty to mitigate thread signalling costs). The reason for knocking up our own here is to get a ceiling on what the absolute limit of potential for this approach is. Assuming this pans out with performance gains in C* proper, we then look to contributing to/forking the udt-java project and see how easy it is to bring performance in line with what we can get with our naive approach (I don't suggest starting here, as the project is using blocking old-IO, and modifying it with latency in mind may be challenging, and we won't know for sure what the best case scenario is). \[1\] http://test-docdb.fnal.gov/0016/001648/002/Potential%20Performance%20Bottleneck%20in%20Linux%20TCP.PDF \[2\] http://cd-docdb.fnal.gov/cgi-bin/RetrieveFile?docid=1968;filename=Performance%20Analysis%20of%20Linux%20Networking%20-%20Packet%20Receiving%20(Official).pdf;version=2 Further related reading: http://public.dhe.ibm.com/software/commerce/doc/mft/cdunix/41/UDTWhitepaper.pdf https://mospace.umsystem.edu/xmlui/bitstream/handle/10355/14482/ChoiUndPerTcp.pdf?sequence=1 https://access.redhat.com/site/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.153.3762rep=rep1type=pdf -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7029) Investigate alternative transport protocols for both client and inter-server communications
[ https://issues.apache.org/jira/browse/CASSANDRA-7029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096823#comment-14096823 ] Robert Stupp commented on CASSANDRA-7029: - Damn - didn't see that mTCP has so strict requirements/constraints. Investigate alternative transport protocols for both client and inter-server communications --- Key: CASSANDRA-7029 URL: https://issues.apache.org/jira/browse/CASSANDRA-7029 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Labels: performance Fix For: 3.0 There are a number of reasons to think we can do better than TCP for our communications: 1) We can actually tolerate sporadic small message losses, so guaranteed delivery isn't essential (although for larger messages it probably is) 2) As shown in \[1\] and \[2\], Linux can behave quite suboptimally with regard to TCP message delivery when the system is under load. Judging from the theoretical description, this is likely to apply even when the system-load is not high, but the number of processes to schedule is high. Cassandra generally has a lot of threads to schedule, so this is quite pertinent for us. UDP performs substantially better here. 3) Even when the system is not under load, UDP has a lower CPU burden, and that burden is constant regardless of the number of connections it processes. 4) On a simple benchmark on my local PC, using non-blocking IO for UDP and busy spinning on IO I can actually push 20-40% more throughput through loopback (where TCP should be optimal, as no latency), even for very small messages. Since we can see networking taking multiple CPUs' worth of time during a stress test, using a busy-spin for ~100micros after last message receipt is almost certainly acceptable, especially as we can (ultimately) process inter-server and client communications on the same thread/socket in this model. 5) We can optimise the threading model heavily: since we generally process very small messages (200 bytes not at all implausible), the thread signalling costs on the processing thread can actually dramatically impede throughput. In general it costs ~10micros to signal (and passing the message to another thread for processing in the current model requires signalling). For 200-byte messages this caps our throughput at 20MB/s. I propose to knock up a highly naive UDP-based connection protocol with super-trivial congestion control over the course of a few days, with the only initial goal being maximum possible performance (not fairness, reliability, or anything else), and trial it in Netty (possibly making some changes to Netty to mitigate thread signalling costs). The reason for knocking up our own here is to get a ceiling on what the absolute limit of potential for this approach is. Assuming this pans out with performance gains in C* proper, we then look to contributing to/forking the udt-java project and see how easy it is to bring performance in line with what we can get with our naive approach (I don't suggest starting here, as the project is using blocking old-IO, and modifying it with latency in mind may be challenging, and we won't know for sure what the best case scenario is). \[1\] http://test-docdb.fnal.gov/0016/001648/002/Potential%20Performance%20Bottleneck%20in%20Linux%20TCP.PDF \[2\] http://cd-docdb.fnal.gov/cgi-bin/RetrieveFile?docid=1968;filename=Performance%20Analysis%20of%20Linux%20Networking%20-%20Packet%20Receiving%20(Official).pdf;version=2 Further related reading: http://public.dhe.ibm.com/software/commerce/doc/mft/cdunix/41/UDTWhitepaper.pdf https://mospace.umsystem.edu/xmlui/bitstream/handle/10355/14482/ChoiUndPerTcp.pdf?sequence=1 https://access.redhat.com/site/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.153.3762rep=rep1type=pdf -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7710) nodetool ring throws exception if run on machine without Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-7710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096824#comment-14096824 ] Jimmy Mårdell commented on CASSANDRA-7710: -- Thanks, yes, that's the one I was looking for! nodetool ring throws exception if run on machine without Cassandra -- Key: CASSANDRA-7710 URL: https://issues.apache.org/jira/browse/CASSANDRA-7710 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Jimmy Mårdell Assignee: Michael Shuler Priority: Minor DatabaseDescriptor.getNumTokens() is invoked in the nodetool ring command which doesn't work so well when running on a machine where Cassandra doesnt' exist. And it has all kind of side effects as well. This seems fixed in 2.1 but would be nice if it was fixed in 2.0 as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7763) cql_tests static_with_empty_clustering test failure
[
https://issues.apache.org/jira/browse/CASSANDRA-7763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7763:
Attachment: 7763.txt
That's not related to CASSANDRA-7455, it's just that the test was introduced
there and exposed the bug. This is actually a regression from CASSANDRA-6934
which mistakenly removed the static case from {{AbstractCType.compare}}. This
was actually indicated in the comments from CASSANDRA-6934 so this is my bad
for not catching it. We do need to special case static since a static cell
(which has empty components for the clustering columns) should still sort
before a non static cell with empty components, which is exactly what this test
is testing.
cql_tests static_with_empty_clustering test failure
---
Key: CASSANDRA-7763
URL: https://issues.apache.org/jira/browse/CASSANDRA-7763
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Sylvain Lebresne
Fix For: 2.1 rc6
Attachments: 7763.txt
{code}
==
FAIL: static_with_empty_clustering_test (cql_tests.TestCQL)
--
Traceback (most recent call last):
File /home/ryan/git/datastax/cassandra-dtest/tools.py, line 213, in
wrapped
f(obj)
File /home/ryan/git/datastax/cassandra-dtest/cql_tests.py, line 4082, in
static_with_empty_clustering_test
assert_one(cursor, SELECT * FROM test, ['partition1', '', 'static
value', 'value'])
File /home/ryan/git/datastax/cassandra-dtest/assertions.py, line 40, in
assert_one
assert res == [expected], res
AssertionError: [[u'partition1', u'', None, None], [u'partition1', u'', None,
None], [u'partition1', u'', None, u'value']]
begin captured logging
dtest: DEBUG: cluster ccm directory: /tmp/dtest-Ex54V7
- end captured logging -
--
Ran 1 test in 6.866s
FAILED (failures=1)
{code}
regression from CASSANDRA-7455?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (CASSANDRA-7188) Wrong class type: class org.apache.cassandra.db.Column in CounterColumn.reconcile
[
https://issues.apache.org/jira/browse/CASSANDRA-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reassigned CASSANDRA-7188:
-
Assignee: Michael Shuler (was: Albert P Tobey)
Wrong class type: class org.apache.cassandra.db.Column in
CounterColumn.reconcile
-
Key: CASSANDRA-7188
URL: https://issues.apache.org/jira/browse/CASSANDRA-7188
Project: Cassandra
Issue Type: Bug
Reporter: Nicolas Lalevée
Assignee: Michael Shuler
When migrating a cluster of 6 nodes from 1.2.11 to 2.0.7, we started to see
on the first migrated node this error:
{noformat}
ERROR [ReplicateOnWriteStage:1] 2014-05-07 11:26:59,779 CassandraDaemon.java
(line 198) Exception in thread Thread[ReplicateOnWriteStage:1,5,main]
java.lang.AssertionError: Wrong class type: class
org.apache.cassandra.db.Column
at
org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:159)
at
org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:109)
at
org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:103)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:112)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.filter.NamesQueryFilter.collectReducedColumns(NamesQueryFilter.java:98)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1540)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1369)
at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
at
org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:55)
at
org.apache.cassandra.db.CounterMutation.makeReplicationMutation(CounterMutation.java:100)
at
org.apache.cassandra.service.StorageProxy$8$1.runMayThrow(StorageProxy.java:1085)
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1916)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
We then saw on the other 5 nodes, still on 1.2.x, this error:
{noformat}
ERROR [MutationStage:2793] 2014-05-07 11:46:12,301 CassandraDaemon.java (line
191) Exception in thread Thread[MutationStage:2793,5,main]
java.lang.AssertionError: Wrong class type: class
org.apache.cassandra.db.Column
at
org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:165)
at
org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:378)
at
org.apache.cassandra.db.AtomicSortedColumns.addColumn(AtomicSortedColumns.java:166)
at
org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:119)
at org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:218)
at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:229)
at
org.apache.cassandra.db.ThreadSafeSortedColumns.addColumnInternal(ThreadSafeSortedColumns.java:108)
at
org.apache.cassandra.db.ThreadSafeSortedColumns.addAllWithSizeDelta(ThreadSafeSortedColumns.java:138)
at
org.apache.cassandra.db.AbstractColumnContainer.addAllWithSizeDelta(AbstractColumnContainer.java:99)
at org.apache.cassandra.db.Memtable.resolve(Memtable.java:205)
at org.apache.cassandra.db.Memtable.put(Memtable.java:168)
at
org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:742)
at org.apache.cassandra.db.Table.apply(Table.java:388)
at org.apache.cassandra.db.Table.apply(Table.java:353)
at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:280)
at
org.apache.cassandra.db.CounterMutation.apply(CounterMutation.java:137)
at
git commit: Skip strict endpoint selection for ranges if RF == nodes
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1.0 ed3694b86 - e4e56c2fd
Skip strict endpoint selection for ranges if RF == nodes
Patch by Sam Tunnicliffe; reviewed by tjake for CASSANDRA-7765
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4e56c2f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4e56c2f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4e56c2f
Branch: refs/heads/cassandra-2.1.0
Commit: e4e56c2fd08037b4ddc7459f341e2c86c272e803
Parents: ed3694b
Author: beobal s...@beobal.com
Authored: Wed Aug 13 21:10:52 2014 +0100
Committer: Jake Luciani j...@apache.org
Committed: Thu Aug 14 09:51:45 2014 -0400
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2913607..058681a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.0-rc6
+ * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765)
* Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741)
* Add tracing entries about concurrent range requests (CASSANDRA-7599)
* (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/src/java/org/apache/cassandra/dht/RangeStreamer.java
--
diff --git a/src/java/org/apache/cassandra/dht/RangeStreamer.java
b/src/java/org/apache/cassandra/dht/RangeStreamer.java
index c5d45f9..d84a951 100644
--- a/src/java/org/apache/cassandra/dht/RangeStreamer.java
+++ b/src/java/org/apache/cassandra/dht/RangeStreamer.java
@@ -128,7 +128,7 @@ public class RangeStreamer
public void addRanges(String keyspaceName, CollectionRangeToken ranges)
{
-MultimapRangeToken, InetAddress rangesForKeyspace =
!DatabaseDescriptor.isReplacing() useStrictConsistency tokens != null
+MultimapRangeToken, InetAddress rangesForKeyspace =
useStrictSourcesForRanges(keyspaceName)
? getAllRangesWithStrictSourcesFor(keyspaceName, ranges) :
getAllRangesWithSourcesFor(keyspaceName, ranges);
if (logger.isDebugEnabled())
@@ -148,6 +148,15 @@ public class RangeStreamer
}
}
+private boolean useStrictSourcesForRanges(String keyspaceName)
+{
+AbstractReplicationStrategy strat =
Keyspace.open(keyspaceName).getReplicationStrategy();
+return !DatabaseDescriptor.isReplacing()
+ useStrictConsistency
+ tokens != null
+ metadata.getAllEndpoints().size() !=
strat.getReplicationFactor();
+}
+
/**
* Get a map of all ranges and their respective sources that are
candidates for streaming the given ranges
* to us. For each range, the list of sources is sorted by proximity
relative to the given destAddress.
[2/3] git commit: Skip strict endpoint selection for ranges if RF == nodes
Skip strict endpoint selection for ranges if RF == nodes
Patch by Sam Tunnicliffe; reviewed by tjake for CASSANDRA-7765
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4e56c2f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4e56c2f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4e56c2f
Branch: refs/heads/cassandra-2.1
Commit: e4e56c2fd08037b4ddc7459f341e2c86c272e803
Parents: ed3694b
Author: beobal s...@beobal.com
Authored: Wed Aug 13 21:10:52 2014 +0100
Committer: Jake Luciani j...@apache.org
Committed: Thu Aug 14 09:51:45 2014 -0400
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2913607..058681a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.0-rc6
+ * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765)
* Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741)
* Add tracing entries about concurrent range requests (CASSANDRA-7599)
* (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/src/java/org/apache/cassandra/dht/RangeStreamer.java
--
diff --git a/src/java/org/apache/cassandra/dht/RangeStreamer.java
b/src/java/org/apache/cassandra/dht/RangeStreamer.java
index c5d45f9..d84a951 100644
--- a/src/java/org/apache/cassandra/dht/RangeStreamer.java
+++ b/src/java/org/apache/cassandra/dht/RangeStreamer.java
@@ -128,7 +128,7 @@ public class RangeStreamer
public void addRanges(String keyspaceName, CollectionRangeToken ranges)
{
-MultimapRangeToken, InetAddress rangesForKeyspace =
!DatabaseDescriptor.isReplacing() useStrictConsistency tokens != null
+MultimapRangeToken, InetAddress rangesForKeyspace =
useStrictSourcesForRanges(keyspaceName)
? getAllRangesWithStrictSourcesFor(keyspaceName, ranges) :
getAllRangesWithSourcesFor(keyspaceName, ranges);
if (logger.isDebugEnabled())
@@ -148,6 +148,15 @@ public class RangeStreamer
}
}
+private boolean useStrictSourcesForRanges(String keyspaceName)
+{
+AbstractReplicationStrategy strat =
Keyspace.open(keyspaceName).getReplicationStrategy();
+return !DatabaseDescriptor.isReplacing()
+ useStrictConsistency
+ tokens != null
+ metadata.getAllEndpoints().size() !=
strat.getReplicationFactor();
+}
+
/**
* Get a map of all ranges and their respective sources that are
candidates for streaming the given ranges
* to us. For each range, the list of sources is sorted by proximity
relative to the given destAddress.
[1/3] git commit: export MALLOC_ARENA_MAX corrrectly
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 74e7f28f7 - 1040b520d export MALLOC_ARENA_MAX corrrectly Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-6126 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed3694b8 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed3694b8 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed3694b8 Branch: refs/heads/cassandra-2.1 Commit: ed3694b868cdfbf674445f878477d9f7589ba86e Parents: 0623ea8 Author: Brandon Williams brandonwilli...@apache.org Authored: Wed Aug 13 13:28:28 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Wed Aug 13 13:29:42 2014 -0500 -- conf/cassandra-env.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed3694b8/conf/cassandra-env.sh -- diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh index 211b81b..43a121c 100644 --- a/conf/cassandra-env.sh +++ b/conf/cassandra-env.sh @@ -144,7 +144,7 @@ esac #HEAP_NEWSIZE=800M # Set this to control the amount of arenas per-thread in glibc -#MALLOC_ARENA_MAX=4 +#export MALLOC_ARENA_MAX=4 if [ x$MAX_HEAP_SIZE = x ] [ x$HEAP_NEWSIZE = x ]; then calculate_heap_sizes @@ -157,7 +157,7 @@ fi if [ x$MALLOC_ARENA_MAX = x ] then -MALLOC_ARENA_MAX=4 +export MALLOC_ARENA_MAX=4 fi # Specifies the default port over which Cassandra will be available for
[3/3] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1
Merge branch 'cassandra-2.1.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1040b520 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1040b520 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1040b520 Branch: refs/heads/cassandra-2.1 Commit: 1040b520d116abb089acfa24d30aba8556f91f0e Parents: 74e7f28 e4e56c2 Author: Jake Luciani j...@apache.org Authored: Thu Aug 14 09:53:38 2014 -0400 Committer: Jake Luciani j...@apache.org Committed: Thu Aug 14 09:53:38 2014 -0400 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++- 2 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/1040b520/CHANGES.txt -- diff --cc CHANGES.txt index f34cec7,058681a..cf2f34a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,27 -1,5 +1,28 @@@ +2.1.1 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569) + * SSTableExport uses correct validator to create string representation of partition + keys (CASSANDRA-7498) + * Avoid NPEs when receiving type changes for an unknown keyspace (CASSANDRA-7689) + * Add support for custom 2i validation (CASSANDRA-7575) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) +Merged from 2.0: + * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Catch errors when the JVM pulls the rug out from GCInspector (CASSANDRA-5345) + * cqlsh fails when version number parts are not int (CASSANDRA-7524) + + 2.1.0-rc6 + * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765) * Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741) * Add tracing entries about concurrent range requests (CASSANDRA-7599) * (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729)
[1/4] git commit: export MALLOC_ARENA_MAX corrrectly
Repository: cassandra Updated Branches: refs/heads/trunk 048f72bd3 - 4304b3a6d export MALLOC_ARENA_MAX corrrectly Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-6126 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed3694b8 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed3694b8 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed3694b8 Branch: refs/heads/trunk Commit: ed3694b868cdfbf674445f878477d9f7589ba86e Parents: 0623ea8 Author: Brandon Williams brandonwilli...@apache.org Authored: Wed Aug 13 13:28:28 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Wed Aug 13 13:29:42 2014 -0500 -- conf/cassandra-env.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed3694b8/conf/cassandra-env.sh -- diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh index 211b81b..43a121c 100644 --- a/conf/cassandra-env.sh +++ b/conf/cassandra-env.sh @@ -144,7 +144,7 @@ esac #HEAP_NEWSIZE=800M # Set this to control the amount of arenas per-thread in glibc -#MALLOC_ARENA_MAX=4 +#export MALLOC_ARENA_MAX=4 if [ x$MAX_HEAP_SIZE = x ] [ x$HEAP_NEWSIZE = x ]; then calculate_heap_sizes @@ -157,7 +157,7 @@ fi if [ x$MALLOC_ARENA_MAX = x ] then -MALLOC_ARENA_MAX=4 +export MALLOC_ARENA_MAX=4 fi # Specifies the default port over which Cassandra will be available for
[4/4] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4304b3a6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4304b3a6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4304b3a6 Branch: refs/heads/trunk Commit: 4304b3a6df8e10e7abd998cd79abe330607d38c2 Parents: 048f72b 1040b52 Author: Jake Luciani j...@apache.org Authored: Thu Aug 14 09:54:22 2014 -0400 Committer: Jake Luciani j...@apache.org Committed: Thu Aug 14 09:54:22 2014 -0400 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++- 2 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4304b3a6/CHANGES.txt --
[3/4] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1
Merge branch 'cassandra-2.1.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1040b520 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1040b520 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1040b520 Branch: refs/heads/trunk Commit: 1040b520d116abb089acfa24d30aba8556f91f0e Parents: 74e7f28 e4e56c2 Author: Jake Luciani j...@apache.org Authored: Thu Aug 14 09:53:38 2014 -0400 Committer: Jake Luciani j...@apache.org Committed: Thu Aug 14 09:53:38 2014 -0400 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++- 2 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/1040b520/CHANGES.txt -- diff --cc CHANGES.txt index f34cec7,058681a..cf2f34a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,27 -1,5 +1,28 @@@ +2.1.1 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569) + * SSTableExport uses correct validator to create string representation of partition + keys (CASSANDRA-7498) + * Avoid NPEs when receiving type changes for an unknown keyspace (CASSANDRA-7689) + * Add support for custom 2i validation (CASSANDRA-7575) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) +Merged from 2.0: + * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Catch errors when the JVM pulls the rug out from GCInspector (CASSANDRA-5345) + * cqlsh fails when version number parts are not int (CASSANDRA-7524) + + 2.1.0-rc6 + * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765) * Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741) * Add tracing entries about concurrent range requests (CASSANDRA-7599) * (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729)
[2/4] git commit: Skip strict endpoint selection for ranges if RF == nodes
Skip strict endpoint selection for ranges if RF == nodes
Patch by Sam Tunnicliffe; reviewed by tjake for CASSANDRA-7765
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4e56c2f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4e56c2f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4e56c2f
Branch: refs/heads/trunk
Commit: e4e56c2fd08037b4ddc7459f341e2c86c272e803
Parents: ed3694b
Author: beobal s...@beobal.com
Authored: Wed Aug 13 21:10:52 2014 +0100
Committer: Jake Luciani j...@apache.org
Committed: Thu Aug 14 09:51:45 2014 -0400
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2913607..058681a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.0-rc6
+ * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765)
* Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741)
* Add tracing entries about concurrent range requests (CASSANDRA-7599)
* (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/src/java/org/apache/cassandra/dht/RangeStreamer.java
--
diff --git a/src/java/org/apache/cassandra/dht/RangeStreamer.java
b/src/java/org/apache/cassandra/dht/RangeStreamer.java
index c5d45f9..d84a951 100644
--- a/src/java/org/apache/cassandra/dht/RangeStreamer.java
+++ b/src/java/org/apache/cassandra/dht/RangeStreamer.java
@@ -128,7 +128,7 @@ public class RangeStreamer
public void addRanges(String keyspaceName, CollectionRangeToken ranges)
{
-MultimapRangeToken, InetAddress rangesForKeyspace =
!DatabaseDescriptor.isReplacing() useStrictConsistency tokens != null
+MultimapRangeToken, InetAddress rangesForKeyspace =
useStrictSourcesForRanges(keyspaceName)
? getAllRangesWithStrictSourcesFor(keyspaceName, ranges) :
getAllRangesWithSourcesFor(keyspaceName, ranges);
if (logger.isDebugEnabled())
@@ -148,6 +148,15 @@ public class RangeStreamer
}
}
+private boolean useStrictSourcesForRanges(String keyspaceName)
+{
+AbstractReplicationStrategy strat =
Keyspace.open(keyspaceName).getReplicationStrategy();
+return !DatabaseDescriptor.isReplacing()
+ useStrictConsistency
+ tokens != null
+ metadata.getAllEndpoints().size() !=
strat.getReplicationFactor();
+}
+
/**
* Get a map of all ranges and their respective sources that are
candidates for streaming the given ranges
* to us. For each range, the list of sources is sorted by proximity
relative to the given destAddress.
buildbot success in ASF Buildbot on cassandra-2.1
The Buildbot has detected a restored build on builder cassandra-2.1 while building cassandra. Full details are available at: http://ci.apache.org/builders/cassandra-2.1/builds/393 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: lares_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra-2.1] 1040b520d116abb089acfa24d30aba8556f91f0e Blamelist: Brandon Williams brandonwilli...@apache.org,Jake Luciani j...@apache.org,beobal s...@beobal.com Build succeeded! sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-5091) Provide a way to sort read results on the coordinator node
[ https://issues.apache.org/jira/browse/CASSANDRA-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096988#comment-14096988 ] Sylvain Lebresne commented on CASSANDRA-5091: - I'll note that ordering on coordinator and query paging don't play very well at all together so I'm not sure how much we can do here. Provide a way to sort read results on the coordinator node -- Key: CASSANDRA-5091 URL: https://issues.apache.org/jira/browse/CASSANDRA-5091 Project: Cassandra Issue Type: Wish Reporter: Sergio Bossa Labels: cql While Cassandra provides a way to scatter/gather queries with secondary indexes, as well as custom secondary indexes implementations, there is currently no way to sort gathered results on the coordinator node, which would be very useful (actually, essential) if the index implementation provides results in sorted order. So, what I'm proposing here is a way to provide (custom) sort implementations, which may either be as easy as just adding one more interface and a bunch of method calls, or get a bit more complex by providing an ORDER BY clause in CQL. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7767) Expose sizes of off-heap data structures via JMX and `nodetool info`
J.B. Langston created CASSANDRA-7767: Summary: Expose sizes of off-heap data structures via JMX and `nodetool info` Key: CASSANDRA-7767 URL: https://issues.apache.org/jira/browse/CASSANDRA-7767 Project: Cassandra Issue Type: New Feature Reporter: J.B. Langston It would be very helpful for troubleshooting memory consumption to know the individual sizes of off-heap data structures such as bloom filters, index summaries, compression metadata, etc. Can we expose this over JMX? Also, since `nodetool info` already shows size of heap, key cache, etc. it seems like a natural place to show this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7386) JBOD threshold to prevent unbalanced disk utilization
[ https://issues.apache.org/jira/browse/CASSANDRA-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lyuben Todorov updated CASSANDRA-7386: -- Attachment: sstable-count-second-run.png mean-writevalue-7disks.png After loading 750GB of data into a 7 disk cluster the test shows what we expected, the drives that had data written to them previously do indeed have lower Write values and these values increase as the rest of the drives begin filling up (view graph). I also tracked sstable creation and compaction. To sum it up, on the first run only 2 drives were used, on the second run another five were added. At the end of the first run d1 and d2 (disk1 and disk2 respectively) were saturated with sstables, d1 had 357 and d2 318. When the second run was started another ks, second_run, was created and an additional 5 disks were used in the node. A majority of sstables were sent to the five new directories as expected (view piechart). The second test to see if busy disks are used for new tables is coming up. JBOD threshold to prevent unbalanced disk utilization - Key: CASSANDRA-7386 URL: https://issues.apache.org/jira/browse/CASSANDRA-7386 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Lohfink Assignee: Lyuben Todorov Priority: Minor Fix For: 2.1.1 Attachments: 7386-v1.patch, 7386v2.diff, Mappe1.ods, mean-writevalue-7disks.png, patch_2_1_branch_proto.diff, sstable-count-second-run.png Currently the pick the disks are picked first by number of current tasks, then by free space. This helps with performance but can lead to large differences in utilization in some (unlikely but possible) scenarios. Ive seen 55% to 10% and heard reports of 90% to 10% on IRC. With both LCS and STCS (although my suspicion is that STCS makes it worse since harder to be balanced). I purpose the algorithm change a little to have some maximum range of utilization where it will pick by free space over load (acknowledging it can be slower). So if a disk A is 30% full and disk B is 5% full it will never pick A over B until it balances out. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7386) JBOD threshold to prevent unbalanced disk utilization
[ https://issues.apache.org/jira/browse/CASSANDRA-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097014#comment-14097014 ] Lyuben Todorov edited comment on CASSANDRA-7386 at 8/14/14 2:41 PM: After loading 750GB of data into a 7 disk cluster the test shows what we expected, the drives that had data written to them previously do indeed have lower Write values and these values increase as the rest of the drives begin filling up (view graph). !https://issues.apache.org/jira/secure/attachment/12661718/mean-writevalue-7disks.png! I also tracked sstable creation and compaction. To sum it up, on the first run only 2 drives were used, on the second run another five were added. At the end of the first run d1 and d2 (disk1 and disk2 respectively) were saturated with sstables, d1 had 357 and d2 318. When the second run was started another ks, second_run, was created and an additional 5 disks were used in the node. A majority of sstables were sent to the five new directories as expected (view piechart). !https://issues.apache.org/jira/secure/attachment/12661719/sstable-count-second-run.png! The second test to see if busy disks are used for new tables is coming up. was (Author: lyubent): After loading 750GB of data into a 7 disk cluster the test shows what we expected, the drives that had data written to them previously do indeed have lower Write values and these values increase as the rest of the drives begin filling up (view graph). I also tracked sstable creation and compaction. To sum it up, on the first run only 2 drives were used, on the second run another five were added. At the end of the first run d1 and d2 (disk1 and disk2 respectively) were saturated with sstables, d1 had 357 and d2 318. When the second run was started another ks, second_run, was created and an additional 5 disks were used in the node. A majority of sstables were sent to the five new directories as expected (view piechart). The second test to see if busy disks are used for new tables is coming up. JBOD threshold to prevent unbalanced disk utilization - Key: CASSANDRA-7386 URL: https://issues.apache.org/jira/browse/CASSANDRA-7386 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Lohfink Assignee: Lyuben Todorov Priority: Minor Fix For: 2.1.1 Attachments: 7386-v1.patch, 7386v2.diff, Mappe1.ods, mean-writevalue-7disks.png, patch_2_1_branch_proto.diff, sstable-count-second-run.png Currently the pick the disks are picked first by number of current tasks, then by free space. This helps with performance but can lead to large differences in utilization in some (unlikely but possible) scenarios. Ive seen 55% to 10% and heard reports of 90% to 10% on IRC. With both LCS and STCS (although my suspicion is that STCS makes it worse since harder to be balanced). I purpose the algorithm change a little to have some maximum range of utilization where it will pick by free space over load (acknowledging it can be slower). So if a disk A is 30% full and disk B is 5% full it will never pick A over B until it balances out. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-7304: Labels: cql protocolv4 (was: cql) Ability to distinguish between NULL and UNSET values in Prepared Statements --- Key: CASSANDRA-7304 URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 Project: Cassandra Issue Type: Improvement Reporter: Drew Kutcharian Labels: cql, protocolv4 Attachments: 7304.patch Currently Cassandra inserts tombstones when a value of a column is bound to NULL in a prepared statement. At higher insert rates managing all these tombstones becomes an unnecessary overhead. This limits the usefulness of the prepared statements since developers have to either create multiple prepared statements (each with a different combination of column names, which at times is just unfeasible because of the sheer number of possible combinations) or fall back to using regular (non-prepared) statements. This JIRA is here to explore the possibility of either: A. Have a flag on prepared statements that once set, tells Cassandra to ignore null columns or B. Have an UNSET value which makes Cassandra skip the null columns and not tombstone them Basically, in the context of a prepared statement, a null value means delete, but we don’t have anything that means ignore (besides creating a new prepared statement without the ignored column). Please refer to the original conversation on DataStax Java Driver mailing list for more background: https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7759) DROP TABLE makes C* unreachable
[
https://issues.apache.org/jira/browse/CASSANDRA-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097049#comment-14097049
]
Fabrice Larcher commented on CASSANDRA-7759:
After a {{DROP TABLE my_old_table;}}, I experienced again the issue. Then I was
unable to kill C*2 (still running) using {{kill PID}}. I forced it with
{{kill -9 PID}}. Then, trying to start C* 2.1.0-rc5 again, I get the
following NPE in the console output :
{code}
ERROR 15:02:52 Exception encountered during startup
java.lang.NullPointerException: null
at org.apache.cassandra.db.Directories.init(Directories.java:191)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:553)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:245)
[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:455)
[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:544)
[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
java.lang.NullPointerException
at org.apache.cassandra.db.Directories.init(Directories.java:191)
at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:553)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:245)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:455)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:544)
Exception encountered during startup: null
{code}
I do not know exactly how recover and I guest that it is related with this
issue.
DROP TABLE makes C* unreachable
---
Key: CASSANDRA-7759
URL: https://issues.apache.org/jira/browse/CASSANDRA-7759
Project: Cassandra
Issue Type: Bug
Environment: CQLSH of C* 2.10.-rc5 with one or two node(s).
Reporter: Fabrice Larcher
After a {{DROP KEYSPACE}} or {{DROP TABLE}} command from _CQLSH_ I often get
the output :
{code}
errors={}, last_host=127.0.0.1
{code}
Then, the application can not access the node anymore (it uses the Java
Driver 2.1.0-rc1). And I get the following stacktraces from the
{{system.log}} file ;
{code}
ERROR [MemtableFlushWriter:5] 2014-08-13 11:26:07,577
CassandraDaemon.java:166 - Exception in thread
Thread[MemtableFlushWriter:5,5,main]
java.lang.RuntimeException: Last written key
DecoratedKey(28149890686391545-8456361251720325,
30313034353666392d666464362d343539362d393537372d653539336430333138396437) =
current key DecoratedKey(108b5f3f-fc06-4a0d-99f1-3c6484b32e04,
31303862356633662d666330362d346130642d393966312d336336343834623332653034)
writing into
./../data/data/metrics/run-a3c1fe80216911e49fd1ab9c3338a2ff/metrics-run.run_probe-tmp-ka-2-Data.db
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
~[guava-16.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]
{code}
I also get the following (perhaps not related)
{code}
ERROR [MemtableFlushWriter:6] 2014-08-13 11:26:08,827
CassandraDaemon.java:166 - Exception in thread
Thread[MemtableFlushWriter:6,5,main]
org.apache.cassandra.serializers.MarshalException: Invalid byte for ascii: -50
at
org.apache.cassandra.serializers.AsciiSerializer.validate(AsciiSerializer.java:39)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
[jira] [Created] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
Paul Pak created CASSANDRA-7768: --- Summary: Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace Key: CASSANDRA-7768 URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 Project: Cassandra Issue Type: Bug Components: Core Reporter: Paul Pak Assignee: Paul Pak Reason why this occurs is because Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Priority: Minor (was: Major) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace --- Key: CASSANDRA-7768 URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 Project: Cassandra Issue Type: Bug Reporter: Paul Pak Assignee: Paul Pak Priority: Minor Reason why this occurs is because Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Component/s: (was: Core) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace --- Key: CASSANDRA-7768 URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 Project: Cassandra Issue Type: Bug Reporter: Paul Pak Assignee: Paul Pak Reason why this occurs is because Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Description: The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded), CQLSSTableWriter builder only loades the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. (was: Reason why this occurs is because Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map.) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace --- Key: CASSANDRA-7768 URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 Project: Cassandra Issue Type: Bug Reporter: Paul Pak Assignee: Paul Pak Priority: Minor The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded), CQLSSTableWriter builder only loades the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Description: The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded in the same keyspace), CQLSSTableWriter builder only loads the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. (was: The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded in the same keyspace), CQLSSTableWriter builder only loades the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map.) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace --- Key: CASSANDRA-7768 URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 Project: Cassandra Issue Type: Bug Reporter: Paul Pak Assignee: Paul Pak Priority: Minor The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded in the same keyspace), CQLSSTableWriter builder only loads the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Description: The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded in the same keyspace), CQLSSTableWriter builder only loades the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. (was: The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded), CQLSSTableWriter builder only loades the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map.) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace --- Key: CASSANDRA-7768 URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 Project: Cassandra Issue Type: Bug Reporter: Paul Pak Assignee: Paul Pak Priority: Minor The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded in the same keyspace), CQLSSTableWriter builder only loades the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[2/3] git commit: Fix compactionstats layout for long ks/cf names
Fix compactionstats layout for long ks/cf names
Patch by Nicolas Lalevée, reviewed by brandonwilliams for CASSANDRA-7263
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4c510dc6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4c510dc6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4c510dc6
Branch: refs/heads/trunk
Commit: 4c510dc6cc1ea3fdb21180cfc74ab6c29b78c24f
Parents: 1040b52
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 14 10:51:57 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 14 10:51:57 2014 -0500
--
.../org/apache/cassandra/tools/NodeTool.java| 66 ++--
1 file changed, 48 insertions(+), 18 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4c510dc6/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java
b/src/java/org/apache/cassandra/tools/NodeTool.java
index b25edd9..6aca507 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1129,26 +1129,56 @@ public class NodeTool
int compactionThroughput = probe.getCompactionThroughput();
CompactionManagerMBean cm = probe.getCompactionManagerProxy();
System.out.println(pending tasks: +
probe.getCompactionMetric(PendingTasks));
-if (cm.getCompactions().size() 0)
-System.out.printf(%25s%16s%16s%16s%16s%10s%10s%n,
compaction type, keyspace, table, completed, total, unit,
progress);
long remainingBytes = 0;
-for (MapString, String c : cm.getCompactions())
-{
-String percentComplete = new Long(c.get(total)) == 0
- ? n/a
- : new
DecimalFormat(0.00).format((double) new Long(c.get(completed)) / new
Long(c.get(total)) * 100) + %;
-System.out.printf(%25s%16s%16s%16s%16s%10s%10s%n,
c.get(taskType), c.get(keyspace), c.get(columnfamily),
c.get(completed), c.get(total), c.get(unit), percentComplete);
-if
(c.get(taskType).equals(OperationType.COMPACTION.toString()))
-remainingBytes += (new Long(c.get(total)) - new
Long(c.get(completed)));
+if (cm.getCompactions().size() 0)
+{
+ListString[] lines = new ArrayList();
+int[] columnSizes = new int[] { 0, 0, 0, 0, 0, 0, 0 };
+
+addLine(lines, columnSizes, compaction type, keyspace,
table, completed, total, unit, progress);
+for (MapString, String c : cm.getCompactions())
+{
+long total = Long.parseLong(c.get(total));
+long completed = Long.parseLong(c.get(completed));
+String taskType = c.get(taskType);
+String keyspace = c.get(keyspace);
+String columnFamily = c.get(columnfamily);
+String unit = c.get(unit);
+String percentComplete = total == 0 ? n/a : new
DecimalFormat(0.00).format((double) completed / total * 100) + %;
+addLine(lines, columnSizes, taskType, keyspace,
columnFamily, Long.toString(completed), Long.toString(total), unit,
percentComplete);
+if (taskType.equals(OperationType.COMPACTION.toString()))
+remainingBytes += total - completed;
+}
+
+StringBuilder buffer = new StringBuilder();
+for (int columnSize : columnSizes) {
+buffer.append(%);
+buffer.append(columnSize + 3);
+buffer.append(s);
+}
+buffer.append(%n);
+String format = buffer.toString();
+
+for (String[] line : lines)
+{
+System.out.printf(format, line[0], line[1], line[2],
line[3], line[4], line[5], line[6]);
+}
+
+String remainingTime = n/a;
+if (compactionThroughput != 0)
+{
+long remainingTimeInSecs = remainingBytes / (1024L * 1024L
* compactionThroughput);
+remainingTime = format(%dh%02dm%02ds,
remainingTimeInSecs / 3600, (remainingTimeInSecs % 3600) / 60,
(remainingTimeInSecs % 60));
+}
+System.out.printf(%25s%10s%n, Active compaction remaining
time : , remainingTime);
+}
+}
+
+private void addLine(ListString[] lines, int[] columnSizes,
String... columns) {
+
[3/3] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fd43c33c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fd43c33c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fd43c33c Branch: refs/heads/trunk Commit: fd43c33c66ac493bf7cf1f7b96c7c93346b407c3 Parents: 4304b3a 4c510dc Author: Brandon Williams brandonwilli...@apache.org Authored: Thu Aug 14 10:53:00 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Thu Aug 14 10:53:00 2014 -0500 -- .../org/apache/cassandra/tools/NodeTool.java| 66 ++-- 1 file changed, 48 insertions(+), 18 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fd43c33c/src/java/org/apache/cassandra/tools/NodeTool.java --
[1/3] git commit: Fix compactionstats layout for long ks/cf names
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 1040b520d - 4c510dc6c
refs/heads/trunk 4304b3a6d - fd43c33c6
Fix compactionstats layout for long ks/cf names
Patch by Nicolas Lalevée, reviewed by brandonwilliams for CASSANDRA-7263
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4c510dc6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4c510dc6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4c510dc6
Branch: refs/heads/cassandra-2.1
Commit: 4c510dc6cc1ea3fdb21180cfc74ab6c29b78c24f
Parents: 1040b52
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 14 10:51:57 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 14 10:51:57 2014 -0500
--
.../org/apache/cassandra/tools/NodeTool.java| 66 ++--
1 file changed, 48 insertions(+), 18 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4c510dc6/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java
b/src/java/org/apache/cassandra/tools/NodeTool.java
index b25edd9..6aca507 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1129,26 +1129,56 @@ public class NodeTool
int compactionThroughput = probe.getCompactionThroughput();
CompactionManagerMBean cm = probe.getCompactionManagerProxy();
System.out.println(pending tasks: +
probe.getCompactionMetric(PendingTasks));
-if (cm.getCompactions().size() 0)
-System.out.printf(%25s%16s%16s%16s%16s%10s%10s%n,
compaction type, keyspace, table, completed, total, unit,
progress);
long remainingBytes = 0;
-for (MapString, String c : cm.getCompactions())
-{
-String percentComplete = new Long(c.get(total)) == 0
- ? n/a
- : new
DecimalFormat(0.00).format((double) new Long(c.get(completed)) / new
Long(c.get(total)) * 100) + %;
-System.out.printf(%25s%16s%16s%16s%16s%10s%10s%n,
c.get(taskType), c.get(keyspace), c.get(columnfamily),
c.get(completed), c.get(total), c.get(unit), percentComplete);
-if
(c.get(taskType).equals(OperationType.COMPACTION.toString()))
-remainingBytes += (new Long(c.get(total)) - new
Long(c.get(completed)));
+if (cm.getCompactions().size() 0)
+{
+ListString[] lines = new ArrayList();
+int[] columnSizes = new int[] { 0, 0, 0, 0, 0, 0, 0 };
+
+addLine(lines, columnSizes, compaction type, keyspace,
table, completed, total, unit, progress);
+for (MapString, String c : cm.getCompactions())
+{
+long total = Long.parseLong(c.get(total));
+long completed = Long.parseLong(c.get(completed));
+String taskType = c.get(taskType);
+String keyspace = c.get(keyspace);
+String columnFamily = c.get(columnfamily);
+String unit = c.get(unit);
+String percentComplete = total == 0 ? n/a : new
DecimalFormat(0.00).format((double) completed / total * 100) + %;
+addLine(lines, columnSizes, taskType, keyspace,
columnFamily, Long.toString(completed), Long.toString(total), unit,
percentComplete);
+if (taskType.equals(OperationType.COMPACTION.toString()))
+remainingBytes += total - completed;
+}
+
+StringBuilder buffer = new StringBuilder();
+for (int columnSize : columnSizes) {
+buffer.append(%);
+buffer.append(columnSize + 3);
+buffer.append(s);
+}
+buffer.append(%n);
+String format = buffer.toString();
+
+for (String[] line : lines)
+{
+System.out.printf(format, line[0], line[1], line[2],
line[3], line[4], line[5], line[6]);
+}
+
+String remainingTime = n/a;
+if (compactionThroughput != 0)
+{
+long remainingTimeInSecs = remainingBytes / (1024L * 1024L
* compactionThroughput);
+remainingTime = format(%dh%02dm%02ds,
remainingTimeInSecs / 3600, (remainingTimeInSecs % 3600) / 60,
(remainingTimeInSecs % 60));
+}
+System.out.printf(%25s%10s%n, Active compaction remaining
time : ,
buildbot failure in ASF Buildbot on cassandra-2.1
The Buildbot has detected a new failure on builder cassandra-2.1 while building cassandra. Full details are available at: http://ci.apache.org/builders/cassandra-2.1/builds/394 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: lares_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra-2.1] 4c510dc6cc1ea3fdb21180cfc74ab6c29b78c24f Blamelist: Brandon Williams brandonwilli...@apache.org BUILD FAILED: failed sincerely, -The Buildbot
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Attachment: trunk-7768-v1.txt Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace --- Key: CASSANDRA-7768 URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 Project: Cassandra Issue Type: Bug Reporter: Paul Pak Assignee: Paul Pak Priority: Minor Attachments: trunk-7768-v1.txt The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded in the same keyspace), CQLSSTableWriter builder only loads the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build on builder cassandra-trunk while building cassandra. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/650 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: lares_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] fd43c33c66ac493bf7cf1f7b96c7c93346b407c3 Blamelist: Brandon Williams brandonwilli...@apache.org Build succeeded! sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-7585) cassandra sstableloader connection refused with inter_node_encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097146#comment-14097146 ] Yuki Morishita commented on CASSANDRA-7585: --- bq. Could we make it possible to state the server_encryption_options on the command line as well as via the config? The command will look horrible... That's why I added option to load yaml file. I could add 8 more command line options for server encryption but it seems redundant. I'd rather deprecate the current way of setting client encryption options. I think I can highlight this more in help string. Will update with other nits fixed. cassandra sstableloader connection refused with inter_node_encryption - Key: CASSANDRA-7585 URL: https://issues.apache.org/jira/browse/CASSANDRA-7585 Project: Cassandra Issue Type: Bug Components: Core, Tools Reporter: Samphel Norden Assignee: Yuki Morishita Fix For: 2.0.10, 2.1.1 Attachments: 7585-2.0.txt cassandra sstableloader connection refused with inter_node_encryption When using sstableloader to import tables (cassandra 2.0.5) with inter-node encryption and client encryption enabled, I get a connection refused error I am using sstableloader -d $myhost -p 9160 -u cassandra -pw cassandra -ciphers TLS_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA -st JKS -tf org.apache.cassandra.thrift.SSLTransportFactory -ts /path/to/truststore -tspw passwd $fullpath/$table Errors out with Streaming session ID: 1bc395c0-fbb2-11e3-9812-73da15121373 WARN 17:13:34,147 Failed attempt 1 to connect to Similar problem reported in cassandra 2.0.8 by another user http://stackoverflow.com/questions/24390604/cassandra-sstableloader-connection-refused-with-inter-node-encryption == Relevant cassandra.yaml snippet (with obfuscation) server_encryption_options: internode_encryption: all keystore:/path/to/keystore keystore_password: passwd truststore:/path/to/truststore truststore_password:passwd # More advanced defaults below: protocol: TLS algorithm: SunX509 store_type: JKS cipher_suites: [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA] require_client_auth: true # enable or disable client/server encryption. client_encryption_options: enabled: true keystore: /path/to/keystore keystore_password: truststorepasswd #require_client_auth: true # Set trustore and truststore_password if require_client_auth is true truststore:/path/to/truststore truststore_password: truststorepasswd # More advanced defaults below: protocol: TLS
[jira] [Commented] (CASSANDRA-7286) Exception: NPE
[ https://issues.apache.org/jira/browse/CASSANDRA-7286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097155#comment-14097155 ] Julien Anguenot commented on CASSANDRA-7286: [~mshuler] I actually have a trace with that NPE from one of our staging Cassandra instances. Possible to sent it to you offline? Exception: NPE --- Key: CASSANDRA-7286 URL: https://issues.apache.org/jira/browse/CASSANDRA-7286 Project: Cassandra Issue Type: Bug Reporter: Julien Anguenot Attachments: npe_cassandra_2_0_8.txt, readstage_npe.txt Sometimes Cassandra nodes (in a multi datacenter deployment) are throwing NPE (see attached stack trace) Let me know what additional information I could provide. Thank you. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7767) Expose sizes of off-heap data structures via JMX and `nodetool info`
[ https://issues.apache.org/jira/browse/CASSANDRA-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097166#comment-14097166 ] Jeremiah Jordan commented on CASSANDRA-7767: A total size could go in info. The individual sizes should go in cfstats since they are per cf, bloom filter size is already in there. Expose sizes of off-heap data structures via JMX and `nodetool info` Key: CASSANDRA-7767 URL: https://issues.apache.org/jira/browse/CASSANDRA-7767 Project: Cassandra Issue Type: New Feature Reporter: J.B. Langston It would be very helpful for troubleshooting memory consumption to know the individual sizes of off-heap data structures such as bloom filters, index summaries, compression metadata, etc. Can we expose this over JMX? Also, since `nodetool info` already shows size of heap, key cache, etc. it seems like a natural place to show this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7748) Get Windows command-line flags in-line with linux
[ https://issues.apache.org/jira/browse/CASSANDRA-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-7748: --- Attachment: 7748_v1.txt I'm not in love with -s being 'show' for verbose command-line args but PowerShell isn't case-sensitive so I can't re-use upper-case V or a long version. Patch attached that gets -v in line w/linux and adds -s for show env. Get Windows command-line flags in-line with linux - Key: CASSANDRA-7748 URL: https://issues.apache.org/jira/browse/CASSANDRA-7748 Project: Cassandra Issue Type: Improvement Reporter: Joshua McKenzie Assignee: Joshua McKenzie Priority: Trivial Labels: Windows Fix For: 2.1.1, 3.0 Attachments: 7748_v1.txt linux was here first. In the PowerShell launch scripts, -v is verbose right now as I missed that it's used for 'version' on linux. Add version print functionality to Windows launch using -v and find another sane flag to use for verbose env output printing. -- This message was sent by Atlassian JIRA (v6.2#6252)
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure on builder cassandra-trunk while building libcloud. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/651 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: lares_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] 988b1392a7aa061b220318f2fd85f727db2e9f61 Blamelist: Tomaz Muraus to...@apache.org BUILD FAILED: failed sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-7759) DROP TABLE makes C* unreachable
[
https://issues.apache.org/jira/browse/CASSANDRA-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097267#comment-14097267
]
T Jake Luciani commented on CASSANDRA-7759:
---
The first error should be fixed by CASSANDRA-7735 which will go into rc6. Once
released please retest once that's out and get back to us
DROP TABLE makes C* unreachable
---
Key: CASSANDRA-7759
URL: https://issues.apache.org/jira/browse/CASSANDRA-7759
Project: Cassandra
Issue Type: Bug
Environment: CQLSH of C* 2.10.-rc5 with one or two node(s).
Reporter: Fabrice Larcher
After a {{DROP KEYSPACE}} or {{DROP TABLE}} command from _CQLSH_ I often get
the output :
{code}
errors={}, last_host=127.0.0.1
{code}
Then, the application can not access the node anymore (it uses the Java
Driver 2.1.0-rc1). And I get the following stacktraces from the
{{system.log}} file ;
{code}
ERROR [MemtableFlushWriter:5] 2014-08-13 11:26:07,577
CassandraDaemon.java:166 - Exception in thread
Thread[MemtableFlushWriter:5,5,main]
java.lang.RuntimeException: Last written key
DecoratedKey(28149890686391545-8456361251720325,
30313034353666392d666464362d343539362d393537372d653539336430333138396437) =
current key DecoratedKey(108b5f3f-fc06-4a0d-99f1-3c6484b32e04,
31303862356633662d666330362d346130642d393966312d336336343834623332653034)
writing into
./../data/data/metrics/run-a3c1fe80216911e49fd1ab9c3338a2ff/metrics-run.run_probe-tmp-ka-2-Data.db
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
~[guava-16.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]
{code}
I also get the following (perhaps not related)
{code}
ERROR [MemtableFlushWriter:6] 2014-08-13 11:26:08,827
CassandraDaemon.java:166 - Exception in thread
Thread[MemtableFlushWriter:6,5,main]
org.apache.cassandra.serializers.MarshalException: Invalid byte for ascii: -50
at
org.apache.cassandra.serializers.AsciiSerializer.validate(AsciiSerializer.java:39)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:78)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at org.apache.cassandra.dht.LocalToken.toString(LocalToken.java:39)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
at org.apache.cassandra.db.DecoratedKey.toString(DecoratedKey.java:118)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
[jira] [Resolved] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[
https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp resolved CASSANDRA-7577.
-
Resolution: Not a Problem
According to http://pypi.python.org/pypi/readline:
Mac OS X, do not ship with GNU readline installed. The readline extension
module in the standard library of Mac system Python uses NetBSD's editline
(libedit) library instead, which is a readline replacement with a less
restrictive software license.
So, you can install it with the command:
{noformat}
sudo easy_install readline
{noformat}
(Found on
https://stackoverflow.com/questions/8884001/python-2-7-with-mac-osx-lion-fn-delete-and-control-r)
cqlsh: CTRL-R history search not working on OSX
---
Key: CASSANDRA-7577
URL: https://issues.apache.org/jira/browse/CASSANDRA-7577
Project: Cassandra
Issue Type: Bug
Environment: OSX - plain Terminal program
C* 2.0.x, 2.1, trunk
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Minor
_recursive-history-search_ via ctrl-R does not work in cqlsh. The history
itself works via cursor up/down.
It works on Linux (and I guess on Windows with cygwin) but not on my Mac.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7286) Exception: NPE
[ https://issues.apache.org/jira/browse/CASSANDRA-7286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097395#comment-14097395 ] Mikhail Stepura commented on CASSANDRA-7286: Looks suspiciously similar to CASSANDRA-7756 or CASSANDRA-7278 Exception: NPE --- Key: CASSANDRA-7286 URL: https://issues.apache.org/jira/browse/CASSANDRA-7286 Project: Cassandra Issue Type: Bug Reporter: Julien Anguenot Attachments: npe_cassandra_2_0_8.txt, readstage_npe.txt Sometimes Cassandra nodes (in a multi datacenter deployment) are throwing NPE (see attached stack trace) Let me know what additional information I could provide. Thank you. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097494#comment-14097494 ] Joshua McKenzie commented on CASSANDRA-7577: I added a warning to cqlsh on Windows if you run without the pyreadline dependency - might be worth considering on OSX as well. cqlsh: CTRL-R history search not working on OSX --- Key: CASSANDRA-7577 URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 Project: Cassandra Issue Type: Bug Environment: OSX - plain Terminal program C* 2.0.x, 2.1, trunk Reporter: Robert Stupp Assignee: Robert Stupp Priority: Minor _recursive-history-search_ via ctrl-R does not work in cqlsh. The history itself works via cursor up/down. It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp reopened CASSANDRA-7577: - cqlsh: CTRL-R history search not working on OSX --- Key: CASSANDRA-7577 URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 Project: Cassandra Issue Type: Bug Environment: OSX - plain Terminal program C* 2.0.x, 2.1, trunk Reporter: Robert Stupp Assignee: Robert Stupp Priority: Minor _recursive-history-search_ via ctrl-R does not work in cqlsh. The history itself works via cursor up/down. It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7561) On DROP we should invalidate CounterKeyCache as well as Key/Row cache
[ https://issues.apache.org/jira/browse/CASSANDRA-7561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-7561: - Attachment: 7561.txt On DROP we should invalidate CounterKeyCache as well as Key/Row cache - Key: CASSANDRA-7561 URL: https://issues.apache.org/jira/browse/CASSANDRA-7561 Project: Cassandra Issue Type: Bug Reporter: Benedict Assignee: Aleksey Yeschenko Priority: Minor Fix For: 2.1.0 Attachments: 7561.txt We should also probably ensure we don't attempt to auto save _any_ of the caches while they are in an inconsistent state (i.e. there are keys present to be saved that should not be restored, or that would throw exceptions when we save (e.g. CounterCacheKey)) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-7577: Fix Version/s: 2.1.1 2.0.11 cqlsh: CTRL-R history search not working on OSX --- Key: CASSANDRA-7577 URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 Project: Cassandra Issue Type: Bug Environment: OSX - plain Terminal program C* 2.0.x, 2.1, trunk Reporter: Robert Stupp Assignee: Robert Stupp Priority: Minor Fix For: 2.0.11, 2.1.1 Attachments: 7577-2.0.txt, 7577-2.1.txt _recursive-history-search_ via ctrl-R does not work in cqlsh. The history itself works via cursor up/down. It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-7577: Attachment: 7577-2.1.txt 7577-2.0.txt [~joshuaMcKenzie] good point. Added patches for 2.0 and 2.1. 2.0 patch also backports the Windows test. cqlsh: CTRL-R history search not working on OSX --- Key: CASSANDRA-7577 URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 Project: Cassandra Issue Type: Bug Environment: OSX - plain Terminal program C* 2.0.x, 2.1, trunk Reporter: Robert Stupp Assignee: Robert Stupp Priority: Minor Fix For: 2.0.11, 2.1.1 Attachments: 7577-2.0.txt, 7577-2.1.txt _recursive-history-search_ via ctrl-R does not work in cqlsh. The history itself works via cursor up/down. It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7641) cqlsh should automatically disable tracing when selecting from system_traces
[ https://issues.apache.org/jira/browse/CASSANDRA-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Stepura updated CASSANDRA-7641: --- Attachment: CASSANDRA-2.0-7641-v3.patch [~philipthompson] your patch disables the tracing if a current keyspace is system_traces even if user selects from a different keyspace. I'm attaching v3 of the patch which address that has slightly simplified logic, please take a look cqlsh should automatically disable tracing when selecting from system_traces Key: CASSANDRA-7641 URL: https://issues.apache.org/jira/browse/CASSANDRA-7641 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Brandon Williams Assignee: Philip Thompson Priority: Minor Labels: lhf Fix For: 2.0.10, 2.1.1 Attachments: CASSANDRA-2.0-7641-v3.patch, cassandra-2.0-7641-2.txt, cassandra-2.0-7641.txt, cassandra-2.1-7641.txt Nobody needs to trace their traces while they're tracing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097570#comment-14097570 ] Joshua McKenzie commented on CASSANDRA-7577: Windows portion of the patches lgtm. Good call on backporting it to 2.0 - no reason not to while we're here. cqlsh: CTRL-R history search not working on OSX --- Key: CASSANDRA-7577 URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 Project: Cassandra Issue Type: Bug Environment: OSX - plain Terminal program C* 2.0.x, 2.1, trunk Reporter: Robert Stupp Assignee: Robert Stupp Priority: Minor Fix For: 2.0.11, 2.1.1 Attachments: 7577-2.0.txt, 7577-2.1.txt _recursive-history-search_ via ctrl-R does not work in cqlsh. The history itself works via cursor up/down. It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7740) Parsing of UDF body is broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-7740:
Attachment: 7740.txt
Patch for first string-style variant.
Will open a separate ticket for pg-style variant.
Parsing of UDF body is broken
-
Key: CASSANDRA-7740
URL: https://issues.apache.org/jira/browse/CASSANDRA-7740
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Robert Stupp
Fix For: 3.0
Attachments: 7740.txt
The parsing of function body introduced by CASSANDRA-7395 is somewhat broken.
It blindly parse everything up to {{END_BODY}}, which as 2 problems:
# it parse function body as if it was part of the CQL syntax, so anything
that don't happen to be a valid CQL token won't even parse.
# something like
{noformat}
CREATE FUNCTION foo() RETURNS text LANGUAGE JAVA BODY return END_BODY;
END_BODY;
{noformat}
will not parse correctly.
I don't think we can accept random syntax like that. A better solution (which
is the one Postgresql uses) is to pass the function body as a normal string.
And in fact I'd be in favor of reusing Postgresql syntax (because why not),
that is to have:
{noformat}
CREATE FUNCTION foo() RETURNS text LANGUAGE JAVA AS 'return END_BODY';
{noformat}
One minor annoyance might be, for certain languages, the necessity to double
every quote inside the string. But in a separate ticket we could introduce
Postregsql solution of adding an [alternate syntax for string
constants|http://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html#SQL-SYNTAX-DOLLAR-QUOTING].
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7641) cqlsh should automatically disable tracing when selecting from system_traces
[ https://issues.apache.org/jira/browse/CASSANDRA-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097574#comment-14097574 ] Philip Thompson commented on CASSANDRA-7641: [~mishail], I like your logic better yes. I'll make the same changes for 2.1 and attach. cqlsh should automatically disable tracing when selecting from system_traces Key: CASSANDRA-7641 URL: https://issues.apache.org/jira/browse/CASSANDRA-7641 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Brandon Williams Assignee: Philip Thompson Priority: Minor Labels: lhf Fix For: 2.0.10, 2.1.1 Attachments: CASSANDRA-2.0-7641-v3.patch, cassandra-2.0-7641-2.txt, cassandra-2.0-7641.txt, cassandra-2.1-7641.txt Nobody needs to trace their traces while they're tracing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7708) UDF schema change events/results
[
https://issues.apache.org/jira/browse/CASSANDRA-7708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7708:
Labels: protocolv4 (was: protocolV4)
UDF schema change events/results
Key: CASSANDRA-7708
URL: https://issues.apache.org/jira/browse/CASSANDRA-7708
Project: Cassandra
Issue Type: Improvement
Reporter: Robert Stupp
Assignee: Robert Stupp
Labels: protocolv4
Schema change notifications for UDF might be interesting for client.
This covers both - the result of {{CREATE}} + {{DROP}} statements and events.
Just adding {{FUNCTION}} as a new target for these events breaks previous
native protocol contract.
Proposal is to introduce a new target {{FUNCTION}} in native protocol v4.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7769) Implement pg-style UDF source declaration
Robert Stupp created CASSANDRA-7769:
---
Summary: Implement pg-style UDF source declaration
Key: CASSANDRA-7769
URL: https://issues.apache.org/jira/browse/CASSANDRA-7769
Project: Cassandra
Issue Type: Improvement
Reporter: Robert Stupp
Assignee: Robert Stupp
Fix For: 3.0
Follow-up of CASSANDRA-7740:
{{$function$...$function$}} in addition to string style variant.
See also
http://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html#SQL-SYNTAX-DOLLAR-QUOTING
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7641) cqlsh should automatically disable tracing when selecting from system_traces
[ https://issues.apache.org/jira/browse/CASSANDRA-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-7641: --- Attachment: cassandra-2.1-7641-v3.txt cqlsh should automatically disable tracing when selecting from system_traces Key: CASSANDRA-7641 URL: https://issues.apache.org/jira/browse/CASSANDRA-7641 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Brandon Williams Assignee: Philip Thompson Priority: Minor Labels: lhf Fix For: 2.0.10, 2.1.1 Attachments: CASSANDRA-2.0-7641-v3.patch, cassandra-2.0-7641-2.txt, cassandra-2.0-7641.txt, cassandra-2.1-7641-v3.txt, cassandra-2.1-7641.txt Nobody needs to trace their traces while they're tracing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7641) cqlsh should automatically disable tracing when selecting from system_traces
[ https://issues.apache.org/jira/browse/CASSANDRA-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097607#comment-14097607 ] Philip Thompson commented on CASSANDRA-7641: I've attached the 2.1 patch, and labeled it v3 to match the 2.0 patch for clarity's sake. I'm having trouble testing it because I'm seeing CASSANDRA-7613 even though it should be resolved. cqlsh should automatically disable tracing when selecting from system_traces Key: CASSANDRA-7641 URL: https://issues.apache.org/jira/browse/CASSANDRA-7641 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Brandon Williams Assignee: Philip Thompson Priority: Minor Labels: lhf Fix For: 2.0.10, 2.1.1 Attachments: CASSANDRA-2.0-7641-v3.patch, cassandra-2.0-7641-2.txt, cassandra-2.0-7641.txt, cassandra-2.1-7641-v3.txt, cassandra-2.1-7641.txt Nobody needs to trace their traces while they're tracing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (CASSANDRA-7613) Error when tracing query with 2.1 cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson reopened CASSANDRA-7613: Reproduced In: 2.1.1 Error when tracing query with 2.1 cqlsh --- Key: CASSANDRA-7613 URL: https://issues.apache.org/jira/browse/CASSANDRA-7613 Project: Cassandra Issue Type: Bug Reporter: T Jake Luciani Fix For: 2.1 rc5 cqlsh isn't working for me in 2.1 when executing a query with TRACING ON; I get the following error myformat_colname() takes exactly 3 arguments (2 given) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7613) Error when tracing query with 2.1 cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097609#comment-14097609 ] Philip Thompson commented on CASSANDRA-7613: I am still able to reproduce on 2.1-HEAD this even though CASSANDRA-7603 has been fixed and the patch is committed. Error when tracing query with 2.1 cqlsh --- Key: CASSANDRA-7613 URL: https://issues.apache.org/jira/browse/CASSANDRA-7613 Project: Cassandra Issue Type: Bug Reporter: T Jake Luciani Fix For: 2.1 rc5 cqlsh isn't working for me in 2.1 when executing a query with TRACING ON; I get the following error myformat_colname() takes exactly 3 arguments (2 given) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7613) Error when tracing query with 2.1 cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097613#comment-14097613 ] Robert Stupp commented on CASSANDRA-7613: - bq. still able to reproduce me too Error when tracing query with 2.1 cqlsh --- Key: CASSANDRA-7613 URL: https://issues.apache.org/jira/browse/CASSANDRA-7613 Project: Cassandra Issue Type: Bug Reporter: T Jake Luciani Fix For: 2.1 rc5 cqlsh isn't working for me in 2.1 when executing a query with TRACING ON; I get the following error myformat_colname() takes exactly 3 arguments (2 given) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7770) NPE when clean up compaction leftover if table is already dropped
Yuki Morishita created CASSANDRA-7770: - Summary: NPE when clean up compaction leftover if table is already dropped Key: CASSANDRA-7770 URL: https://issues.apache.org/jira/browse/CASSANDRA-7770 Project: Cassandra Issue Type: Bug Reporter: Yuki Morishita Assignee: Yuki Morishita Priority: Minor Fix For: 2.1.0 Attachments: 0001-add-check-before-cleaning-up-compaction-leftovers.patch As reported in CASSANDRA-7759, Directories throws NPE when trying to remove compaction leftovers on already dropped table. Attaching patch to check if table exists in schema before clean up. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (CASSANDRA-7613) Error when tracing query with 2.1 cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Stepura reassigned CASSANDRA-7613: -- Assignee: Mikhail Stepura Error when tracing query with 2.1 cqlsh --- Key: CASSANDRA-7613 URL: https://issues.apache.org/jira/browse/CASSANDRA-7613 Project: Cassandra Issue Type: Bug Reporter: T Jake Luciani Assignee: Mikhail Stepura Fix For: 2.1 rc5 cqlsh isn't working for me in 2.1 when executing a query with TRACING ON; I get the following error myformat_colname() takes exactly 3 arguments (2 given) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7759) DROP TABLE makes C* unreachable
[
https://issues.apache.org/jira/browse/CASSANDRA-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097620#comment-14097620
]
Yuki Morishita commented on CASSANDRA-7759:
---
[~fabrice.larcher] I think NPE on the comment above is different problem from
description.
I filed CASSANDRA-7770 and attached fix there.
DROP TABLE makes C* unreachable
---
Key: CASSANDRA-7759
URL: https://issues.apache.org/jira/browse/CASSANDRA-7759
Project: Cassandra
Issue Type: Bug
Environment: CQLSH of C* 2.10.-rc5 with one or two node(s).
Reporter: Fabrice Larcher
After a {{DROP KEYSPACE}} or {{DROP TABLE}} command from _CQLSH_ I often get
the output :
{code}
errors={}, last_host=127.0.0.1
{code}
Then, the application can not access the node anymore (it uses the Java
Driver 2.1.0-rc1). And I get the following stacktraces from the
{{system.log}} file ;
{code}
ERROR [MemtableFlushWriter:5] 2014-08-13 11:26:07,577
CassandraDaemon.java:166 - Exception in thread
Thread[MemtableFlushWriter:5,5,main]
java.lang.RuntimeException: Last written key
DecoratedKey(28149890686391545-8456361251720325,
30313034353666392d666464362d343539362d393537372d653539336430333138396437) =
current key DecoratedKey(108b5f3f-fc06-4a0d-99f1-3c6484b32e04,
31303862356633662d666330362d346130642d393966312d336336343834623332653034)
writing into
./../data/data/metrics/run-a3c1fe80216911e49fd1ab9c3338a2ff/metrics-run.run_probe-tmp-ka-2-Data.db
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
~[guava-16.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]
{code}
I also get the following (perhaps not related)
{code}
ERROR [MemtableFlushWriter:6] 2014-08-13 11:26:08,827
CassandraDaemon.java:166 - Exception in thread
Thread[MemtableFlushWriter:6,5,main]
org.apache.cassandra.serializers.MarshalException: Invalid byte for ascii: -50
at
org.apache.cassandra.serializers.AsciiSerializer.validate(AsciiSerializer.java:39)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:78)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at org.apache.cassandra.dht.LocalToken.toString(LocalToken.java:39)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
at org.apache.cassandra.db.DecoratedKey.toString(DecoratedKey.java:118)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
[jira] [Commented] (CASSANDRA-7770) NPE when clean up compaction leftover if table is already dropped
[ https://issues.apache.org/jira/browse/CASSANDRA-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097630#comment-14097630 ] Jonathan Ellis commented on CASSANDRA-7770: --- Shouldn't we also clean out the sstables involved, or do we already have code to clean out sstables that we don't have schema for? NPE when clean up compaction leftover if table is already dropped - Key: CASSANDRA-7770 URL: https://issues.apache.org/jira/browse/CASSANDRA-7770 Project: Cassandra Issue Type: Bug Reporter: Yuki Morishita Assignee: Yuki Morishita Priority: Minor Fix For: 2.1.0 Attachments: 0001-add-check-before-cleaning-up-compaction-leftovers.patch As reported in CASSANDRA-7759, Directories throws NPE when trying to remove compaction leftovers on already dropped table. Attaching patch to check if table exists in schema before clean up. -- This message was sent by Atlassian JIRA (v6.2#6252)
[4/5] git commit: merge
merge Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27cbf553 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27cbf553 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27cbf553 Branch: refs/heads/cassandra-2.1 Commit: 27cbf55343fe59e9c8e98fd8857e57adc672b5a5 Parents: 4c510dc 44764c0 Author: Brandon Williams brandonwilli...@apache.org Authored: Thu Aug 14 15:51:01 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Thu Aug 14 15:51:01 2014 -0500 -- CHANGES.txt | 1 + .../hadoop/cql3/CqlBulkOutputFormat.java| 106 ++ .../hadoop/cql3/CqlBulkRecordWriter.java| 199 +++ 3 files changed, 306 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/27cbf553/CHANGES.txt -- diff --cc CHANGES.txt index cf2f34a,1ac22f9..4e1c2b0 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,42 -1,7 +1,43 @@@ -2.0.10 +2.1.1 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569) + * SSTableExport uses correct validator to create string representation of partition + keys (CASSANDRA-7498) + * Avoid NPEs when receiving type changes for an unknown keyspace (CASSANDRA-7689) + * Add support for custom 2i validation (CASSANDRA-7575) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) +Merged from 2.0: * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508) * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Catch errors when the JVM pulls the rug out from GCInspector (CASSANDRA-5345) + * cqlsh fails when version number parts are not int (CASSANDRA-7524) + + +2.1.0-rc6 + * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765) + * Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741) + * Add tracing entries about concurrent range requests (CASSANDRA-7599) + * (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729) + * Remove netty buffer ref-counting (CASSANDRA-7735) + * Pass mutated cf to index updater for use by PRSI (CASSANDRA-7742) + * Include stress yaml example in release and deb (CASSANDRA-7717) + * workaround for netty issue causing corrupted data off the wire (CASSANDRA-7695) + * cqlsh DESC CLUSTER fails retrieving ring information (CASSANDRA-7687) + * Fix binding null values inside UDT (CASSANDRA-7685) + * Fix UDT field selection with empty fields (CASSANDRA-7670) + * Bogus deserialization of static cells from sstable (CASSANDRA-7684) +Merged from 2.0: + * Remove duplicates from StorageService.getJoiningNodes (CASSANDRA-7478) + * Clone token map outside of hot gossip loops (CASSANDRA-7758) * Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752) * Do not flush on truncate if durable_writes is false (CASSANDRA-7750) * Give CRR a default input_cql Statement (CASSANDRA-7226)
[5/5] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c4077ad6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c4077ad6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c4077ad6 Branch: refs/heads/trunk Commit: c4077ad6615d6f96da1588a68c1805f160d6ad0d Parents: fd43c33 27cbf55 Author: Brandon Williams brandonwilli...@apache.org Authored: Thu Aug 14 15:51:09 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Thu Aug 14 15:51:09 2014 -0500 -- CHANGES.txt | 1 + .../hadoop/cql3/CqlBulkOutputFormat.java| 106 ++ .../hadoop/cql3/CqlBulkRecordWriter.java| 199 +++ 3 files changed, 306 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c4077ad6/CHANGES.txt --
[1/5] git commit: backport CASSANDRA-6927 to 2.0
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 4c510dc6c - 27cbf5534
refs/heads/trunk fd43c33c6 - c4077ad66
backport CASSANDRA-6927 to 2.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44764c03
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44764c03
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44764c03
Branch: refs/heads/cassandra-2.1
Commit: 44764c03e892bfdb7294ad32d9ff703909186917
Parents: efde6ae
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Aug 13 14:14:38 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Aug 13 14:14:38 2014 -0500
--
CHANGES.txt | 1 +
.../hadoop/AbstractBulkOutputFormat.java| 73 ++
.../hadoop/AbstractBulkRecordWriter.java| 251 ++
.../cassandra/hadoop/BulkOutputFormat.java | 49 +---
.../cassandra/hadoop/BulkRecordWriter.java | 259 ++-
.../hadoop/cql3/CqlBulkOutputFormat.java| 106
.../hadoop/cql3/CqlBulkRecordWriter.java| 199 ++
.../cassandra/hadoop/cql3/CqlConfigHelper.java | 2 +-
.../cassandra/hadoop/cql3/CqlOutputFormat.java | 2 +-
.../io/sstable/AbstractSSTableSimpleWriter.java | 10 +-
.../cassandra/io/sstable/CQLSSTableWriter.java | 3 +-
11 files changed, 663 insertions(+), 292 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/44764c03/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3d31948..1ac22f9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.10
+ * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927)
* Don't depend on cassandra config for nodetool ring (CASSANDRA-7508)
* (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703)
* Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/44764c03/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
b/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
new file mode 100644
index 000..c0e91da
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop;
+
+
+import java.io.IOException;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapreduce.*;
+
+public abstract class AbstractBulkOutputFormatK, V extends OutputFormatK, V
+implements org.apache.hadoop.mapred.OutputFormatK, V
+{
+@Override
+public void checkOutputSpecs(JobContext context)
+{
+checkOutputSpecs(HadoopCompat.getConfiguration(context));
+}
+
+private void checkOutputSpecs(Configuration conf)
+{
+if (ConfigHelper.getOutputKeyspace(conf) == null)
+{
+throw new UnsupportedOperationException(you must set the keyspace
with setColumnFamily());
+}
+}
+
+@Override
+public OutputCommitter getOutputCommitter(TaskAttemptContext context)
throws IOException, InterruptedException
+{
+return new NullOutputCommitter();
+}
+
+/** Fills the deprecated OutputFormat interface for streaming. */
+@Deprecated
+public void checkOutputSpecs(org.apache.hadoop.fs.FileSystem filesystem,
org.apache.hadoop.mapred.JobConf job) throws IOException
+{
+checkOutputSpecs(job);
+}
+
+public static class NullOutputCommitter extends OutputCommitter
+{
+public void abortTask(TaskAttemptContext taskContext) { }
+
+public void cleanupJob(JobContext jobContext) { }
+
+public void commitTask(TaskAttemptContext taskContext) { }
+
+public boolean needsTaskCommit(TaskAttemptContext taskContext)
+
[3/5] git commit: merge
merge Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27cbf553 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27cbf553 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27cbf553 Branch: refs/heads/trunk Commit: 27cbf55343fe59e9c8e98fd8857e57adc672b5a5 Parents: 4c510dc 44764c0 Author: Brandon Williams brandonwilli...@apache.org Authored: Thu Aug 14 15:51:01 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Thu Aug 14 15:51:01 2014 -0500 -- CHANGES.txt | 1 + .../hadoop/cql3/CqlBulkOutputFormat.java| 106 ++ .../hadoop/cql3/CqlBulkRecordWriter.java| 199 +++ 3 files changed, 306 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/27cbf553/CHANGES.txt -- diff --cc CHANGES.txt index cf2f34a,1ac22f9..4e1c2b0 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,42 -1,7 +1,43 @@@ -2.0.10 +2.1.1 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569) + * SSTableExport uses correct validator to create string representation of partition + keys (CASSANDRA-7498) + * Avoid NPEs when receiving type changes for an unknown keyspace (CASSANDRA-7689) + * Add support for custom 2i validation (CASSANDRA-7575) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) +Merged from 2.0: * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508) * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Catch errors when the JVM pulls the rug out from GCInspector (CASSANDRA-5345) + * cqlsh fails when version number parts are not int (CASSANDRA-7524) + + +2.1.0-rc6 + * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765) + * Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741) + * Add tracing entries about concurrent range requests (CASSANDRA-7599) + * (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729) + * Remove netty buffer ref-counting (CASSANDRA-7735) + * Pass mutated cf to index updater for use by PRSI (CASSANDRA-7742) + * Include stress yaml example in release and deb (CASSANDRA-7717) + * workaround for netty issue causing corrupted data off the wire (CASSANDRA-7695) + * cqlsh DESC CLUSTER fails retrieving ring information (CASSANDRA-7687) + * Fix binding null values inside UDT (CASSANDRA-7685) + * Fix UDT field selection with empty fields (CASSANDRA-7670) + * Bogus deserialization of static cells from sstable (CASSANDRA-7684) +Merged from 2.0: + * Remove duplicates from StorageService.getJoiningNodes (CASSANDRA-7478) + * Clone token map outside of hot gossip loops (CASSANDRA-7758) * Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752) * Do not flush on truncate if durable_writes is false (CASSANDRA-7750) * Give CRR a default input_cql Statement (CASSANDRA-7226)
[2/5] git commit: backport CASSANDRA-6927 to 2.0
backport CASSANDRA-6927 to 2.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44764c03
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44764c03
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44764c03
Branch: refs/heads/trunk
Commit: 44764c03e892bfdb7294ad32d9ff703909186917
Parents: efde6ae
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Aug 13 14:14:38 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Aug 13 14:14:38 2014 -0500
--
CHANGES.txt | 1 +
.../hadoop/AbstractBulkOutputFormat.java| 73 ++
.../hadoop/AbstractBulkRecordWriter.java| 251 ++
.../cassandra/hadoop/BulkOutputFormat.java | 49 +---
.../cassandra/hadoop/BulkRecordWriter.java | 259 ++-
.../hadoop/cql3/CqlBulkOutputFormat.java| 106
.../hadoop/cql3/CqlBulkRecordWriter.java| 199 ++
.../cassandra/hadoop/cql3/CqlConfigHelper.java | 2 +-
.../cassandra/hadoop/cql3/CqlOutputFormat.java | 2 +-
.../io/sstable/AbstractSSTableSimpleWriter.java | 10 +-
.../cassandra/io/sstable/CQLSSTableWriter.java | 3 +-
11 files changed, 663 insertions(+), 292 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/44764c03/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3d31948..1ac22f9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.10
+ * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927)
* Don't depend on cassandra config for nodetool ring (CASSANDRA-7508)
* (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703)
* Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/44764c03/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
b/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
new file mode 100644
index 000..c0e91da
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop;
+
+
+import java.io.IOException;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapreduce.*;
+
+public abstract class AbstractBulkOutputFormatK, V extends OutputFormatK, V
+implements org.apache.hadoop.mapred.OutputFormatK, V
+{
+@Override
+public void checkOutputSpecs(JobContext context)
+{
+checkOutputSpecs(HadoopCompat.getConfiguration(context));
+}
+
+private void checkOutputSpecs(Configuration conf)
+{
+if (ConfigHelper.getOutputKeyspace(conf) == null)
+{
+throw new UnsupportedOperationException(you must set the keyspace
with setColumnFamily());
+}
+}
+
+@Override
+public OutputCommitter getOutputCommitter(TaskAttemptContext context)
throws IOException, InterruptedException
+{
+return new NullOutputCommitter();
+}
+
+/** Fills the deprecated OutputFormat interface for streaming. */
+@Deprecated
+public void checkOutputSpecs(org.apache.hadoop.fs.FileSystem filesystem,
org.apache.hadoop.mapred.JobConf job) throws IOException
+{
+checkOutputSpecs(job);
+}
+
+public static class NullOutputCommitter extends OutputCommitter
+{
+public void abortTask(TaskAttemptContext taskContext) { }
+
+public void cleanupJob(JobContext jobContext) { }
+
+public void commitTask(TaskAttemptContext taskContext) { }
+
+public boolean needsTaskCommit(TaskAttemptContext taskContext)
+{
+return false;
+}
+
+public void setupJob(JobContext jobContext) { }
+
+public void
[jira] [Created] (CASSANDRA-7771) Allow multiple 2ndary index on the same column
Sylvain Lebresne created CASSANDRA-7771: --- Summary: Allow multiple 2ndary index on the same column Key: CASSANDRA-7771 URL: https://issues.apache.org/jira/browse/CASSANDRA-7771 Project: Cassandra Issue Type: Improvement Reporter: Sylvain Lebresne Priority: Minor Fix For: 3.0 Currently, the code assumes that we'll only have one 2ndary index per column. This has been reasonable so far but stop being it with CASSANDRA-6382 (you might want to index multiple fields of the same UDT column) and CASSANDRA-7458 (you may want to have one normal index an multiple functional index for the same column). So we should consider removing that assumption in the code, which is mainly 2 places: # in the schema: each ColumnDefinition only has infos for one index. This part should probably be tackled in CASSANDRA-6717 so I'm marking this issue as a follow-up of CASSANDRA-6717. # in the 2ndary index API: this is the part I'm suggesting we fix in this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
