[jira] [Commented] (CASSANDRA-15663) DESCRIBE KEYSPACE does not properly quote table names
[
https://issues.apache.org/jira/browse/CASSANDRA-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221615#comment-17221615
]
Alan Boudreault commented on CASSANDRA-15663:
-
[~adelapena] This will be fixed for the next release:
https://datastax-oss.atlassian.net/browse/PYTHON-1269.
> DESCRIBE KEYSPACE does not properly quote table names
> -
>
> Key: CASSANDRA-15663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15663
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Syntax
>Reporter: Oskar Liljeblad
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.11.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> How to reproduce (3.11.6) - cqlsh:
> {code}
> CREATE KEYSPACE test1 WITH replication = \{'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE test1."default" (id text PRIMARY KEY, data text, etag text);
> DESCRIBE KEYSPACE test1;
> {code}
> Output will be:
> {code}
> CREATE TABLE test1.default (
> id text PRIMARY KEY,
> data text,
> etag text
> ) WITH [..]
> {code}
> Output should be:
> {code}
> CREATE TABLE test1."default" (
> id text PRIMARY KEY,
> data text,
> etag text
> ) WITH [..]
> {code}
> If you try to run {{CREATE TABLE test1.default [..]}} you will get an error
> SyntaxException: line 1:19 no viable alternative at input 'default' (CREATE
> TABLE test1.[default]...)
> Oskar Liljeblad
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188051#comment-17188051
]
Alan Boudreault commented on CASSANDRA-15299:
-
The python driver implementation is almost done. As I mentioned, it´s a full
support of the checksumming feature but this iteration simply wraps every
message (no write coalescer).
I plan to release the checksumming in the next version of the driver, which is
planned in 2-3 weeks. However, this change makes the protocol V5 unusable with
current 4.0 beta releases. If you think it´s a problem for the python driver,
let me know.
The PR is here for testing:
[https://github.com/datastax/python-driver/pull/1091]
Let me know if I can help in any other way.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170358#comment-17170358
]
Alan Boudreault commented on CASSANDRA-15299:
-
FYI, we plan to start working on the python driver implementation soon. Since
this driver does not have a write coalescer, we want to provide an
_experimental_ checksumming support that simply wrap every message. We will
evaluate the performance and the addition of a write coalescer in a second
iteration.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15663) DESCRIBE KEYSPACE does not properly quote table names
[
https://issues.apache.org/jira/browse/CASSANDRA-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139873#comment-17139873
]
Alan Boudreault commented on CASSANDRA-15663:
-
[~Ge] We just released cassandra driver 3.24. geomet is not required
anymore at runtime.
> DESCRIBE KEYSPACE does not properly quote table names
> -
>
> Key: CASSANDRA-15663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15663
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Syntax
>Reporter: Oskar Liljeblad
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.11.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> How to reproduce (3.11.6) - cqlsh:
> {code}
> CREATE KEYSPACE test1 WITH replication = \{'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE test1."default" (id text PRIMARY KEY, data text, etag text);
> DESCRIBE KEYSPACE test1;
> {code}
> Output will be:
> {code}
> CREATE TABLE test1.default (
> id text PRIMARY KEY,
> data text,
> etag text
> ) WITH [..]
> {code}
> Output should be:
> {code}
> CREATE TABLE test1."default" (
> id text PRIMARY KEY,
> data text,
> etag text
> ) WITH [..]
> {code}
> If you try to run {{CREATE TABLE test1.default [..]}} you will get an error
> SyntaxException: line 1:19 no viable alternative at input 'default' (CREATE
> TABLE test1.[default]...)
> Oskar Liljeblad
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15862) Use "allow list" or "safe list" instead of the term "whitelist"
[
https://issues.apache.org/jira/browse/CASSANDRA-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17133375#comment-17133375
]
Alan Boudreault commented on CASSANDRA-15862:
-
Just created the python driver ticket about this:
https://datastax-oss.atlassian.net/browse/PYTHON-1250
> Use "allow list" or "safe list" instead of the term "whitelist"
>
>
> Key: CASSANDRA-15862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15862
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Ash Berlin-Taylor
>Priority: Normal
>
> Language matters. I'd like to remove all references in Apache Airflow to
> whitelist or black list, and the Cassandra Python API has some that we can't
> easily remove.
> The recent global events have made this even more relevant, but this has been
> on my radar for a while now. Here is a well written article for why I think
> it matters
> https://www.ncsc.gov.uk/blog-post/terminology-its-not-black-and-white
> {quote}It's fairly common to say whitelisting and blacklisting to describe
> desirable and undesirable things in cyber security.
> However, there's an issue with the terminology. It only makes sense if you
> equate white with 'good, permitted, safe' and black with 'bad, dangerous,
> forbidden'. There are some obvious problems with this. {quote}
> My exposure to is via the Python API where there is the
> cassandra.pollicies.WhiteListRoundRobinPolicy class. I propose that this be
> renamed to AllowListRoundRobinPolicy instead. I do not know if there are
> other references.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15677) Topology events are not sent to clients if the nodes use the same network interface
[
https://issues.apache.org/jira/browse/CASSANDRA-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17104902#comment-17104902
]
Alan Boudreault commented on CASSANDRA-15677:
-
I've unassigned myself in case someone else is looking for a task to pick since
I haven't been able to get back to this yet. This ticket only affect a cluster
that is using a single interface for all nodes (different ports). >From my
discussion with Ariel. CASSANDRA-7544 was tested by running manually all dtests
with the single interface ccm option. There is actually no specific dtests for
that and it's not something that is run regularly. So to complete this ticket:
* If there is no way yet to easily run the dtests with the single interface
mode, it might be a good idea to add something for this.
* Check if there is a test for those topology events (expected)
* Run it and see why it is not failing with the single interface mode
* Run the entire dtests suite with the single interface mode and ensure
everything is OK
> Topology events are not sent to clients if the nodes use the same network
> interface
> ---
>
> Key: CASSANDRA-15677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15677
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Alan Boudreault
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-rc
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *This bug only happens when the cassandra nodes are configured to use a
> single network interface (ip) but different ports. See CASSANDRA-7544.*
> Issue: The topology events aren't sent to clients. The problem is that the
> port is not taken into account when determining if we send it or not:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/Server.java#L624
> To reproduce:
> {code}
> # I think the cassandra-test branch is required to get the -S option
> (USE_SINGLE_INTERFACE)
> ccm create -n4 local40 -v 4.0-alpha2 -S
> {code}
>
> Then run this small python driver script:
> {code}
> import time
> from cassandra.cluster import Cluster
> cluster = Cluster()
> session = cluster.connect()
> while True:
> print(cluster.metadata.all_hosts())
> print([h.is_up for h in cluster.metadata.all_hosts()])
> time.sleep(5)
> {code}
> Then decommission a node:
> {code}
> ccm node2 nodetool disablebinary
> ccm node2 nodetool decommission
> {code}
>
> You should see that the node is never removed from the client cluster
> metadata and the reconnector started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15677) Topology events are not sent to clients if the nodes use the same network interface
[
https://issues.apache.org/jira/browse/CASSANDRA-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault reassigned CASSANDRA-15677:
---
Assignee: (was: Alan Boudreault)
> Topology events are not sent to clients if the nodes use the same network
> interface
> ---
>
> Key: CASSANDRA-15677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15677
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Alan Boudreault
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-rc
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *This bug only happens when the cassandra nodes are configured to use a
> single network interface (ip) but different ports. See CASSANDRA-7544.*
> Issue: The topology events aren't sent to clients. The problem is that the
> port is not taken into account when determining if we send it or not:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/Server.java#L624
> To reproduce:
> {code}
> # I think the cassandra-test branch is required to get the -S option
> (USE_SINGLE_INTERFACE)
> ccm create -n4 local40 -v 4.0-alpha2 -S
> {code}
>
> Then run this small python driver script:
> {code}
> import time
> from cassandra.cluster import Cluster
> cluster = Cluster()
> session = cluster.connect()
> while True:
> print(cluster.metadata.all_hosts())
> print([h.is_up for h in cluster.metadata.all_hosts()])
> time.sleep(5)
> {code}
> Then decommission a node:
> {code}
> ccm node2 nodetool disablebinary
> ccm node2 nodetool decommission
> {code}
>
> You should see that the node is never removed from the client cluster
> metadata and the reconnector started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15762) Update Bundled Python Driver to the latest release
[ https://issues.apache.org/jira/browse/CASSANDRA-15762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-15762: Fix Version/s: 4.0-rc > Update Bundled Python Driver to the latest release > -- > > Key: CASSANDRA-15762 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15762 > Project: Cassandra > Issue Type: Task >Reporter: Alan Boudreault >Priority: Normal > Fix For: 4.0-rc > > > This is not urgent for now since there are some missing 4.x features that > need to be implemented on the driver (checksum). This ticket is to make sure > we update the bundled driver before GA. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15762) Update Bundled Python Driver to the latest release
Alan Boudreault created CASSANDRA-15762: --- Summary: Update Bundled Python Driver to the latest release Key: CASSANDRA-15762 URL: https://issues.apache.org/jira/browse/CASSANDRA-15762 Project: Cassandra Issue Type: Task Reporter: Alan Boudreault This is not urgent for now since there are some missing 4.x features that need to be implemented on the driver (checksum). This ticket is to make sure we update the bundled driver before GA. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15731) cassandra-stress fails when we specify a port per node
[
https://issues.apache.org/jira/browse/CASSANDRA-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-15731:
Workflow: Cassandra Default Workflow (was: Cassandra Bug Workflow)
Issue Type: New Feature (was: Bug)
> cassandra-stress fails when we specify a port per node
> ---
>
> Key: CASSANDRA-15731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15731
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Alan Boudreault
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.x
>
>
> {code}
> $ ./tools/bin/cassandra-stress write n=10 -node 127.0.0.1:9046,127.0.0.1:9048
> ...
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Failed to add
> contact point: 127.0.0.1:9046
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:148)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:116)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:156)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:143)
> at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Caused by: java.lang.IllegalArgumentException: Failed to add contact point:
> 127.0.0.1:9046
> at
> com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:922)
> at
> org.apache.cassandra.stress.util.JavaDriverClient.connect(JavaDriverClient.java:131)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:139)
> ... 6 more
> Caused by: java.net.UnknownHostException: 127.0.0.1:9046: invalid IPv6 address
> at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1341)
> at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
> at
> com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:919)
> ... 8 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15731) cassandra-stress fails when we specify a port per node
[
https://issues.apache.org/jira/browse/CASSANDRA-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-15731:
Fix Version/s: (was: 4.0-beta)
4.x
> cassandra-stress fails when we specify a port per node
> ---
>
> Key: CASSANDRA-15731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15731
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.x
>
>
> {code}
> $ ./tools/bin/cassandra-stress write n=10 -node 127.0.0.1:9046,127.0.0.1:9048
> ...
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Failed to add
> contact point: 127.0.0.1:9046
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:148)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:116)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:156)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:143)
> at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Caused by: java.lang.IllegalArgumentException: Failed to add contact point:
> 127.0.0.1:9046
> at
> com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:922)
> at
> org.apache.cassandra.stress.util.JavaDriverClient.connect(JavaDriverClient.java:131)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:139)
> ... 6 more
> Caused by: java.net.UnknownHostException: 127.0.0.1:9046: invalid IPv6 address
> at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1341)
> at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
> at
> com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:919)
> ... 8 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15731) cassandra-stress fails when we specify a port per node
[
https://issues.apache.org/jira/browse/CASSANDRA-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085799#comment-17085799
]
Alan Boudreault commented on CASSANDRA-15731:
-
Just a note to mention that the use of *-port* has a different purpose. It
allows to specify the port for all nodes. With CASSANDRA-7544, I'd would have
expected the support for cassandra-stress to support port per node as well. So
something like this: *-node whitelist 127.0.0.1:9046,127.0.0.1:9048*. However,
this is definitively not a urgent issue and I'm +1 to change the fixVersion to
_4.x_. Thanks [~adelapena]
> cassandra-stress fails when we specify a port per node
> ---
>
> Key: CASSANDRA-15731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15731
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.0-beta
>
>
> {code}
> $ ./tools/bin/cassandra-stress write n=10 -node 127.0.0.1:9046,127.0.0.1:9048
> ...
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Failed to add
> contact point: 127.0.0.1:9046
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:148)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:116)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:156)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:143)
> at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Caused by: java.lang.IllegalArgumentException: Failed to add contact point:
> 127.0.0.1:9046
> at
> com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:922)
> at
> org.apache.cassandra.stress.util.JavaDriverClient.connect(JavaDriverClient.java:131)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:139)
> ... 6 more
> Caused by: java.net.UnknownHostException: 127.0.0.1:9046: invalid IPv6 address
> at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1341)
> at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
> at
> com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:919)
> ... 8 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15731) cassandra-stress fails when we specify a port per node
[
https://issues.apache.org/jira/browse/CASSANDRA-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-15731:
Description:
{code}
$ ./tools/bin/cassandra-stress write n=10 -node 127.0.0.1:9046,127.0.0.1:9048
...
java.lang.RuntimeException: java.lang.IllegalArgumentException: Failed to add
contact point: 127.0.0.1:9046
at
org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:148)
at
org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:116)
at
org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
at
org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:156)
at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
at org.apache.cassandra.stress.Stress.run(Stress.java:143)
at org.apache.cassandra.stress.Stress.main(Stress.java:62)
Caused by: java.lang.IllegalArgumentException: Failed to add contact point:
127.0.0.1:9046
at
com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:922)
at
org.apache.cassandra.stress.util.JavaDriverClient.connect(JavaDriverClient.java:131)
at
org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:139)
... 6 more
Caused by: java.net.UnknownHostException: 127.0.0.1:9046: invalid IPv6 address
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1341)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
at
com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:919)
... 8 more
{code}
was:
{code}
$ ./tools/bin/cassandra-stress write n=10 -node 127.0.0.1:9046
...
java.lang.RuntimeException: java.lang.IllegalArgumentException: Failed to add
contact point: 127.0.0.1:9046
at
org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:148)
at
org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:116)
at
org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
at
org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:156)
at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
at org.apache.cassandra.stress.Stress.run(Stress.java:143)
at org.apache.cassandra.stress.Stress.main(Stress.java:62)
Caused by: java.lang.IllegalArgumentException: Failed to add contact point:
127.0.0.1:9046
at
com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:922)
at
org.apache.cassandra.stress.util.JavaDriverClient.connect(JavaDriverClient.java:131)
at
org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:139)
... 6 more
Caused by: java.net.UnknownHostException: 127.0.0.1:9046: invalid IPv6 address
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1341)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
at
com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:919)
... 8 more
{code}
> cassandra-stress fails when we specify a port per node
> ---
>
> Key: CASSANDRA-15731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15731
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Priority: Normal
> Fix For: 4.0-beta
>
>
> {code}
> $ ./tools/bin/cassandra-stress write n=10 -node 127.0.0.1:9046,127.0.0.1:9048
> ...
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Failed to add
> contact point: 127.0.0.1:9046
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:148)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:116)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:156)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:143)
> at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Caused by: java.lang.IllegalArgumentException: Failed to add contact point:
> 127.0.0.1:9046
> at
> com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:922)
> at
> org.apache.cassandra.stress.util.JavaDriverClient.connect(JavaDriverClient.java:131)
> at
>
[jira] [Updated] (CASSANDRA-15731) cassandra-stress fails when we specify a port per node
[
https://issues.apache.org/jira/browse/CASSANDRA-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-15731:
Summary: cassandra-stress fails when we specify a port per node (was:
cassandra-stress fails when we specify a node port)
> cassandra-stress fails when we specify a port per node
> ---
>
> Key: CASSANDRA-15731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15731
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Priority: Normal
> Fix For: 4.0-beta
>
>
> {code}
> $ ./tools/bin/cassandra-stress write n=10 -node 127.0.0.1:9046
> ...
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Failed to add
> contact point: 127.0.0.1:9046
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:148)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:116)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:156)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:143)
> at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Caused by: java.lang.IllegalArgumentException: Failed to add contact point:
> 127.0.0.1:9046
> at
> com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:922)
> at
> org.apache.cassandra.stress.util.JavaDriverClient.connect(JavaDriverClient.java:131)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:139)
> ... 6 more
> Caused by: java.net.UnknownHostException: 127.0.0.1:9046: invalid IPv6 address
> at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1341)
> at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
> at
> com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:919)
> ... 8 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15731) cassandra-stress fails when we specify a node port
[
https://issues.apache.org/jira/browse/CASSANDRA-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-15731:
Fix Version/s: 4.0-beta
> cassandra-stress fails when we specify a node port
> ---
>
> Key: CASSANDRA-15731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15731
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Priority: Normal
> Fix For: 4.0-beta
>
>
> {code}
> $ ./tools/bin/cassandra-stress write n=10 -node 127.0.0.1:9046
> ...
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Failed to add
> contact point: 127.0.0.1:9046
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:148)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:116)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:156)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:143)
> at org.apache.cassandra.stress.Stress.main(Stress.java:62)
> Caused by: java.lang.IllegalArgumentException: Failed to add contact point:
> 127.0.0.1:9046
> at
> com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:922)
> at
> org.apache.cassandra.stress.util.JavaDriverClient.connect(JavaDriverClient.java:131)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:139)
> ... 6 more
> Caused by: java.net.UnknownHostException: 127.0.0.1:9046: invalid IPv6 address
> at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1341)
> at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
> at
> com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:919)
> ... 8 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15731) cassandra-stress fails when we specify a node port
Alan Boudreault created CASSANDRA-15731:
---
Summary: cassandra-stress fails when we specify a node port
Key: CASSANDRA-15731
URL: https://issues.apache.org/jira/browse/CASSANDRA-15731
Project: Cassandra
Issue Type: Bug
Reporter: Alan Boudreault
{code}
$ ./tools/bin/cassandra-stress write n=10 -node 127.0.0.1:9046
...
java.lang.RuntimeException: java.lang.IllegalArgumentException: Failed to add
contact point: 127.0.0.1:9046
at
org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:148)
at
org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:116)
at
org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
at
org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:156)
at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
at org.apache.cassandra.stress.Stress.run(Stress.java:143)
at org.apache.cassandra.stress.Stress.main(Stress.java:62)
Caused by: java.lang.IllegalArgumentException: Failed to add contact point:
127.0.0.1:9046
at
com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:922)
at
org.apache.cassandra.stress.util.JavaDriverClient.connect(JavaDriverClient.java:131)
at
org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:139)
... 6 more
Caused by: java.net.UnknownHostException: 127.0.0.1:9046: invalid IPv6 address
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1341)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
at
com.datastax.driver.core.Cluster$Builder.addContactPoint(Cluster.java:919)
... 8 more
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15573) Python 3.8 fails to execute cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073295#comment-17073295
]
Alan Boudreault commented on CASSANDRA-15573:
-
This might help to fix this issue. We'll need to use the newer State class
instead of Pattern for 3.8+: https://github.com/python/typeshed/pull/3412
> Python 3.8 fails to execute cqlsh
> -
>
> Key: CASSANDRA-15573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15573
> Project: Cassandra
> Issue Type: Sub-task
> Components: Tool/cqlsh
>Reporter: Yuki Morishita
>Priority: Normal
>
> Python 3.8 renamed sre_parse.Pattern to sre_parse.State (see
> https://bugs.python.org/issue34681 and corresponding pull request
> https://github.com/python/cpython/pull/9310)
> So when executing cqlsh with Python 3.8, it throws error:
> {code}
> Traceback (most recent call last):
> File ".\bin\cqlsh.py", line 175, in
> from cqlshlib import cql3handling, cqlhandling, pylexotron, sslhandling,
> cqlshhandling
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cql3handling.py", line 19,
> in
> from cqlshlib.cqlhandling import CqlParsingRuleSet, Hint
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cqlhandling.py", line 23,
> in
> from cqlshlib import pylexotron, util
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 342,
> in
> class ParsingRuleSet:
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 343,
> in ParsingRuleSet
> RuleSpecScanner = SaferScanner([
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\saferscanner.py", line 74,
> in __init__
> s = re.sre_parse.Pattern()
> AttributeError: module 'sre_parse' has no attribute 'Pattern'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15680) Transient Replication: java.lang.AssertionError null error
Alan Boudreault created CASSANDRA-15680:
---
Summary: Transient Replication: java.lang.AssertionError null
error
Key: CASSANDRA-15680
URL: https://issues.apache.org/jira/browse/CASSANDRA-15680
Project: Cassandra
Issue Type: Bug
Components: Feature/Transient Replication
Reporter: Alan Boudreault
Attachments: debug.log
This bug happens randomly. It seems that for some reason,
_Replicas.ReplicaCount.allReplicas()_ or _Replicas.ReplicaCount.fullReplicas()_
can be *null* instead of *0* ?
stacktrace:
{code}
ERROR [Native-Transport-Requests-1] 2020-04-01 12:00:20,024 Message.java:874 -
Unexpected exception during request; channel = [id: 0x9891b4c8,
L:/127.0.0.1:9042 - R:/127.0.0.1:48542]
java.lang.AssertionError: null
at
org.apache.cassandra.exceptions.UnavailableException.create(UnavailableException.java:38)
at
org.apache.cassandra.locator.ReplicaPlans.assureSufficientLiveReplicas(ReplicaPlans.java:126)
at
org.apache.cassandra.locator.ReplicaPlans.assureSufficientLiveReplicasForRead(ReplicaPlans.java:109)
at
org.apache.cassandra.locator.ReplicaPlans.forRead(ReplicaPlans.java:593)
at
org.apache.cassandra.service.reads.AbstractReadExecutor.getReadExecutor(AbstractReadExecutor.java:191)
at
org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1798)
at
org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1709)
at
org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1626)
at
org.apache.cassandra.db.SinglePartitionReadCommand.execute(SinglePartitionReadCommand.java:382)
at
org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:69)
at
org.apache.cassandra.service.pager.SinglePartitionPager.fetchPage(SinglePartitionPager.java:32)
at
org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:352)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:400)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:250)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:88)
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
at
org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:489)
at
org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:467)
at
org.apache.cassandra.cql3.CustomPayloadMirroringQueryHandler.processPrepared(CustomPayloadMirroringQueryHandler.java:67)
at
org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:140)
at
org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
at
org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
at
org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
{code}
I can reproduce relativity easily with this python driver test (1 out of 5
runs):
https://github.com/datastax/python-driver/blob/master/tests/integration/long/test_loadbalancingpolicies.py#L622
This was tested with 4.0-alpha3 and trunk. I can help to debug if needed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15659) Better support of Python 3 for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-15659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072118#comment-17072118 ] Alan Boudreault commented on CASSANDRA-15659: - Hello, That makes sense Patrick. I think Python 3.6 is a good choice for the official python3 supported version. And it´s correct to only run dtests with this version. However, I think also it would be better to relax the requirement for the different distributions (removing the strong check). All Python3 versions can be supported at *best effort*. So my suggestion: 1. Remove the strong python 3.6 requirement check 2. Do some manual testing to ensure we can at least connect to a cluster and execute queries with all py3 runtimes. 3. Fix obvious issues if there are. 4. (optional) add a simple smoke test in dtest that is run for all runtimes. I am going to check when I could try to help with the python3 support. > Better support of Python 3 for cqlsh > > > Key: CASSANDRA-15659 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15659 > Project: Cassandra > Issue Type: Task > Components: Tool/cqlsh >Reporter: Stefan Miklosovic >Priority: Normal > Fix For: 4.0-alpha > > > From mailing list: > [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E] > > As of today (24/3/2020) and current trunk, there is Python 3.6 supported (1) > but there is not any 3.6 version ootb in Debian for example. E.g. Buster has > Python 3.7 and other (recent) releases have version 2.7. This means that if > one wants to use Python 3 in Debian, he has to use 3.6 but it is not in the > repository so he has to download / compile / install it on his own. > There should be some sane Python 3 version supported which is as well present > in Debian repository (or requirement to run with 3.6 should be relaxed) . > (1) > [https://github.com/apache/cassandra/blob/bf9a1d487b9ba469e8d740cf7d1cd419535a7e79/bin/cqlsh#L57-L65] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15677) Topology events are not sent to clients if the nodes use the same network interface
[
https://issues.apache.org/jira/browse/CASSANDRA-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071241#comment-17071241
]
Alan Boudreault commented on CASSANDRA-15677:
-
Sure, will setup cassandra-dtests and add a test for this.
> Topology events are not sent to clients if the nodes use the same network
> interface
> ---
>
> Key: CASSANDRA-15677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15677
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Alan Boudreault
>Assignee: Alan Boudreault
>Priority: Normal
> Fix For: 4.0-rc
>
>
> *This bug only happens when the cassandra nodes are configured to use a
> single network interface (ip) but different ports. See CASSANDRA-7544.*
> Issue: The topology events aren't sent to clients. The problem is that the
> port is not taken into account when determining if we send it or not:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/Server.java#L624
> To reproduce:
> {code}
> # I think the cassandra-test branch is required to get the -S option
> (USE_SINGLE_INTERFACE)
> ccm create -n4 local40 -v 4.0-alpha2 -S
> {code}
>
> Then run this small python driver script:
> {code}
> import time
> from cassandra.cluster import Cluster
> cluster = Cluster()
> session = cluster.connect()
> while True:
> print(cluster.metadata.all_hosts())
> print([h.is_up for h in cluster.metadata.all_hosts()])
> time.sleep(5)
> {code}
> Then decommission a node:
> {code}
> ccm node2 nodetool disablebinary
> ccm node2 nodetool decommission
> {code}
>
> You should see that the node is never removed from the client cluster
> metadata and the reconnector started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15677) Topology events are not sent to clients if the nodes use the same network interface
[
https://issues.apache.org/jira/browse/CASSANDRA-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071200#comment-17071200
]
Alan Boudreault commented on CASSANDRA-15677:
-
Here is my proposed patch: https://github.com/apache/cassandra/pull/499
> Topology events are not sent to clients if the nodes use the same network
> interface
> ---
>
> Key: CASSANDRA-15677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15677
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Alan Boudreault
>Priority: Normal
> Fix For: 4.0-rc
>
>
> *This bug only happens when the cassandra nodes are configured to use a
> single network interface (ip) but different ports. See CASSANDRA-7544.*
> Issue: The topology events aren't sent to clients. The problem is that the
> port is not taken into account when determining if we send it or not:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/Server.java#L624
> To reproduce:
> {code}
> # I think the cassandra-test branch is required to get the -S option
> (USE_SINGLE_INTERFACE)
> ccm create -n4 local40 -v 4.0-alpha2 -S
> {code}
>
> Then run this small python driver script:
> {code}
> import time
> from cassandra.cluster import Cluster
> cluster = Cluster()
> session = cluster.connect()
> while True:
> print(cluster.metadata.all_hosts())
> print([h.is_up for h in cluster.metadata.all_hosts()])
> time.sleep(5)
> {code}
> Then decommission a node:
> {code}
> ccm node2 nodetool disablebinary
> ccm node2 nodetool decommission
> {code}
>
> You should see that the node is never removed from the client cluster
> metadata and the reconnector started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15677) Topology events are not sent to clients if the nodes use the same network interface
[
https://issues.apache.org/jira/browse/CASSANDRA-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-15677:
Fix Version/s: 4.0-rc
> Topology events are not sent to clients if the nodes use the same network
> interface
> ---
>
> Key: CASSANDRA-15677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15677
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Alan Boudreault
>Priority: Normal
> Fix For: 4.0-rc
>
>
> *This bug only happens when the cassandra nodes are configured to use a
> single network interface (ip) but different ports. See CASSANDRA-7544.*
> Issue: The topology events aren't sent to clients. The problem is that the
> port is not taken into account when determining if we send it or not:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/Server.java#L624
> To reproduce:
> {code}
> # I think the cassandra-test branch is required to get the -S option
> (USE_SINGLE_INTERFACE)
> ccm create -n4 local40 -v 4.0-alpha2 -S
> {code}
>
> Then run this small python driver script:
> {code}
> import time
> from cassandra.cluster import Cluster
> cluster = Cluster()
> session = cluster.connect()
> while True:
> print(cluster.metadata.all_hosts())
> print([h.is_up for h in cluster.metadata.all_hosts()])
> time.sleep(5)
> {code}
> Then decommission a node:
> {code}
> ccm node2 nodetool disablebinary
> ccm node2 nodetool decommission
> {code}
>
> You should see that the node is never removed from the client cluster
> metadata and the reconnector started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15677) Topology events are not sent to clients if the nodes use the same network interface
Alan Boudreault created CASSANDRA-15677:
---
Summary: Topology events are not sent to clients if the nodes use
the same network interface
Key: CASSANDRA-15677
URL: https://issues.apache.org/jira/browse/CASSANDRA-15677
Project: Cassandra
Issue Type: Bug
Reporter: Alan Boudreault
Assignee: Alan Boudreault
*This bug only happens when the cassandra nodes are configured to use a single
network interface (ip) but different ports. See CASSANDRA-7544.*
Issue: The topology events aren't sent to clients. The problem is that the port
is not taken into account when determining if we send it or not:
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/Server.java#L624
To reproduce:
{code}
# I think the cassandra-test branch is required to get the -S option
(USE_SINGLE_INTERFACE)
ccm create -n4 local40 -v 4.0-alpha2 -S
{code}
Then run this small python driver script:
{code}
import time
from cassandra.cluster import Cluster
cluster = Cluster()
session = cluster.connect()
while True:
print(cluster.metadata.all_hosts())
print([h.is_up for h in cluster.metadata.all_hosts()])
time.sleep(5)
{code}
Then decommission a node:
{code}
ccm node2 nodetool disablebinary
ccm node2 nodetool decommission
{code}
You should see that the node is never removed from the client cluster metadata
and the reconnector started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15670) Transient Replication: unable to insert data when the keyspace is configured with the SimpleStrategy
[
https://issues.apache.org/jira/browse/CASSANDRA-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-15670:
Fix Version/s: 4.0-rc
> Transient Replication: unable to insert data when the keyspace is configured
> with the SimpleStrategy
>
>
> Key: CASSANDRA-15670
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15670
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Transient Replication
>Reporter: Alan Boudreault
>Priority: Normal
> Fix For: 4.0-rc
>
>
> An error is thrown then trying to insert data with the transient replication
> + SimpleStrategy configured.
> Test case:
> {code:java}
> CREATE KEYSPACE test_tr WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '3/1'};
> CREATE TABLE test_tr.users (id int PRIMARY KEY, username text) with
> read_repair ='NONE';
> INSERT INTO test_tr.users (id, username) VALUES (1, 'alan');{code}
>
> traceback:
> {code:java}
> ERROR [Native-Transport-Requests-8] 2020-03-27 10:27:17,188
> ErrorMessage.java:450 - Unexpected exception during request
> java.lang.ClassCastException: org.apache.cassandra.locator.SimpleStrategy
> cannot be cast to org.apache.cassandra.locator.NetworkTopologyStrategy
> at
> org.apache.cassandra.db.ConsistencyLevel.eachQuorumForRead(ConsistencyLevel.java:103)
> at
> org.apache.cassandra.db.ConsistencyLevel.eachQuorumForWrite(ConsistencyLevel.java:112)
> at
> org.apache.cassandra.locator.ReplicaPlans$2.select(ReplicaPlans.java:409)
> at
> org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:353)
> at
> org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:348)
> at
> org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:341)
> at
> org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:330)
> at
> org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:1171)
> at
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:713)
> at
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:951)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:475)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:453)
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:247)
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:233)
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:108)
> at
> org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
> at
> org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
> at
> org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> {code}
>
> -->
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/ConsistencyLevel.java#L103
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15670) Transient Replication: unable to insert data when the keyspace is configured with the SimpleStrategy
Alan Boudreault created CASSANDRA-15670:
---
Summary: Transient Replication: unable to insert data when the
keyspace is configured with the SimpleStrategy
Key: CASSANDRA-15670
URL: https://issues.apache.org/jira/browse/CASSANDRA-15670
Project: Cassandra
Issue Type: Bug
Components: Feature/Transient Replication
Reporter: Alan Boudreault
An error is thrown then trying to insert data with the transient replication +
SimpleStrategy configured.
Test case:
{code:java}
CREATE KEYSPACE test_tr WITH replication = {'class': 'SimpleStrategy',
'replication_factor': '3/1'};
CREATE TABLE test_tr.users (id int PRIMARY KEY, username text) with read_repair
='NONE';
INSERT INTO test_tr.users (id, username) VALUES (1, 'alan');{code}
traceback:
{code:java}
ERROR [Native-Transport-Requests-8] 2020-03-27 10:27:17,188
ErrorMessage.java:450 - Unexpected exception during request
java.lang.ClassCastException: org.apache.cassandra.locator.SimpleStrategy
cannot be cast to org.apache.cassandra.locator.NetworkTopologyStrategy
at
org.apache.cassandra.db.ConsistencyLevel.eachQuorumForRead(ConsistencyLevel.java:103)
at
org.apache.cassandra.db.ConsistencyLevel.eachQuorumForWrite(ConsistencyLevel.java:112)
at
org.apache.cassandra.locator.ReplicaPlans$2.select(ReplicaPlans.java:409)
at
org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:353)
at
org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:348)
at
org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:341)
at
org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:330)
at
org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:1171)
at
org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:713)
at
org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:951)
at
org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:475)
at
org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:453)
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:247)
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:233)
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:108)
at
org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
at
org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
at
org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
{code}
-->
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/ConsistencyLevel.java#L103
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15032) Adjust transient replication keyspace replication options to be more friendly to naive clients
[
https://issues.apache.org/jira/browse/CASSANDRA-15032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068785#comment-17068785
]
Alan Boudreault commented on CASSANDRA-15032:
-
That was a good idea. Datastax drivers are currently adding the parsing fix so
this might not be required anymore.
> Adjust transient replication keyspace replication options to be more friendly
> to naive clients
> --
>
> Key: CASSANDRA-15032
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15032
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Transient Replication
>Reporter: Andy Tolbert
>Priority: Normal
>
> To specify the number of transient replicas, replication options are
> specified like:
> {code}
> ALTER KEYSPACE foo WITH REPLICATION = {'class' : 'NetworkTopologyStrategy',
> 'DC1' : '3/1'};
> ALTER KEYSPACE foo WITH REPLICATION = {'class' : 'SimpleStrategy',
> 'replication_factor' : '3/1'}
> {code}
> It occurred to me that existing client drivers that parse keyspace options
> may not handle this gracefully.
> For example, the datastax java driver tries to parse {{3/1}} as a number and
> fails. In this case, the parsing error is not fatal, its just that the
> metadata for that keyspace in the driver is incomplete, and things like token
> aware routing can't be utilized.
> It is possible that other libraries may not handle this as well.
> As an alternative, I propose adding a separate option like:
> {{'transient_replicas': 1}}. {{replication_factor}} would represent the
> total number of replicas (full and transient) in this case. Something similar
> could be done for the NTS case, but might be slightly clumsy to express.
> This would allow existing client libraries to continue working, and while
> things like routing may be suboptimal (i.e. driver won't know to
> differentiate between replicas and transient replicas), at least parsing
> won't fail in possibly fatal ways.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12041) Add CDC to describe table
[
https://issues.apache.org/jira/browse/CASSANDRA-12041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392939#comment-16392939
]
Alan Boudreault commented on CASSANDRA-12041:
-
[~spo...@gmail.com] I can see that the value of the CDC property is NULL in the
DB. On the driver side, nothing changed. Table options that have a value of
None are ignored during a export_as_string. My understanding is that cdc should
probably be `false` instead of null when not specified during a table creation.
Is this null expected?
> Add CDC to describe table
> -
>
> Key: CASSANDRA-12041
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12041
> Project: Cassandra
> Issue Type: Sub-task
> Components: Tools
>Reporter: Carl Yeksigian
>Assignee: Stefania
>Priority: Major
> Labels: client-impacting
> Fix For: 3.8
>
>
> Currently we do not output CDC with {{DESCRIBE TABLE}}, but should include
> that for 3.8+ tables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12490) Add sequence distribution type to cassandra stress
[ https://issues.apache.org/jira/browse/CASSANDRA-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15578207#comment-15578207 ] Alan Boudreault commented on CASSANDRA-12490: - Awesome. Thanks Ben. > Add sequence distribution type to cassandra stress > -- > > Key: CASSANDRA-12490 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12490 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Ben Slater >Assignee: Ben Slater >Priority: Minor > Fix For: 3.10 > > Attachments: 12490-trunk.patch, 12490.yaml, 12490update-trunk.patch, > cqlstress-seq-example.yaml > > > When using the write command, cassandra stress sequentially generates seeds. > This ensures generated values don't overlap (unless the sequence wraps) > providing more predictable number of inserted records (and generating a base > set of data without wasted writes). > When using a yaml stress spec there is no sequenced distribution available. > It think it would be useful to have this for doing initial load of data for > testing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12490) Add sequence distribution type to cassandra stress
[ https://issues.apache.org/jira/browse/CASSANDRA-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15540529#comment-15540529 ] Alan Boudreault commented on CASSANDRA-12490: - Hey Ben, It looks like when using seq() on partition keys, we are limited to a rate of threads=1. Can you confirm? I wonder if we could do something about this. > Add sequence distribution type to cassandra stress > -- > > Key: CASSANDRA-12490 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12490 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Ben Slater >Assignee: Ben Slater >Priority: Minor > Fix For: 3.10 > > Attachments: 12490-trunk.patch, 12490.yaml, cqlstress-seq-example.yaml > > > When using the write command, cassandra stress sequentially generates seeds. > This ensures generated values don't overlap (unless the sequence wraps) > providing more predictable number of inserted records (and generating a base > set of data without wasted writes). > When using a yaml stress spec there is no sequenced distribution available. > It think it would be useful to have this for doing initial load of data for > testing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11138) cassandra-stress tool - clustering key values not distributed
[
https://issues.apache.org/jira/browse/CASSANDRA-11138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526118#comment-15526118
]
Alan Boudreault commented on CASSANDRA-11138:
-
I confirm that the bug I mentioned in CASSANDRA-12490 is fixed with this patch.
> cassandra-stress tool - clustering key values not distributed
> -
>
> Key: CASSANDRA-11138
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11138
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Cassandra 2.2.4, Centos 6.5, Java 8
>Reporter: Ralf Steppacher
> Labels: stress
> Attachments: 11138-trunk.patch
>
>
> I am trying to get the stress tool to generate random values for three
> clustering keys. I am trying to simulate collecting events per user id (text,
> partition key). Events have a session type (text), event type (text), and
> creation time (timestamp) (clustering keys, in that order). For testing
> purposes I ended up with the following column spec:
> {noformat}
> columnspec:
> - name: created_at
> cluster: uniform(10..10)
> - name: event_type
> size: uniform(5..10)
> population: uniform(1..30)
> cluster: uniform(1..30)
> - name: session_type
> size: fixed(5)
> population: uniform(1..4)
> cluster: uniform(1..4)
> - name: user_id
> size: fixed(15)
> population: uniform(1..100)
> - name: message
> size: uniform(10..100)
> population: uniform(1..100B)
> {noformat}
> My expectation was that this would lead to anywhere between 10 and 1200 rows
> to be created per partition key. But it seems that exactly 10 rows are being
> created, with the {{created_at}} timestamp being the only variable that is
> assigned variable values (per partition key). The {{session_type}} and
> {{event_type}} variables are assigned fixed values. This is even the case if
> I set the cluster distribution to uniform(30..30) and uniform(4..4)
> respectively. With this setting I expected 1200 rows per partition key to be
> created, as announced when running the stress tool, but it is still 10.
> {noformat}
> [rsteppac@centos bin]$ ./cassandra-stress user
> profile=../batch_too_large.yaml ops\(insert=1\) -log level=verbose
> file=~/centos_eventy_patient_session_event_timestamp_insert_only.log -node
> 10.211.55.8
> …
> Created schema. Sleeping 1s for propagation.
> Generating batches with [1..1] partitions and [1..1] rows (of [1200..1200]
> total rows in the partitions)
> Improvement over 4 threadCount: 19%
> ...
> {noformat}
> Sample of generated data:
> {noformat}
> cqlsh> select user_id, event_type, session_type, created_at from
> stresscql.batch_too_large LIMIT 30 ;
> user_id | event_type | session_type | created_at
> -+--+--+--
> %\x7f\x03/.d29 08:14:11+
> %\x7f\x03/.d29 04:04:56+
> %\x7f\x03/.d29 00:39:23+
> %\x7f\x03/.d29 19:56:30+
> %\x7f\x03/.d29 20:46:26+
> %\x7f\x03/.d29 03:27:17+
> %\x7f\x03/.d29 23:30:34+
> %\x7f\x03/.d29 02:41:28+
> %\x7f\x03/.d29 07:23:48+
> %\x7f\x03/.d29 23:23:04+
> N!\x0eUA7^r7d\x06J 17:48:51+
> N!\x0eUA7^r7d\x06J 06:21:13+
> N!\x0eUA7^r7d\x06J 03:34:41+
> N!\x0eUA7^r7d\x06J 05:26:21+
> N!\x0eUA7^r7d\x06J 01:31:24+
> N!\x0eUA7^r7d\x06J 14:22:43+
> N!\x0eUA7^r7d\x06J 14:54:29+
> N!\x0eUA7^r7d\x06J 13:31:54+
> N!\x0eUA7^r7d\x06J 06:38:40+
> N!\x0eUA7^r7d\x06J 21:16:47+
> oy\x1c0077H"i\x07\x13_%\x06 || \nz@Qj\x1cB |E}P^k | 2014-11-23
> 17:05:45+
>
[jira] [Commented] (CASSANDRA-12490) Add sequence distribution type to cassandra stress
[ https://issues.apache.org/jira/browse/CASSANDRA-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524585#comment-15524585 ] Alan Boudreault commented on CASSANDRA-12490: - Thanks Ben! > Add sequence distribution type to cassandra stress > -- > > Key: CASSANDRA-12490 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12490 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Ben Slater >Assignee: Ben Slater >Priority: Minor > Fix For: 3.10 > > Attachments: 12490-trunk.patch, 12490.yaml, cqlstress-seq-example.yaml > > > When using the write command, cassandra stress sequentially generates seeds. > This ensures generated values don't overlap (unless the sequence wraps) > providing more predictable number of inserted records (and generating a base > set of data without wasted writes). > When using a yaml stress spec there is no sequenced distribution available. > It think it would be useful to have this for doing initial load of data for > testing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12490) Add sequence distribution type to cassandra stress
[
https://issues.apache.org/jira/browse/CASSANDRA-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523039#comment-15523039
]
Alan Boudreault commented on CASSANDRA-12490:
-
[~slater_ben] I've been testing this feature during the weekend. At first
sight, it worked well and it's very useful. However, I'm experiencing
unexpected results when I add a third clustered columns. I've attached my yaml
configuration as a test case. There are some comments in the file but here is a
brief description of the issue:
cassandra-stress user profile=12490.yaml ops\(insert=1\) n=10 -rate threads=1
{code}
--> PRIMARY KEY ((stid, year, month), day, hour, minute)
{code}
{code}
- name: day
cluster: fixed(30)
population: seq(1..30)
- name: hour
cluster: fixed(24)
population: seq(1..24)
- name: minute
cluster: fixed(60)
population: seq(1..60)
{code}
With 3 clustered columns, it looks like only the last one is considered. So,
with n=10, I got 600 rows total.. when I should have (60*24*30) rows per
partition. If I remove the minute in the clustering columns, things work as
expected: 7200 rows total (10*24*30).
> Add sequence distribution type to cassandra stress
> --
>
> Key: CASSANDRA-12490
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12490
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Ben Slater
>Assignee: Ben Slater
>Priority: Minor
> Fix For: 3.10
>
> Attachments: 12490-trunk.patch, 12490.yaml, cqlstress-seq-example.yaml
>
>
> When using the write command, cassandra stress sequentially generates seeds.
> This ensures generated values don't overlap (unless the sequence wraps)
> providing more predictable number of inserted records (and generating a base
> set of data without wasted writes).
> When using a yaml stress spec there is no sequenced distribution available.
> It think it would be useful to have this for doing initial load of data for
> testing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12490) Add sequence distribution type to cassandra stress
[ https://issues.apache.org/jira/browse/CASSANDRA-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-12490: Attachment: 12490.yaml > Add sequence distribution type to cassandra stress > -- > > Key: CASSANDRA-12490 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12490 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Ben Slater >Assignee: Ben Slater >Priority: Minor > Fix For: 3.10 > > Attachments: 12490-trunk.patch, 12490.yaml, cqlstress-seq-example.yaml > > > When using the write command, cassandra stress sequentially generates seeds. > This ensures generated values don't overlap (unless the sequence wraps) > providing more predictable number of inserted records (and generating a base > set of data without wasted writes). > When using a yaml stress spec there is no sequenced distribution available. > It think it would be useful to have this for doing initial load of data for > testing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10870) pushed_notifications_test.py:TestPushedNotifications.restart_node_test flapping on C* 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-10870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15108904#comment-15108904 ] Alan Boudreault commented on CASSANDRA-10870: - [~Stefania] I think you worked on this test. Do you have any idea what could be the problem? > pushed_notifications_test.py:TestPushedNotifications.restart_node_test > flapping on C* 2.1 > - > > Key: CASSANDRA-10870 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10870 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jim Witschey >Assignee: DS Test Eng > Fix For: 2.1.x > > > This test flaps on CassCI on 2.1. [~aboudreault] Do I remember correctly that > you did some work on these tests in the past few months? If so, could you > have a look and see if there's some assumption the test makes that don't hold > for 2.1? > Oddly, it fails frequently under JDK8: > http://cassci.datastax.com/job/cassandra-2.1_dtest_jdk8/lastCompletedBuild/testReport/pushed_notifications_test/TestPushedNotifications/restart_node_test/history/ > but less frequently on JDK7: > http://cassci.datastax.com/job/cassandra-2.1_dtest/lastCompletedBuild/testReport/pushed_notifications_test/TestPushedNotifications/restart_node_test/history/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10987) MV add_node_after_mv_test is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089194#comment-15089194
]
Alan Boudreault commented on CASSANDRA-10987:
-
This failure is probably the same than 10978.
> MV add_node_after_mv_test is failing on trunk
> -
>
> Key: CASSANDRA-10987
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10987
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Alan Boudreault
> Fix For: 3.x
>
>
> This failure seems to be flaky.
> http://cassci.datastax.com/job/trunk_dtest/897/testReport/materialized_views_test/TestMaterializedViews/add_node_after_mv_test
> {code}
> ==
> ERROR: add_node_after_mv_test (materialized_views_test.TestMaterializedViews)
> --
> Traceback (most recent call last):
> File "/home/aboudreault/git/cstar/cassandra-dtest/dtest.py", line 558, in
> tearDown
> raise AssertionError('Unexpected error in %s node log: %s' % (node.name,
> errors))
> AssertionError: Unexpected error in node4 node log: ['ERROR [main] 2016-01-08
> 08:03:35,980 MigrationManager.java:164 - Migration task failed to
> complete\nERROR [main] 2016-01-08 08:03:36,980 MigrationManager.java:164 -
> Migration task failed to complete']
> >> begin captured logging <<
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-W5Ng_M
> dtest: DEBUG: removing ccm cluster test at: /tmp/dtest-W5Ng_M
> dtest: DEBUG: clearing ssl stores from [/tmp/dtest-W5Ng_M] directory
> - >> end captured logging << -
> --
> Ran 1 test in 90.385s
> FAILED (errors=1)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-10986) MV add_node_after_mv_test is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-10986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault resolved CASSANDRA-10986.
-
Resolution: Duplicate
connection issue during the creation, duplicate of CASSANDRA-10987
> MV add_node_after_mv_test is failing on trunk
> -
>
> Key: CASSANDRA-10986
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10986
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Alan Boudreault
> Fix For: 3.x
>
>
> This failure seems to be flaky.
> http://cassci.datastax.com/job/trunk_dtest/897/testReport/materialized_views_test/TestMaterializedViews/add_node_after_mv_test
> {code}
> ==
> ERROR: add_node_after_mv_test (materialized_views_test.TestMaterializedViews)
> --
> Traceback (most recent call last):
> File "/home/aboudreault/git/cstar/cassandra-dtest/dtest.py", line 558, in
> tearDown
> raise AssertionError('Unexpected error in %s node log: %s' % (node.name,
> errors))
> AssertionError: Unexpected error in node4 node log: ['ERROR [main] 2016-01-08
> 08:03:35,980 MigrationManager.java:164 - Migration task failed to
> complete\nERROR [main] 2016-01-08 08:03:36,980 MigrationManager.java:164 -
> Migration task failed to complete']
> >> begin captured logging <<
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-W5Ng_M
> dtest: DEBUG: removing ccm cluster test at: /tmp/dtest-W5Ng_M
> dtest: DEBUG: clearing ssl stores from [/tmp/dtest-W5Ng_M] directory
> - >> end captured logging << -
> --
> Ran 1 test in 90.385s
> FAILED (errors=1)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10987) MV add_node_after_mv_test is failing on trunk
Alan Boudreault created CASSANDRA-10987:
---
Summary: MV add_node_after_mv_test is failing on trunk
Key: CASSANDRA-10987
URL: https://issues.apache.org/jira/browse/CASSANDRA-10987
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Alan Boudreault
Fix For: 3.x
This failure seems to be flaky.
http://cassci.datastax.com/job/trunk_dtest/897/testReport/materialized_views_test/TestMaterializedViews/add_node_after_mv_test
{code}
==
ERROR: add_node_after_mv_test (materialized_views_test.TestMaterializedViews)
--
Traceback (most recent call last):
File "/home/aboudreault/git/cstar/cassandra-dtest/dtest.py", line 558, in
tearDown
raise AssertionError('Unexpected error in %s node log: %s' % (node.name,
errors))
AssertionError: Unexpected error in node4 node log: ['ERROR [main] 2016-01-08
08:03:35,980 MigrationManager.java:164 - Migration task failed to
complete\nERROR [main] 2016-01-08 08:03:36,980 MigrationManager.java:164 -
Migration task failed to complete']
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-W5Ng_M
dtest: DEBUG: removing ccm cluster test at: /tmp/dtest-W5Ng_M
dtest: DEBUG: clearing ssl stores from [/tmp/dtest-W5Ng_M] directory
- >> end captured logging << -
--
Ran 1 test in 90.385s
FAILED (errors=1)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10986) MV add_node_after_mv_test is failing on trunk
Alan Boudreault created CASSANDRA-10986:
---
Summary: MV add_node_after_mv_test is failing on trunk
Key: CASSANDRA-10986
URL: https://issues.apache.org/jira/browse/CASSANDRA-10986
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Alan Boudreault
Fix For: 3.x
This failure seems to be flaky.
http://cassci.datastax.com/job/trunk_dtest/897/testReport/materialized_views_test/TestMaterializedViews/add_node_after_mv_test
{code}
==
ERROR: add_node_after_mv_test (materialized_views_test.TestMaterializedViews)
--
Traceback (most recent call last):
File "/home/aboudreault/git/cstar/cassandra-dtest/dtest.py", line 558, in
tearDown
raise AssertionError('Unexpected error in %s node log: %s' % (node.name,
errors))
AssertionError: Unexpected error in node4 node log: ['ERROR [main] 2016-01-08
08:03:35,980 MigrationManager.java:164 - Migration task failed to
complete\nERROR [main] 2016-01-08 08:03:36,980 MigrationManager.java:164 -
Migration task failed to complete']
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-W5Ng_M
dtest: DEBUG: removing ccm cluster test at: /tmp/dtest-W5Ng_M
dtest: DEBUG: clearing ssl stores from [/tmp/dtest-W5Ng_M] directory
- >> end captured logging << -
--
Ran 1 test in 90.385s
FAILED (errors=1)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10973) disk_balance_decommission_test is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-10973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10973:
Description:
http://cassci.datastax.com/job/trunk_dtest/891/testReport/junit/disk_balance_test/TestDiskBalance/disk_balance_decommission_test/
{code}
==
FAIL: disk_balance_decommission_test (disk_balance_test.TestDiskBalance)
--
Traceback (most recent call last):
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
74, in disk_balance_decommission_test
self.assert_balanced(node)
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
127, in assert_balanced
assert_almost_equal(*sums, error=0.2, error_message=node.name)
File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 65, in
assert_almost_equal
assert vmin > vmax * (1.0 - error) or vmin == vmax, "values not within
%.2f%% of the max: %s (%s)" % (error * 100, args, kwargs['error_message'])
AssertionError: values not within 20.00% of the max: (482095, 477840, 612940)
(node2)
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-SLbi3e
- >> end captured logging << -
--
Ran 1 test in 121.295s
FAILED (failures=1)
{code}
was:
http://cassci.datastax.com/job/trunk_dtest/891/testReport/junit/disk_balance_test/TestDiskBalance/disk_balance_bootstrap_test/
{code}
==
FAIL: disk_balance_decommission_test (disk_balance_test.TestDiskBalance)
--
Traceback (most recent call last):
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
74, in disk_balance_decommission_test
self.assert_balanced(node)
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
127, in assert_balanced
assert_almost_equal(*sums, error=0.2, error_message=node.name)
File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 65, in
assert_almost_equal
assert vmin > vmax * (1.0 - error) or vmin == vmax, "values not within
%.2f%% of the max: %s (%s)" % (error * 100, args, kwargs['error_message'])
AssertionError: values not within 20.00% of the max: (482095, 477840, 612940)
(node2)
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-SLbi3e
- >> end captured logging << -
--
Ran 1 test in 121.295s
FAILED (failures=1)
{code}
> disk_balance_decommission_test is failing on trunk
> --
>
> Key: CASSANDRA-10973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10973
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Alan Boudreault
> Fix For: 3.2
>
>
> http://cassci.datastax.com/job/trunk_dtest/891/testReport/junit/disk_balance_test/TestDiskBalance/disk_balance_decommission_test/
> {code}
> ==
> FAIL: disk_balance_decommission_test (disk_balance_test.TestDiskBalance)
> --
> Traceback (most recent call last):
> File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py",
> line 74, in disk_balance_decommission_test
> self.assert_balanced(node)
> File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py",
> line 127, in assert_balanced
> assert_almost_equal(*sums, error=0.2, error_message=node.name)
> File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 65,
> in assert_almost_equal
> assert vmin > vmax * (1.0 - error) or vmin == vmax, "values not within
> %.2f%% of the max: %s (%s)" % (error * 100, args, kwargs['error_message'])
> AssertionError: values not within 20.00% of the max: (482095, 477840, 612940)
> (node2)
> >> begin captured logging <<
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-SLbi3e
> - >> end captured logging << -
> --
> Ran 1 test in 121.295s
> FAILED (failures=1)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10973) disk_balance_decommission_test is failing on trunk
Alan Boudreault created CASSANDRA-10973:
---
Summary: disk_balance_decommission_test is failing on trunk
Key: CASSANDRA-10973
URL: https://issues.apache.org/jira/browse/CASSANDRA-10973
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Alan Boudreault
Fix For: 3.2
{code}
==
FAIL: disk_balance_decommission_test (disk_balance_test.TestDiskBalance)
--
Traceback (most recent call last):
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
74, in disk_balance_decommission_test
self.assert_balanced(node)
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
127, in assert_balanced
assert_almost_equal(*sums, error=0.2, error_message=node.name)
File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 65, in
assert_almost_equal
assert vmin > vmax * (1.0 - error) or vmin == vmax, "values not within
%.2f%% of the max: %s (%s)" % (error * 100, args, kwargs['error_message'])
AssertionError: values not within 20.00% of the max: (482095, 477840, 612940)
(node2)
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-SLbi3e
- >> end captured logging << -
--
Ran 1 test in 121.295s
FAILED (failures=1)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10974) disk_balance_bootstrap_test is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10974:
Description:
http://cassci.datastax.com/job/trunk_dtest/891/testReport/junit/disk_balance_test/TestDiskBalance/disk_balance_bootstrap_test/
{code}
==
FAIL: disk_balance_bootstrap_test (disk_balance_test.TestDiskBalance)
--
Traceback (most recent call last):
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
51, in disk_balance_bootstrap_test
self.assert_balanced(node)
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
127, in assert_balanced
assert_almost_equal(*sums, error=0.2, error_message=node.name)
File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 65, in
assert_almost_equal
assert vmin > vmax * (1.0 - error) or vmin == vmax, "values not within
%.2f%% of the max: %s (%s)" % (error * 100, args, kwargs['error_message'])
AssertionError: values not within 20.00% of the max: (529955, 386060, 473640)
(node4)
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-nNoQzp
- >> end captured logging << -
--
Ran 1 test in 114.862s
FAILED (failures=1)
{code}
was:
{code}
==
FAIL: disk_balance_bootstrap_test (disk_balance_test.TestDiskBalance)
--
Traceback (most recent call last):
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
51, in disk_balance_bootstrap_test
self.assert_balanced(node)
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
127, in assert_balanced
assert_almost_equal(*sums, error=0.2, error_message=node.name)
File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 65, in
assert_almost_equal
assert vmin > vmax * (1.0 - error) or vmin == vmax, "values not within
%.2f%% of the max: %s (%s)" % (error * 100, args, kwargs['error_message'])
AssertionError: values not within 20.00% of the max: (529955, 386060, 473640)
(node4)
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-nNoQzp
- >> end captured logging << -
--
Ran 1 test in 114.862s
FAILED (failures=1)
{code}
> disk_balance_bootstrap_test is failing on trunk
> ---
>
> Key: CASSANDRA-10974
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10974
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Alan Boudreault
> Fix For: 3.2
>
>
> http://cassci.datastax.com/job/trunk_dtest/891/testReport/junit/disk_balance_test/TestDiskBalance/disk_balance_bootstrap_test/
> {code}
> ==
> FAIL: disk_balance_bootstrap_test (disk_balance_test.TestDiskBalance)
> --
> Traceback (most recent call last):
> File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py",
> line 51, in disk_balance_bootstrap_test
> self.assert_balanced(node)
> File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py",
> line 127, in assert_balanced
> assert_almost_equal(*sums, error=0.2, error_message=node.name)
> File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 65,
> in assert_almost_equal
> assert vmin > vmax * (1.0 - error) or vmin == vmax, "values not within
> %.2f%% of the max: %s (%s)" % (error * 100, args, kwargs['error_message'])
> AssertionError: values not within 20.00% of the max: (529955, 386060, 473640)
> (node4)
> >> begin captured logging <<
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-nNoQzp
> - >> end captured logging << -
> --
> Ran 1 test in 114.862s
> FAILED (failures=1)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10973) disk_balance_decommission_test is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-10973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10973:
Description:
http://cassci.datastax.com/job/trunk_dtest/891/testReport/junit/disk_balance_test/TestDiskBalance/disk_balance_bootstrap_test/
{code}
==
FAIL: disk_balance_decommission_test (disk_balance_test.TestDiskBalance)
--
Traceback (most recent call last):
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
74, in disk_balance_decommission_test
self.assert_balanced(node)
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
127, in assert_balanced
assert_almost_equal(*sums, error=0.2, error_message=node.name)
File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 65, in
assert_almost_equal
assert vmin > vmax * (1.0 - error) or vmin == vmax, "values not within
%.2f%% of the max: %s (%s)" % (error * 100, args, kwargs['error_message'])
AssertionError: values not within 20.00% of the max: (482095, 477840, 612940)
(node2)
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-SLbi3e
- >> end captured logging << -
--
Ran 1 test in 121.295s
FAILED (failures=1)
{code}
was:
{code}
==
FAIL: disk_balance_decommission_test (disk_balance_test.TestDiskBalance)
--
Traceback (most recent call last):
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
74, in disk_balance_decommission_test
self.assert_balanced(node)
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
127, in assert_balanced
assert_almost_equal(*sums, error=0.2, error_message=node.name)
File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 65, in
assert_almost_equal
assert vmin > vmax * (1.0 - error) or vmin == vmax, "values not within
%.2f%% of the max: %s (%s)" % (error * 100, args, kwargs['error_message'])
AssertionError: values not within 20.00% of the max: (482095, 477840, 612940)
(node2)
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-SLbi3e
- >> end captured logging << -
--
Ran 1 test in 121.295s
FAILED (failures=1)
{code}
> disk_balance_decommission_test is failing on trunk
> --
>
> Key: CASSANDRA-10973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10973
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Alan Boudreault
> Fix For: 3.2
>
>
> http://cassci.datastax.com/job/trunk_dtest/891/testReport/junit/disk_balance_test/TestDiskBalance/disk_balance_bootstrap_test/
> {code}
> ==
> FAIL: disk_balance_decommission_test (disk_balance_test.TestDiskBalance)
> --
> Traceback (most recent call last):
> File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py",
> line 74, in disk_balance_decommission_test
> self.assert_balanced(node)
> File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py",
> line 127, in assert_balanced
> assert_almost_equal(*sums, error=0.2, error_message=node.name)
> File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 65,
> in assert_almost_equal
> assert vmin > vmax * (1.0 - error) or vmin == vmax, "values not within
> %.2f%% of the max: %s (%s)" % (error * 100, args, kwargs['error_message'])
> AssertionError: values not within 20.00% of the max: (482095, 477840, 612940)
> (node2)
> >> begin captured logging <<
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-SLbi3e
> - >> end captured logging << -
> --
> Ran 1 test in 121.295s
> FAILED (failures=1)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10974) disk_balance_bootstrap_test is failing on trunk
Alan Boudreault created CASSANDRA-10974:
---
Summary: disk_balance_bootstrap_test is failing on trunk
Key: CASSANDRA-10974
URL: https://issues.apache.org/jira/browse/CASSANDRA-10974
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Alan Boudreault
Fix For: 3.2
{code}
==
FAIL: disk_balance_bootstrap_test (disk_balance_test.TestDiskBalance)
--
Traceback (most recent call last):
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
51, in disk_balance_bootstrap_test
self.assert_balanced(node)
File "/home/aboudreault/git/cstar/cassandra-dtest/disk_balance_test.py", line
127, in assert_balanced
assert_almost_equal(*sums, error=0.2, error_message=node.name)
File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 65, in
assert_almost_equal
assert vmin > vmax * (1.0 - error) or vmin == vmax, "values not within
%.2f%% of the max: %s (%s)" % (error * 100, args, kwargs['error_message'])
AssertionError: values not within 20.00% of the max: (529955, 386060, 473640)
(node4)
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-nNoQzp
- >> end captured logging << -
--
Ran 1 test in 114.862s
FAILED (failures=1)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10977) MV view_tombstone_test is failing on trunk
Alan Boudreault created CASSANDRA-10977:
---
Summary: MV view_tombstone_test is failing on trunk
Key: CASSANDRA-10977
URL: https://issues.apache.org/jira/browse/CASSANDRA-10977
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Alan Boudreault
Fix For: 3.2
http://cassci.datastax.com/job/trunk_dtest/893/testReport/materialized_views_test/TestMaterializedViews/view_tombstone_test/
{code}
==
FAIL: view_tombstone_test (materialized_views_test.TestMaterializedViews)
--
Traceback (most recent call last):
File
"/home/aboudreault/git/cstar/cassandra-dtest/materialized_views_test.py", line
735, in view_tombstone_test
assert_none(session, "SELECT * FROM t_by_v WHERE v = 1")
File "/home/aboudreault/git/cstar/cassandra-dtest/assertions.py", line 44, in
assert_none
assert list_res == [], "Expected nothing from %s, but got %s" % (query,
list_res)
AssertionError: Expected nothing from SELECT * FROM t_by_v WHERE v = 1, but got
[[1, 1, u'b', None]]
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-MFSCKQ
- >> end captured logging << -
--
Ran 1 test in 27.986s
FAILED (failures=1)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10978) MV add_write_survey_node_after_mv_test is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-10978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10978:
Summary: MV add_write_survey_node_after_mv_test is failing on trunk (was:
MV add_write_survey_node_after_mv_test is failing)
> MV add_write_survey_node_after_mv_test is failing on trunk
> --
>
> Key: CASSANDRA-10978
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10978
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Alan Boudreault
> Labels: dtest
> Fix For: 3.2
>
>
> http://cassci.datastax.com/job/trunk_dtest/893/testReport/materialized_views_test/TestMaterializedViews/add_write_survey_node_after_mv_test/
> {code}
> ==
> ERROR: add_write_survey_node_after_mv_test
> (materialized_views_test.TestMaterializedViews)
> --
> Traceback (most recent call last):
> File "/home/aboudreault/git/cstar/cassandra-dtest/dtest.py", line 558, in
> tearDown
> raise AssertionError('Unexpected error in %s node log: %s' % (node.name,
> errors))
> AssertionError: Unexpected error in node4 node log: ['ERROR [main] 2016-01-06
> 17:03:41,614 MigrationManager.java:164 - Migration task failed to
> complete\nERROR [main] 2016-01-06 17:03:42,614 MigrationManager.java:164 -
> Migration task failed to complete']
> >> begin captured logging <<
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-14GpwW
> dtest: DEBUG: removing ccm cluster test at: /tmp/dtest-14GpwW
> dtest: DEBUG: clearing ssl stores from [/tmp/dtest-14GpwW] directory
> - >> end captured logging << -
> --
> Ran 1 test in 85.369s
> FAILED (errors=1)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10978) MV add_write_survey_node_after_mv_test is failing
Alan Boudreault created CASSANDRA-10978:
---
Summary: MV add_write_survey_node_after_mv_test is failing
Key: CASSANDRA-10978
URL: https://issues.apache.org/jira/browse/CASSANDRA-10978
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Alan Boudreault
Fix For: 3.2
http://cassci.datastax.com/job/trunk_dtest/893/testReport/materialized_views_test/TestMaterializedViews/add_write_survey_node_after_mv_test/
{code}
==
ERROR: add_write_survey_node_after_mv_test
(materialized_views_test.TestMaterializedViews)
--
Traceback (most recent call last):
File "/home/aboudreault/git/cstar/cassandra-dtest/dtest.py", line 558, in
tearDown
raise AssertionError('Unexpected error in %s node log: %s' % (node.name,
errors))
AssertionError: Unexpected error in node4 node log: ['ERROR [main] 2016-01-06
17:03:41,614 MigrationManager.java:164 - Migration task failed to
complete\nERROR [main] 2016-01-06 17:03:42,614 MigrationManager.java:164 -
Migration task failed to complete']
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-14GpwW
dtest: DEBUG: removing ccm cluster test at: /tmp/dtest-14GpwW
dtest: DEBUG: clearing ssl stores from [/tmp/dtest-14GpwW] directory
- >> end captured logging << -
--
Ran 1 test in 85.369s
FAILED (errors=1)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10776) Prepare of statements after table creation fail with unconfigured column family
[
https://issues.apache.org/jira/browse/CASSANDRA-10776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10776:
Component/s: CQL
> Prepare of statements after table creation fail with unconfigured column
> family
> ---
>
> Key: CASSANDRA-10776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10776
> Project: Cassandra
> Issue Type: Bug
>Reporter: Adam Dougal
>
> Cassandra 2.1.8
> We have multiple app instances trying to create the same table using IF NOT
> EXISTS.
> We check for schema agreement via the Java Driver before and after every
> statement.
> After creating the table we then prepare statements and we sometimes get:
> {code}
> com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured
> columnfamily locks
> at
> com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:50)
> ~[cassandra-driver-core-2.1.8.jar:na]
> at
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> ~[cassandra-driver-core-2.1.8.jar:na]
> at
> com.datastax.driver.core.AbstractSession.prepare(AbstractSession.java:79)
> ~[cassandra-driver-core-2.1.8.jar:na]
> at
> uk.sky.cirrus.locking.CassandraLockingMechanism.init(CassandraLockingMechanism.java:69)
> ~[main/:na]
> at uk.sky.cirrus.locking.Lock.acquire(Lock.java:35) [main/:na]
> at uk.sky.cirrus.CqlMigratorImpl.migrate(CqlMigratorImpl.java:83)
> [main/:na]
> at
> uk.sky.cirrus.locking.LockVerificationTest.lambda$shouldManageContentionsForSchemaMigrate$0(LockVerificationTest.java:90)
> [test/:na]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_60]
> {code}
> Looking at the server logs we get:
> {code}
> ava.lang.RuntimeException:
> org.apache.cassandra.exceptions.ConfigurationException: Column family ID
> mismatch (found 90bbb372-9446-11e5-b1ca-8119a6964819; expected
> 90b87f20-9446-11e5-b1ca-8119a6964819)
> at org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1145)
> ~[main/:na]
> at
> org.apache.cassandra.db.DefsTables.updateColumnFamily(DefsTables.java:422)
> ~[main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:295)
> ~[main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeSchemaInternal(DefsTables.java:194)
> ~[main/:na]
> at org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:166)
> ~[main/:na]
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49)
> ~[main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> {code}
> We found this issue which is marked as resolved:
> https://issues.apache.org/jira/browse/CASSANDRA-8387
> Does the IF NOT EXISTS just check the local node?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10776) Prepare of statements after table creation fail with unconfigured column family
[
https://issues.apache.org/jira/browse/CASSANDRA-10776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10776:
Component/s: (was: CQL)
> Prepare of statements after table creation fail with unconfigured column
> family
> ---
>
> Key: CASSANDRA-10776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10776
> Project: Cassandra
> Issue Type: Bug
>Reporter: Adam Dougal
>
> Cassandra 2.1.8
> We have multiple app instances trying to create the same table using IF NOT
> EXISTS.
> We check for schema agreement via the Java Driver before and after every
> statement.
> After creating the table we then prepare statements and we sometimes get:
> {code}
> com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured
> columnfamily locks
> at
> com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:50)
> ~[cassandra-driver-core-2.1.8.jar:na]
> at
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> ~[cassandra-driver-core-2.1.8.jar:na]
> at
> com.datastax.driver.core.AbstractSession.prepare(AbstractSession.java:79)
> ~[cassandra-driver-core-2.1.8.jar:na]
> at
> uk.sky.cirrus.locking.CassandraLockingMechanism.init(CassandraLockingMechanism.java:69)
> ~[main/:na]
> at uk.sky.cirrus.locking.Lock.acquire(Lock.java:35) [main/:na]
> at uk.sky.cirrus.CqlMigratorImpl.migrate(CqlMigratorImpl.java:83)
> [main/:na]
> at
> uk.sky.cirrus.locking.LockVerificationTest.lambda$shouldManageContentionsForSchemaMigrate$0(LockVerificationTest.java:90)
> [test/:na]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_60]
> {code}
> Looking at the server logs we get:
> {code}
> ava.lang.RuntimeException:
> org.apache.cassandra.exceptions.ConfigurationException: Column family ID
> mismatch (found 90bbb372-9446-11e5-b1ca-8119a6964819; expected
> 90b87f20-9446-11e5-b1ca-8119a6964819)
> at org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1145)
> ~[main/:na]
> at
> org.apache.cassandra.db.DefsTables.updateColumnFamily(DefsTables.java:422)
> ~[main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:295)
> ~[main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeSchemaInternal(DefsTables.java:194)
> ~[main/:na]
> at org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:166)
> ~[main/:na]
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49)
> ~[main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> {code}
> We found this issue which is marked as resolved:
> https://issues.apache.org/jira/browse/CASSANDRA-8387
> Does the IF NOT EXISTS just check the local node?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10780) Exception encountered during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-10780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029904#comment-15029904
]
Alan Boudreault commented on CASSANDRA-10780:
-
[~mlowicki] Do you have this exception on each startup? Can you provide the
complete system.log using the loglevel debug?
> Exception encountered during startup
>
>
> Key: CASSANDRA-10780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10780
> Project: Cassandra
> Issue Type: Bug
> Environment: C* 2.1.11 on Debian Wheezy
>Reporter: mlowicki
>
> {code}
> ERROR [main] 2015-11-27 12:39:42,659 CassandraDaemon.java:579 - Exception
> encountered during startup
> org.apache.cassandra.io.FSReadError: java.lang.NullPointerException
> at
> org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:663)
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:306)
> [apache-cassandra-2.1.11.jar:2.1.11]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:562)
> [apache-cassandra-2.1.11.jar:2.1.11]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:651)
> [apache-cassandra-2.1.11.jar:2.1.11]
> Caused by: java.lang.NullPointerException: null
> at
> org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:655)
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> ... 3 common frames omitted
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10614) AssertionError while flushing memtables
[
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14989560#comment-14989560
]
Alan Boudreault commented on CASSANDRA-10614:
-
on it!
> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: debug.log, system.log
>
>
> While running mvbench against a single local node, I noticed this stacktrace
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
> at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
> ~[main/:na]
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
> ~[main/:na]
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352)
> ~[main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> at
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
> ~[guava-18.0.jar:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10614) AssertionError while flushing memtables
[
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990145#comment-14990145
]
Alan Boudreault commented on CASSANDRA-10614:
-
"Bootstrap Completed" \o/. Thanks!
> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: debug.log, system.log
>
>
> While running mvbench against a single local node, I noticed this stacktrace
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
> at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
> ~[main/:na]
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
> ~[main/:na]
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352)
> ~[main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> at
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
> ~[guava-18.0.jar:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10614) AssertionError while flushing memtables
[
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10614:
Attachment: debug.log
system.log
I've been able to reproduce the issue with the same test case than this
morning. Here are the [^system.log] and [^debug.log].
> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: debug.log, system.log
>
>
> While running mvbench against a single local node, I noticed this stacktrace
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
> at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
> ~[main/:na]
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
> ~[main/:na]
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352)
> ~[main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> at
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
> ~[guava-18.0.jar:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10621) Error while bootstraping a new node with Materialized Views
[
https://issues.apache.org/jira/browse/CASSANDRA-10621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982637#comment-14982637
]
Alan Boudreault commented on CASSANDRA-10621:
-
[~jkni] I tested the patch and the bootstrap works well now. Thanks!!
> Error while bootstraping a new node with Materialized Views
> ---
>
> Key: CASSANDRA-10621
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10621
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Joel Knighton
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: system.log
>
>
> If I try to add a new node in a cluster that has materialized views, I get
> the following error:
> {code}
> ERROR 19:05:15 Unknown exception caught while attempting to update
> MaterializedView! mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> [main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> [main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> [main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 19:05:15 Error applying streamed sstable:
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> ~[main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> WARN 19:05:18 Writing large partition mview/genre_to_user:genre_3 (116986142
> bytes)
> WARN 19:05:21 Writing large partition mview/genre_to_user:genre_2 (116985746
> bytes)
> WARN 19:05:24 Writing large partition mview/genre_to_user:genre_5 (116986337
> bytes)
> ERROR 19:05:33 Unknown exception caught while attempting to update
> MaterializedView! mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> [main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> [main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> [main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>
[jira] [Commented] (CASSANDRA-10614) AssertionError while flushing memtables
[
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982904#comment-14982904
]
Alan Boudreault commented on CASSANDRA-10614:
-
I'm experiencing this crash during my tests, seems related to this patch:
{code}
ERROR [MemtableFlushWriter:4] 2015-10-30 17:06:52,604 CassandraDaemon.java:195
- Exception in thread Thread[MemtableFlushWriter:4,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
~[main/:na]
at
org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
~[main/:na]
at
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:111)
~[main/:na]
at
org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
~[main/:na]
at
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
~[main/:na]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
~[main/:na]
at
org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
~[guava-18.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
~[main/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
ERROR 17:06:52 Exception in thread Thread[MemtableFlushWriter:4,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
~[main/:na]
at
org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
~[main/:na]
at
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:111)
~[main/:na]
at
org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
~[main/:na]
at
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
~[main/:na]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
~[main/:na]
at
org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
~[guava-18.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
~[main/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
{code}
> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
>
> While running mvbench against a single local node, I noticed this stacktrace
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
> at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
> ~[main/:na]
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
>
[jira] [Comment Edited] (CASSANDRA-10614) AssertionError while flushing memtables
[
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982904#comment-14982904
]
Alan Boudreault edited comment on CASSANDRA-10614 at 10/30/15 5:17 PM:
---
I'm experiencing this crash during my tests, seems related to this patch. My
test is bootstraping a new node with MVs.
{code}
ERROR [MemtableFlushWriter:4] 2015-10-30 17:06:52,604 CassandraDaemon.java:195
- Exception in thread Thread[MemtableFlushWriter:4,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
~[main/:na]
at
org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
~[main/:na]
at
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:111)
~[main/:na]
at
org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
~[main/:na]
at
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
~[main/:na]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
~[main/:na]
at
org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
~[guava-18.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
~[main/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
ERROR 17:06:52 Exception in thread Thread[MemtableFlushWriter:4,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
~[main/:na]
at
org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
~[main/:na]
at
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:111)
~[main/:na]
at
org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
~[main/:na]
at
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
~[main/:na]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
~[main/:na]
at
org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
~[guava-18.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
~[main/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
{code}
was (Author: aboudreault):
I'm experiencing this crash during my tests, seems related to this patch:
{code}
ERROR [MemtableFlushWriter:4] 2015-10-30 17:06:52,604 CassandraDaemon.java:195
- Exception in thread Thread[MemtableFlushWriter:4,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
~[main/:na]
at
org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
~[main/:na]
at
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:111)
~[main/:na]
at
org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
[jira] [Commented] (CASSANDRA-10614) AssertionError while flushing memtables
[
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980551#comment-14980551
]
Alan Boudreault commented on CASSANDRA-10614:
-
[~thobbs] I confirm that your patch fixes my performance regression issue.
> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
> Issue Type: Bug
> Components: Core, Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
>
> While running mvbench against a single local node, I noticed this stacktrace
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
> at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
> ~[main/:na]
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
> ~[main/:na]
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352)
> ~[main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> at
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
> ~[guava-18.0.jar:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10609) MV performance regression
[
https://issues.apache.org/jira/browse/CASSANDRA-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980557#comment-14980557
]
Alan Boudreault commented on CASSANDRA-10609:
-
This issue will be fix with CASSANDRA-10614
> MV performance regression
> -
>
> Key: CASSANDRA-10609
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10609
> Project: Cassandra
> Issue Type: Bug
> Environment: EC2
>Reporter: Alan Boudreault
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
>
> I've noticed an important MV performance regression that has been introduced
> in 3.0.0-rc1. The issue has been introduced by CASSANDRA-9664.
> * I'm using mvbench to test with RF=3
> * I confirm it's not a driver issue.
> {code}
> EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100
> --num-artists 1 -n 50 --endpoint node1"
> 3.0.0-beta2 (alpha2 java driver)
> ---
> total
> count = 461601
> mean rate = 1923.21 calls/second
> 1-minute rate = 1937.82 calls/second
> 5-minute rate = 1424.09 calls/second
> 15-minute rate = 1058.28 calls/second
>min = 1.90 milliseconds
>max = 3707.76 milliseconds
> mean = 516.42 milliseconds
> stddev = 457.41 milliseconds
> median = 390.07 milliseconds
> 75% <= 775.95 milliseconds
> 95% <= 1417.67 milliseconds
> 98% <= 1728.05 milliseconds
> 99% <= 1954.55 milliseconds
> 99.9% <= 2566.91 milliseconds
> 3.0.0-rc1 (alpha3 java driver)
> -
> total
> count = 310373
> mean rate = 272.25 calls/second
> 1-minute rate = 0.00 calls/second
> 5-minute rate = 45.47 calls/second
> 15-minute rate = 295.94 calls/second
>min = 1.05 milliseconds
>max = 10468.98 milliseconds
> mean = 492.99 milliseconds
> stddev = 510.42 milliseconds
> median = 281.02 milliseconds
> 75% <= 696.25 milliseconds
> 95% <= 1434.45 milliseconds
> 98% <= 1820.33 milliseconds
> 99% <= 2080.37 milliseconds
> 99.9% <= 4362.08 milliseconds
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10621) Error while bootstraping a new node with Materialized Views
Alan Boudreault created CASSANDRA-10621:
---
Summary: Error while bootstraping a new node with Materialized
Views
Key: CASSANDRA-10621
URL: https://issues.apache.org/jira/browse/CASSANDRA-10621
Project: Cassandra
Issue Type: Bug
Reporter: Alan Boudreault
Assignee: Joel Knighton
Priority: Critical
Fix For: 3.0.0
If I try to add a new node in a cluster that has materialized views, I get the
following error:
{code}
ERROR 19:05:15 Unknown exception caught while attempting to update
MaterializedView! mview.user_playlists
java.lang.RuntimeException: Trying to get the view natural endpoint on a
non-data replica
at
org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
~[main/:na]
at
org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
~[main/:na]
at
org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
~[main/:na]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) [main/:na]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) [main/:na]
at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
[main/:na]
at
org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR 19:05:15 Error applying streamed sstable:
java.lang.RuntimeException: Trying to get the view natural endpoint on a
non-data replica
at
org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
~[main/:na]
at
org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
~[main/:na]
at
org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
~[main/:na]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) ~[main/:na]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) ~[main/:na]
at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
~[main/:na]
at
org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
WARN 19:05:18 Writing large partition mview/genre_to_user:genre_3 (116986142
bytes)
WARN 19:05:21 Writing large partition mview/genre_to_user:genre_2 (116985746
bytes)
WARN 19:05:24 Writing large partition mview/genre_to_user:genre_5 (116986337
bytes)
ERROR 19:05:33 Unknown exception caught while attempting to update
MaterializedView! mview.user_playlists
java.lang.RuntimeException: Trying to get the view natural endpoint on a
non-data replica
at
org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
~[main/:na]
at
org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
~[main/:na]
at
org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
~[main/:na]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) [main/:na]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) [main/:na]
at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
[main/:na]
at
org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR 19:05:33 Error applying streamed sstable:
java.lang.RuntimeException: Trying to get the view natural endpoint on a
non-data replica
at
[jira] [Updated] (CASSANDRA-10621) Error while bootstraping a new node with Materialized Views
[
https://issues.apache.org/jira/browse/CASSANDRA-10621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10621:
Attachment: system.log
[~jkni] here is the system.log
> Error while bootstraping a new node with Materialized Views
> ---
>
> Key: CASSANDRA-10621
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10621
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Joel Knighton
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: system.log
>
>
> If I try to add a new node in a cluster that has materialized views, I get
> the following error:
> {code}
> ERROR 19:05:15 Unknown exception caught while attempting to update
> MaterializedView! mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> [main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> [main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> [main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 19:05:15 Error applying streamed sstable:
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> ~[main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> WARN 19:05:18 Writing large partition mview/genre_to_user:genre_3 (116986142
> bytes)
> WARN 19:05:21 Writing large partition mview/genre_to_user:genre_2 (116985746
> bytes)
> WARN 19:05:24 Writing large partition mview/genre_to_user:genre_5 (116986337
> bytes)
> ERROR 19:05:33 Unknown exception caught while attempting to update
> MaterializedView! mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> [main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> [main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> [main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at
[jira] [Reopened] (CASSANDRA-9664) Allow MV's select statements to be more complex
[
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault reopened CASSANDRA-9664:
Reproduced In: 3.0.0 rc1
Reopening this issue. This change caused an important regression with MV
performance. I'm using mvbench to test with RF=3 and I confirm it's not a
driver issue.
{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists
1 -n 50 --endpoint node1"
3.0.0-beta2 (alpha2 java driver)
---
total
count = 461601
mean rate = 1923.21 calls/second
1-minute rate = 1937.82 calls/second
5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
min = 1.90 milliseconds
max = 3707.76 milliseconds
mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
75% <= 775.95 milliseconds
95% <= 1417.67 milliseconds
98% <= 1728.05 milliseconds
99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds
3.0.0-rc1 (alpha3 java driver)
-
total
count = 310373
mean rate = 272.25 calls/second
1-minute rate = 0.00 calls/second
5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
min = 1.05 milliseconds
max = 10468.98 milliseconds
mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
75% <= 696.25 milliseconds
95% <= 1434.45 milliseconds
98% <= 1820.33 milliseconds
99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}
[~thobbs] any idea what could be wrong?
> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
> Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add
> support for a syntax which includes a {{SELECT}} statement, but only allows
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity
> with the normal CQL {{SELECT}} statement, specifically simple functions in
> the selected columns, as well as specifying a {{WHERE}} clause.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-9664) Allow MV's select statements to be more complex
[
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978456#comment-14978456
]
Alan Boudreault edited comment on CASSANDRA-9664 at 10/28/15 1:59 PM:
--
Reopening this issue. This change caused an important regression with MV
performance. I'm using mvbench to test with RF=3 and I confirm it's not a
driver issue.
{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists
1 -n 50 --endpoint node1"
3.0.0-beta2 (alpha2 java driver)
---
total
count = 461601
mean rate = 1923.21 calls/second
1-minute rate = 1937.82 calls/second
5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
min = 1.90 milliseconds
max = 3707.76 milliseconds
mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
75% <= 775.95 milliseconds
95% <= 1417.67 milliseconds
98% <= 1728.05 milliseconds
99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds
3.0.0-rc1 (alpha3 java driver)
-
total
count = 310373
mean rate = 272.25 calls/second
1-minute rate = 0.00 calls/second
5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
min = 1.05 milliseconds
max = 10468.98 milliseconds
mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
75% <= 696.25 milliseconds
95% <= 1434.45 milliseconds
98% <= 1820.33 milliseconds
99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}
[~thobbs] any idea what could make things getting slower?
was (Author: aboudreault):
Reopening this issue. This change caused an important regression with MV
performance. I'm using mvbench to test with RF=3 and I confirm it's not a
driver issue.
{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists
1 -n 50 --endpoint node1"
3.0.0-beta2 (alpha2 java driver)
---
total
count = 461601
mean rate = 1923.21 calls/second
1-minute rate = 1937.82 calls/second
5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
min = 1.90 milliseconds
max = 3707.76 milliseconds
mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
75% <= 775.95 milliseconds
95% <= 1417.67 milliseconds
98% <= 1728.05 milliseconds
99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds
3.0.0-rc1 (alpha3 java driver)
-
total
count = 310373
mean rate = 272.25 calls/second
1-minute rate = 0.00 calls/second
5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
min = 1.05 milliseconds
max = 10468.98 milliseconds
mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
75% <= 696.25 milliseconds
95% <= 1434.45 milliseconds
98% <= 1820.33 milliseconds
99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}
[~thobbs] any idea what could be wrong?
> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
> Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add
> support for a syntax which includes a {{SELECT}} statement, but only allows
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity
> with the normal CQL {{SELECT}} statement, specifically simple functions in
> the selected columns, as well as specifying a {{WHERE}} clause.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-9664) Allow MV's select statements to be more complex
[
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault resolved CASSANDRA-9664.
Resolution: Fixed
Closing in favor of CASSANDRA-10609.
> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
> Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add
> support for a syntax which includes a {{SELECT}} statement, but only allows
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity
> with the normal CQL {{SELECT}} statement, specifically simple functions in
> the selected columns, as well as specifying a {{WHERE}} clause.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10609) MV performance regression
Alan Boudreault created CASSANDRA-10609:
---
Summary: MV performance regression
Key: CASSANDRA-10609
URL: https://issues.apache.org/jira/browse/CASSANDRA-10609
Project: Cassandra
Issue Type: Bug
Environment: EC2
Reporter: Alan Boudreault
Assignee: Tyler Hobbs
Priority: Critical
Fix For: 3.0.0
I've noticed an important MV performance regression that has been introduced in
3.0.0-rc1. The issue has been introduced by #9664.
* I'm using mvbench to test with RF=3
* I confirm it's not a driver issue.
{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists
1 -n 50 --endpoint node1"
3.0.0-beta2 (alpha2 java driver)
---
total
count = 461601
mean rate = 1923.21 calls/second
1-minute rate = 1937.82 calls/second
5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
min = 1.90 milliseconds
max = 3707.76 milliseconds
mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
75% <= 775.95 milliseconds
95% <= 1417.67 milliseconds
98% <= 1728.05 milliseconds
99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds
3.0.0-rc1 (alpha3 java driver)
-
total
count = 310373
mean rate = 272.25 calls/second
1-minute rate = 0.00 calls/second
5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
min = 1.05 milliseconds
max = 10468.98 milliseconds
mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
75% <= 696.25 milliseconds
95% <= 1434.45 milliseconds
98% <= 1820.33 milliseconds
99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10609) MV performance regression
[
https://issues.apache.org/jira/browse/CASSANDRA-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10609:
Description:
I've noticed an important MV performance regression that has been introduced in
3.0.0-rc1. The issue has been introduced by CASSANDRA-9664.
* I'm using mvbench to test with RF=3
* I confirm it's not a driver issue.
{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists
1 -n 50 --endpoint node1"
3.0.0-beta2 (alpha2 java driver)
---
total
count = 461601
mean rate = 1923.21 calls/second
1-minute rate = 1937.82 calls/second
5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
min = 1.90 milliseconds
max = 3707.76 milliseconds
mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
75% <= 775.95 milliseconds
95% <= 1417.67 milliseconds
98% <= 1728.05 milliseconds
99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds
3.0.0-rc1 (alpha3 java driver)
-
total
count = 310373
mean rate = 272.25 calls/second
1-minute rate = 0.00 calls/second
5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
min = 1.05 milliseconds
max = 10468.98 milliseconds
mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
75% <= 696.25 milliseconds
95% <= 1434.45 milliseconds
98% <= 1820.33 milliseconds
99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}
was:
I've noticed an important MV performance regression that has been introduced in
3.0.0-rc1. The issue has been introduced by #9664.
* I'm using mvbench to test with RF=3
* I confirm it's not a driver issue.
{code}
EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 --num-artists
1 -n 50 --endpoint node1"
3.0.0-beta2 (alpha2 java driver)
---
total
count = 461601
mean rate = 1923.21 calls/second
1-minute rate = 1937.82 calls/second
5-minute rate = 1424.09 calls/second
15-minute rate = 1058.28 calls/second
min = 1.90 milliseconds
max = 3707.76 milliseconds
mean = 516.42 milliseconds
stddev = 457.41 milliseconds
median = 390.07 milliseconds
75% <= 775.95 milliseconds
95% <= 1417.67 milliseconds
98% <= 1728.05 milliseconds
99% <= 1954.55 milliseconds
99.9% <= 2566.91 milliseconds
3.0.0-rc1 (alpha3 java driver)
-
total
count = 310373
mean rate = 272.25 calls/second
1-minute rate = 0.00 calls/second
5-minute rate = 45.47 calls/second
15-minute rate = 295.94 calls/second
min = 1.05 milliseconds
max = 10468.98 milliseconds
mean = 492.99 milliseconds
stddev = 510.42 milliseconds
median = 281.02 milliseconds
75% <= 696.25 milliseconds
95% <= 1434.45 milliseconds
98% <= 1820.33 milliseconds
99% <= 2080.37 milliseconds
99.9% <= 4362.08 milliseconds
{code}
> MV performance regression
> -
>
> Key: CASSANDRA-10609
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10609
> Project: Cassandra
> Issue Type: Bug
> Environment: EC2
>Reporter: Alan Boudreault
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
>
> I've noticed an important MV performance regression that has been introduced
> in 3.0.0-rc1. The issue has been introduced by CASSANDRA-9664.
> * I'm using mvbench to test with RF=3
> * I confirm it's not a driver issue.
> {code}
> EC2 RF=3 (i2.2xlarge, also tried on i2.4xlarge)
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100
> --num-artists 1 -n 50 --endpoint node1"
> 3.0.0-beta2 (alpha2 java driver)
> ---
> total
> count = 461601
> mean rate = 1923.21 calls/second
> 1-minute rate = 1937.82 calls/second
> 5-minute rate = 1424.09 calls/second
> 15-minute rate = 1058.28 calls/second
>min = 1.90 milliseconds
>max = 3707.76 milliseconds
> mean = 516.42 milliseconds
> stddev = 457.41 milliseconds
> median = 390.07 milliseconds
> 75% <= 775.95 milliseconds
> 95% <=
[jira] [Commented] (CASSANDRA-9664) Allow MV's select statements to be more complex
[
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978474#comment-14978474
]
Alan Boudreault commented on CASSANDRA-9664:
sure
> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Carl Yeksigian
>Assignee: Tyler Hobbs
> Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add
> support for a syntax which includes a {{SELECT}} statement, but only allows
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity
> with the normal CQL {{SELECT}} statement, specifically simple functions in
> the selected columns, as well as specifying a {{WHERE}} clause.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10614) AssertionError while flushing memtables
[
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979252#comment-14979252
]
Alan Boudreault commented on CASSANDRA-10614:
-
[~thobbs] I'll verify that.
It's was a one-off. It would be good to setup some regular jobs in jenkins as
you said. Adding this in the backlog.
> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
> Issue Type: Bug
> Components: Core, Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
>
> While running mvbench against a single local node, I noticed this stacktrace
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
> at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
> ~[main/:na]
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
> ~[main/:na]
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352)
> ~[main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> at
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
> ~[guava-18.0.jar:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10554) Batch that updates two or more table can produce unreadable SSTable (was: Auto Bootstraping a new node fails)
[
https://issues.apache.org/jira/browse/CASSANDRA-10554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14976730#comment-14976730
]
Alan Boudreault commented on CASSANDRA-10554:
-
I confirm that my bootstrap issue is now resolved. Thanks!
> Batch that updates two or more table can produce unreadable SSTable (was:
> Auto Bootstraping a new node fails)
> -
>
> Key: CASSANDRA-10554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10554
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: 0001-Add-debug.txt, 10554.cql, debug.log, system.log,
> test.sh
>
>
> I've been trying to add a new node in my 3.0 cluster and it seems to fail.
> All my nodes are using apache/cassandra-3.0.0 branch. At the beginning, I can
> see the following error:
> {code}
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 42 files(1910066622 bytes), sending 0 files(0 bytes)
> WARN 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.187.114 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 38 files(2323537628 bytes), sending 0 files(0 bytes)
> WARN 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:56
[jira] [Commented] (CASSANDRA-10554) Batch that updates two or more table can produce unreadable SSTable (was: Auto Bootstraping a new node fails)
[
https://issues.apache.org/jira/browse/CASSANDRA-10554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14975281#comment-14975281
]
Alan Boudreault commented on CASSANDRA-10554:
-
The patch works well. I'll rework on the bootstraping stuff tomorrow morning
> Batch that updates two or more table can produce unreadable SSTable (was:
> Auto Bootstraping a new node fails)
> -
>
> Key: CASSANDRA-10554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10554
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: 0001-Add-debug.txt, 10554.cql, debug.log, system.log,
> test.sh
>
>
> I've been trying to add a new node in my 3.0 cluster and it seems to fail.
> All my nodes are using apache/cassandra-3.0.0 branch. At the beginning, I can
> see the following error:
> {code}
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 42 files(1910066622 bytes), sending 0 files(0 bytes)
> WARN 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.187.114 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 38 files(2323537628 bytes), sending 0 files(0 bytes)
> WARN 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
>
[jira] [Commented] (CASSANDRA-10554) Auto Bootstraping a new node fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969752#comment-14969752
]
Alan Boudreault commented on CASSANDRA-10554:
-
Thanks [~yukim]. Let me know if I can help in any way.
> Auto Bootstraping a new node fails
> --
>
> Key: CASSANDRA-10554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10554
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Yuki Morishita
> Fix For: 3.0.0
>
> Attachments: 0001-Add-debug.txt, debug.log, system.log, test.sh
>
>
> I've been trying to add a new node in my 3.0 cluster and it seems to fail.
> All my nodes are using apache/cassandra-3.0.0 branch. At the beginning, I can
> see the following error:
> {code}
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 42 files(1910066622 bytes), sending 0 files(0 bytes)
> WARN 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.187.114 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 38 files(2323537628 bytes), sending 0 files(0 bytes)
> WARN 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.184.198 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 35
[jira] [Commented] (CASSANDRA-10554) Auto Bootstraping a new node fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969187#comment-14969187
]
Alan Boudreault commented on CASSANDRA-10554:
-
While trying to produce a local test case, I've noticed that I can reproduce
the error easily just by restarting a node. The error might not be related to
the bootstraping anymore at this point since I can see it with a simple node
restart. Here the error produce by my test case:
{code}
ERROR [SSTableBatchOpen:2] 2015-10-22 09:48:04,721 CassandraDaemon.java:195 -
Exception in thread Thread[SSTableBatchOpen:2,5,main]
java.lang.RuntimeException: Unknown column added_time during deserialization
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:477)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:366)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:525)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [SSTableBatchOpen:2] 2015-10-22 09:48:04,721 CassandraDaemon.java:195 -
Exception in thread Thread[SSTableBatchOpen:2,5,main]
java.lang.RuntimeException: Unknown column added_time during deserialization
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:477)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:366)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:525)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [SSTableBatchOpen:1] 2015-10-22 09:48:04,721 CassandraDaemon.java:195 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.RuntimeException: Unknown column added_time during deserialization
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:477)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:366)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:525)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR [SSTableBatchOpen:1] 2015-10-22 09:48:04,721 CassandraDaemon.java:195 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.RuntimeException: Unknown column added_time during deserialization
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:477)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:366)
~[main/:na]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:525)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
[jira] [Updated] (CASSANDRA-10554) Auto Bootstraping a new node fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10554:
Attachment: test.sh
> Auto Bootstraping a new node fails
> --
>
> Key: CASSANDRA-10554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10554
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Yuki Morishita
> Fix For: 3.0.0
>
> Attachments: 0001-Add-debug.txt, debug.log, system.log, test.sh
>
>
> I've been trying to add a new node in my 3.0 cluster and it seems to fail.
> All my nodes are using apache/cassandra-3.0.0 branch. At the beginning, I can
> see the following error:
> {code}
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 42 files(1910066622 bytes), sending 0 files(0 bytes)
> WARN 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.187.114 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 38 files(2323537628 bytes), sending 0 files(0 bytes)
> WARN 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.184.198 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 35 files(2069893759 bytes), sending 0 files(0 bytes)
> WARN 18:45:57 [Stream
[jira] [Updated] (CASSANDRA-10554) Auto Bootstraping a new node fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10554:
Attachment: system.log
Here's the system log. I will try to create a small test case that someone can
reproduce easily with CCM.
> Auto Bootstraping a new node fails
> --
>
> Key: CASSANDRA-10554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10554
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Yuki Morishita
> Fix For: 3.0.0
>
> Attachments: 0001-Add-debug.txt, system.log
>
>
> I've been trying to add a new node in my 3.0 cluster and it seems to fail.
> All my nodes are using apache/cassandra-3.0.0 branch. At the beginning, I can
> see the following error:
> {code}
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 42 files(1910066622 bytes), sending 0 files(0 bytes)
> WARN 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.187.114 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 38 files(2323537628 bytes), sending 0 files(0 bytes)
> WARN 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.184.198 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed.
[jira] [Updated] (CASSANDRA-10554) Auto Bootstraping a new node fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10554:
Attachment: debug.log
Here's the debug.log.
> Auto Bootstraping a new node fails
> --
>
> Key: CASSANDRA-10554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10554
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Yuki Morishita
> Fix For: 3.0.0
>
> Attachments: 0001-Add-debug.txt, debug.log, system.log
>
>
> I've been trying to add a new node in my 3.0 cluster and it seems to fail.
> All my nodes are using apache/cassandra-3.0.0 branch. At the beginning, I can
> see the following error:
> {code}
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 42 files(1910066622 bytes), sending 0 files(0 bytes)
> WARN 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.187.114 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 38 files(2323537628 bytes), sending 0 files(0 bytes)
> WARN 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.184.198 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 35 files(2069893759 bytes), sending 0 files(0 bytes)
> WARN 18:45:57
[jira] [Commented] (CASSANDRA-10554) Auto Bootstraping a new node fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964107#comment-14964107
]
Alan Boudreault commented on CASSANDRA-10554:
-
FYI, I'm giving it a new try with a completely new cluster. Will let you know
how it goes.
> Auto Bootstraping a new node fails
> --
>
> Key: CASSANDRA-10554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10554
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
> Fix For: 3.0.0
>
>
> I've been trying to add a new node in my 3.0 cluster and it seems to fail.
> All my nodes are using apache/cassandra-3.0.0 branch. At the beginning, I can
> see the following error:
> {code}
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 42 files(1910066622 bytes), sending 0 files(0 bytes)
> WARN 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.187.114 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 38 files(2323537628 bytes), sending 0 files(0 bytes)
> WARN 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
> occurred
> java.lang.IllegalArgumentException: Unknown type 0
> at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
> /54.210.184.198 is complete
> INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 35 files(2069893759 bytes), sending 0 files(0 bytes)
> WARN 18:45:57 [Stream
[jira] [Created] (CASSANDRA-10554) Auto Bootstraping a new node fails
Alan Boudreault created CASSANDRA-10554:
---
Summary: Auto Bootstraping a new node fails
Key: CASSANDRA-10554
URL: https://issues.apache.org/jira/browse/CASSANDRA-10554
Project: Cassandra
Issue Type: Bug
Reporter: Alan Boudreault
Fix For: 3.0.0
I've been trying to add a new node in my 3.0 cluster and it seems to fail. All
my nodes are using apache/cassandra-3.0.0 branch. At the beginning, I can see
the following error:
{code}
INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
completed. Receiving 42 files(1910066622 bytes), sending 0 files(0 bytes)
WARN 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
following error
java.lang.RuntimeException: Unknown column added_time during deserialization
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
~[main/:na]
at
org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
~[main/:na]
at
org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
~[main/:na]
at
org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
[main/:na]
at
org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
[main/:na]
at
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
[main/:na]
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
occurred
java.lang.IllegalArgumentException: Unknown type 0
at
org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
~[main/:na]
at
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
~[main/:na]
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
~[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
/54.210.187.114 is complete
INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
completed. Receiving 38 files(2323537628 bytes), sending 0 files(0 bytes)
WARN 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
following error
java.lang.RuntimeException: Unknown column added_time during deserialization
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
~[main/:na]
at
org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
~[main/:na]
at
org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
~[main/:na]
at
org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
[main/:na]
at
org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
[main/:na]
at
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
[main/:na]
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Streaming error
occurred
java.lang.IllegalArgumentException: Unknown type 0
at
org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
~[main/:na]
at
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
~[main/:na]
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
~[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Session with
/54.210.184.198 is complete
INFO 18:45:56 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
completed. Receiving 35 files(2069893759 bytes), sending 0 files(0 bytes)
WARN 18:45:57 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
following error
java.lang.AssertionError: null
at
org.apache.cassandra.db.PartitionColumns$Builder.add(PartitionColumns.java:168)
~[main/:na]
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:333)
~[main/:na]
at
org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
~[main/:na]
at
[jira] [Commented] (CASSANDRA-10554) Auto Bootstraping a new node fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964420#comment-14964420
]
Alan Boudreault commented on CASSANDRA-10554:
-
No luck. Tried to use nodetool to resume somehow the bootstraping but got:
{code}
...
[2015-10-20 02:34:50,848] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 190%)
[2015-10-20 02:34:50,848] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 191%)
[2015-10-20 02:34:50,848] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 192%)
[2015-10-20 02:34:50,849] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 193%)
[2015-10-20 02:34:50,849] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 194%)
[2015-10-20 02:34:50,849] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 195%)
[2015-10-20 02:34:50,849] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 196%)
[2015-10-20 02:34:50,849] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 197%)
[2015-10-20 02:34:50,849] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 198%)
[2015-10-20 02:34:50,849] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 200%)
[2015-10-20 02:34:50,850] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 201%)
[2015-10-20 02:34:50,850] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 202%)
[2015-10-20 02:34:50,850] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 203%)
[2015-10-20 02:34:50,850] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 204%)
[2015-10-20 02:34:50,850] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 205%)
[2015-10-20 02:34:50,850] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 206%)
[2015-10-20 02:34:50,851] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 207%)
[2015-10-20 02:34:50,851] received file
/mnt/data/cassandra/data/manual/artist_to_user-6a01f82076aa11e59305f5ca02614f5c/ma-6-big-Data.db
(progress: 208%)
[2015-10-20 02:34:50,882] session with /54.210.184.198 complete (progress: 208%)
[2015-10-20 02:34:50,883] Stream failed
{code}
> Auto Bootstraping a new node fails
> --
>
> Key: CASSANDRA-10554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10554
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
> Fix For: 3.0.0
>
>
> I've been trying to add a new node in my 3.0 cluster and it seems to fail.
> All my nodes are using apache/cassandra-3.0.0 branch. At the beginning, I can
> see the following error:
> {code}
> INFO 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a ID#0] Prepare
> completed. Receiving 42 files(1910066622 bytes), sending 0 files(0 bytes)
> WARN 18:45:55 [Stream #9f95fa90-7691-11e5-931f-5b735851f84a] Retrying for
> following error
> java.lang.RuntimeException: Unknown column added_time during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [main/:na]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [main/:na]
> at
>
[jira] [Updated] (CASSANDRA-10519) RepairException: [repair #... on .../..., (...,...]] Validation failed in /w.x.y.z
[
https://issues.apache.org/jira/browse/CASSANDRA-10519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10519:
Priority: Minor (was: Major)
> RepairException: [repair #... on .../..., (...,...]] Validation failed in
> /w.x.y.z
> --
>
> Key: CASSANDRA-10519
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10519
> Project: Cassandra
> Issue Type: Bug
> Environment: CentOS 7, JDK 8u60, Cassandra 2.2.2 (upgraded from 2.1.5)
>Reporter: Gábor Auth
>Priority: Minor
>
> Sometimes the repair fails:
> {code}
> ERROR [Repair#3:1] 2015-10-14 06:22:56,490 CassandraDaemon.java:185 -
> Exception in thread Thread[Repair#3:1,5,RMI Runtime]
> com.google.common.util.concurrent.UncheckedExecutionException:
> org.apache.cassandra.exceptions.RepairException: [repair
> #018adc70-723c-11e5-b0d8-6b2151e4d388 on keyspace/table,
> (2414492737393085601,27880539413409
> 54029]] Validation failed in /w.y.x.z
> at
> com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1387)
> ~[guava-16.0.jar:na]
> at
> com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1373)
> ~[guava-16.0.jar:na]
> at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:169)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_60]
> Caused by: org.apache.cassandra.exceptions.RepairException: [repair
> #018adc70-723c-11e5-b0d8-6b2151e4d388 on keyspace/table,
> (2414492737393085601,2788053941340954029]] Validation failed in /w.y.x.z
> at
> org.apache.cassandra.repair.ValidationTask.treeReceived(ValidationTask.java:64)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:183)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:399)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:163)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> ... 3 common frames omitted
> {code}
> And here is the w.y.x.z side:
> {code}
> ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,487
> CompactionManager.java:1053 - Cannot start multiple repair sessions over the
> same sstables
> ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,487 Validator.java:246 -
> Failed creating a merkle tree for [repair
> #018adc70-723c-11e5-b0d8-6b2151e4d388 on keyspace/table,
> (2414492737393085601,2788053941340954029]], /a.b.c.d (see log for details)
> ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,488 CassandraDaemon.java:185
> - Exception in thread Thread[ValidationExecutor:7,1,main]
> java.lang.RuntimeException: Cannot start multiple repair sessions over the
> same sstables
> at
> org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1054)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:86)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:652)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> ...
> ERROR [Reference-Reaper:1] 2015-10-14 06:23:21,439 Ref.java:187 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@74fc054a) to class
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1949471967:/home/cassandra/dsc-cassandra-2.2.2/bin/../data/data/keyspace/table-b15521b062e4bbedcdee5e027297/la-1195-big
> was not released before the reference was garbage collected
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10539) Different encodings used between nodes can cause inconsistently generated prepared statement ids
[
https://issues.apache.org/jira/browse/CASSANDRA-10539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14960913#comment-14960913
]
Alan Boudreault commented on CASSANDRA-10539:
-
[~andrew.tolbert] Is this present in all versions?
> Different encodings used between nodes can cause inconsistently generated
> prepared statement ids
> -
>
> Key: CASSANDRA-10539
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10539
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Priority: Minor
>
> [From the java-driver mailing
> list|https://groups.google.com/a/lists.datastax.com/forum/#!topic/java-driver-user/3Aa7s0u2ZrI]
> / [JAVA-955|https://datastax-oss.atlassian.net/browse/JAVA-955]
> If you have nodes in your cluster that are using a different default
> character set it's possible for nodes to generate different prepared
> statement ids for the same 'keyspace + query string' combination. I imagine
> this is not a very typical or desired configuration (thus the low severity).
> This is because
> [MD5Digest.compute(String)|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/utils/MD5Digest.java#L51-L54]
> uses
> [String.getBytes()|http://docs.oracle.com/javase/7/docs/api/java/lang/String.html#getBytes()]
> which relies on the default charset.
> In the general case this is fine, but if you use some characters in your
> query string such as
> [Character.MAX_VALUE|http://docs.oracle.com/javase/7/docs/api/java/lang/Character.html#MAX_VALUE]
> ('\u') the byte representation may vary based on the coding.
> I was able to reproduce this configuring a 2-node cluster with node1 using
> file.encoding {{UTF-8}} and node2 using file.encoding {{ISO-8859-1}}. The
> java-driver test that demonstrates this can be found
> [here|https://github.com/datastax/java-driver/blob/java955/driver-core/src/test/java/com/datastax/driver/core/RetryOnUnpreparedTest.java].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10532) Allow LWT operation on static column with only partition keys
[
https://issues.apache.org/jira/browse/CASSANDRA-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14960920#comment-14960920
]
Alan Boudreault commented on CASSANDRA-10532:
-
[~slebresne] What dev should we assign for LWT stuff? Joel maybe?
> Allow LWT operation on static column with only partition keys
> -
>
> Key: CASSANDRA-10532
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10532
> Project: Cassandra
> Issue Type: Bug
> Components: API
> Environment: C* 2.2.0
>Reporter: DOAN DuyHai
>
> Schema
> {code:sql}
> CREATE TABLE IF NOT EXISTS achilles_embedded.entity_with_static_column(
> id bigint,
> uuid uuid,
> static_col text static,
> value text,
> PRIMARY KEY(id, uuid));
> {code}
> When trying to prepare the following query
> {code:sql}
> DELETE static_col FROM achilles_embedded.entity_with_static_column WHERE
> id=:id_Eq IF static_col=:static_col;
> {code}
> I got the error *DELETE statements must restrict all PRIMARY KEY columns with
> equality relations in order to use IF conditions, but column 'uuid' is not
> restricted*
> Since the mutation only impacts the static column and the CAS check is on the
> static column, it makes sense to provide only partition key
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10519) RepairException: [repair #... on .../..., (...,...]] Validation failed in /w.x.y.z
[
https://issues.apache.org/jira/browse/CASSANDRA-10519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10519:
Reproduced In: 2.2.2
[~gabor.auth] Are you able to reproduce this easily? If so, can you attach a
test case or the procedures.
[~yukim] How critical would you qualify a repair failing that way?
> RepairException: [repair #... on .../..., (...,...]] Validation failed in
> /w.x.y.z
> --
>
> Key: CASSANDRA-10519
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10519
> Project: Cassandra
> Issue Type: Bug
> Environment: CentOS 7, JDK 8u60, Cassandra 2.2.2 (upgraded from 2.1.5)
>Reporter: Gábor Auth
>
> Sometimes the repair fails:
> {code}
> ERROR [Repair#3:1] 2015-10-14 06:22:56,490 CassandraDaemon.java:185 -
> Exception in thread Thread[Repair#3:1,5,RMI Runtime]
> com.google.common.util.concurrent.UncheckedExecutionException:
> org.apache.cassandra.exceptions.RepairException: [repair
> #018adc70-723c-11e5-b0d8-6b2151e4d388 on keyspace/table,
> (2414492737393085601,27880539413409
> 54029]] Validation failed in /w.y.x.z
> at
> com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1387)
> ~[guava-16.0.jar:na]
> at
> com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1373)
> ~[guava-16.0.jar:na]
> at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:169)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_60]
> Caused by: org.apache.cassandra.exceptions.RepairException: [repair
> #018adc70-723c-11e5-b0d8-6b2151e4d388 on keyspace/table,
> (2414492737393085601,2788053941340954029]] Validation failed in /w.y.x.z
> at
> org.apache.cassandra.repair.ValidationTask.treeReceived(ValidationTask.java:64)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:183)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:399)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:163)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> ... 3 common frames omitted
> {code}
> And here is the w.y.x.z side:
> {code}
> ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,487
> CompactionManager.java:1053 - Cannot start multiple repair sessions over the
> same sstables
> ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,487 Validator.java:246 -
> Failed creating a merkle tree for [repair
> #018adc70-723c-11e5-b0d8-6b2151e4d388 on keyspace/table,
> (2414492737393085601,2788053941340954029]], /a.b.c.d (see log for details)
> ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,488 CassandraDaemon.java:185
> - Exception in thread Thread[ValidationExecutor:7,1,main]
> java.lang.RuntimeException: Cannot start multiple repair sessions over the
> same sstables
> at
> org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1054)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:86)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at
> org.apache.cassandra.db.compaction.CompactionManager$10.call(CompactionManager.java:652)
> ~[apache-cassandra-2.2.2.jar:2.2.2]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> ...
> ERROR [Reference-Reaper:1] 2015-10-14 06:23:21,439 Ref.java:187 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@74fc054a) to class
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1949471967:/home/cassandra/dsc-cassandra-2.2.2/bin/../data/data/keyspace/table-b15521b062e4bbedcdee5e027297/la-1195-big
> was not released before the reference was garbage collected
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10492) Fix failing upgrade tests on 2.1->2.2 path
[
https://issues.apache.org/jira/browse/CASSANDRA-10492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10492:
Labels: triage (was: )
> Fix failing upgrade tests on 2.1->2.2 path
> --
>
> Key: CASSANDRA-10492
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10492
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jim Witschey
> Labels: triage
> Fix For: 2.1.x, 2.2.x
>
>
> 2 dtests for the storage engine upgrade on the 2.1->2.2 path fail:
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-21_HEAD-22_tarball/lastCompletedBuild/testReport/
> (If there are 3 failures at that link, don't worry; one was a dtest bug
> that's been fixed.)
> {{upgrade_tests/cql_tests.py:TestCQL.large_count_test}} flaps when a TRUNCATE
> operation times out:
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-21_HEAD-22_tarball/20/testReport/upgrade_tests.cql_tests/TestCQL/large_count_test/history/
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions}}
> also fails on this upgrade path:
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-21_HEAD-22_tarball/15/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_failure_threshold_deletions/history/
> These are in the same ticket for the moment, but if one becomes a longer
> project, it should be broken out into a separate ticket.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10510) Compacted SSTables failing to get removed, overflowing disk
[ https://issues.apache.org/jira/browse/CASSANDRA-10510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14955393#comment-14955393 ] Alan Boudreault commented on CASSANDRA-10510: - Hello [~Bj0rn], Unfortunately, Cassandra 2.0 is EOL (End of Life). Please reopen this ticket if you can reproduce the bug in 2.1+. > Compacted SSTables failing to get removed, overflowing disk > --- > > Key: CASSANDRA-10510 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10510 > Project: Cassandra > Issue Type: Bug >Reporter: Björn Hegerfors > Attachments: nonReleasedSSTables.txt > > > Short version: it appears that if the resulting SSTable of a compaction > enters another compaction soon after, the SSTables participating in the > former compaction don't get deleted from disk until Cassandra is restarted. > We have run into a big problem after applying CASSANDRA-10276 and > CASSANDRA-10280, backported to 2.0.14. But the bug we're seeing is not > introduced by these patches, it has just made itself very apparent and > harmful. > Here's what has happened. We had repair running on our table that is a time > series and uses DTCS. The ring was split into 5016 small ranges being > repaired one after the other (using parallel repair, i.e. not snapshot > repair). This causes a flood of tiny SSTables to get streamed into all nodes > (we don't use vnodes), with timestamp ranges similar to existing SSTables on > disk. The problem with that is the sheer number of SSTables, disk usage is > not affected. This has been reported before, see CASSANDRA-9644. These > SSTables are streamed continuously for up to a couple of days. > The patches were applied to fix the problem of ending up with tens of > thousands of SSTables that would never get touched by DTCS. But now that DTCS > does touch them, we have run into a new problem instead. While disk usage was > in the 25-30% neighborhood before repairs began, disk usage started growing > fast when these continuous streams started coming in. Eventually, a couple of > nodes ran out of disk, which led us to stop all the repairing on the cluster. > This didn't reduce the disk usage. Compactions were of course very active. > More than doubling disk usage should not be possible, regardless of the > choices your compaction strategy makes. And we were not getting magnitudes of > data streamed in. Large quantities of SSTables, yes, but this was the nodes > creating more data out of thin air. > We have a tool to show timestamp and size metadata of SSTables. What we > found, looking at all non-tmp data files, was something akin to duplicates of > almost all the largest SSTables. Not quite exact replicas, but there were > these multi-gigabyte SSTables covering exactly the same range of timestamps > and differing in size by mere kilobytes. There were typically 3 of each of > the largest SSTables, sometimes even more. > Here's what I suspect: DTCS is the only compaction strategy that would > commonly finish compacting a really large SSTable and on the very next run of > the compaction strategy nominate the result for yet another compaction. Even > together with tiny SSTables, which certainly happens in our scenario. > Potentially, the large SSTable that participated in the first compaction > might even get nominated again by DTCS, if for some reason it can be returned > by getUncompactingSSTables. > Whatever the reason, I have collected evidence showing that these large > "duplicate" SSTables are of the same "lineage". Only one should remain on > disk: the latest one. The older ones have already been compacted, resulting > in the newer ones. But for some reason, they never got deleted from disk. And > this was really harmful when combining DTCS with continuously streaming in > tiny SSTables. The same but worse would happen without the patches and > uncapped max_sstable_age_days. > Attached is one occurrence of 3 duplicated SSTables, their metadata and log > lines about their compactions. You can see how similar they were to each > other. SSTable generations 374277, 374249, 373702 (the large one), 374305, > 374231 and 374333 completed compaction at 04:05:26,878, yet they were all > still on disk over 6 hours later. At 04:05:26,898 the result, 374373, entered > another compaction with 375174. They also stayed around after that compaction > finished. Literally all SSTables named in these log lines were still on disk > when I checked! Only one should have remained: 375189. > Now this was just one random example from the data I collected. This happened > everywhere. Some SSTables should probably have been deleted a day before. > However, once we restarted the nodes, all of the duplicates were suddenly > gone! -- This message was sent by Atlassian JIRA
[jira] [Updated] (CASSANDRA-10502) Cassandra query degradation with high frequency updated tables
[
https://issues.apache.org/jira/browse/CASSANDRA-10502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10502:
Reproduced In: 2.2.x
> Cassandra query degradation with high frequency updated tables
> --
>
> Key: CASSANDRA-10502
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10502
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Dodong Juan
>
> Hi,
> So we are developing a system that computes profile of things that it
> observes. The observation comes in form of events. Each thing that it
> observe has an id and each thing has a set of subthings in it which has
> measurement of some kind. Roughly there are about 500 subthings within each
> thing. We receive events containing measurements of these 500 subthings every
> 10 seconds or so.
> So as we receive events, we read the old profile value, calculate the new
> profile based on the new value and save it back.
> One of the things we observe are the processes running on the server.
> We use the following schema to hold the profile.
> {noformat}
> CREATE TABLE processinfometric_profile (
> profilecontext text,
> id text,
> month text,
> day text,
> hour text,
> minute text,
> command text,
> cpu map,
> majorfaults map,
> minorfaults map,
> nice map,
> pagefaults map,
> pid map,
> ppid map,
> priority map,
> resident map,
> rss map,
> sharesize map,
> size map,
> starttime map,
> state map,
> threads map,
> user map,
> vsize map,
> PRIMARY KEY ((profilecontext, id, month, day, hour, minute), command)
> ) WITH CLUSTERING ORDER BY (command ASC)
> AND bloom_filter_fp_chance = 0.1
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {noformat}
> This profile will then be use for certain analytics that can use in the
> context of the ‘thing’ or in the context of specific thing and subthing.
> A profile can be defined as monthly, daily, hourly. So in case of monthly the
> month will be set to the current month (i.e. ‘Oct’) and the day and hour will
> be set to empty ‘’ string.
> The problem that we have observed is that over time (actually in just a
> matter of hours) we will see a huge degradation of query response for the
> monthly profile. At the start it will be respinding in 10-100 ms and after a
> couple of hours it will go to 2000-3000 ms . If you leave it for a couple of
> days you will start experiencing readtimeouts . The query is basically just :
> {noformat}
> select * from myprofile where id=‘1’ and month=‘Oct’ and day=‘’ and hour=‘'
> and minute=''
> {noformat}
> This will have only about 500 rows or so.
> We were using Cassandra 2.2.1 , but upgraded to 2.2.2 to see if it fixed the
> issue to no avail. And since this is a test, we are running on a single node.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10502) Cassandra query degradation with high frequency updated tables
[
https://issues.apache.org/jira/browse/CASSANDRA-10502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10502:
Labels: perfomance query triage (was: perfomance query)
> Cassandra query degradation with high frequency updated tables
> --
>
> Key: CASSANDRA-10502
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10502
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Dodong Juan
>Priority: Minor
> Labels: perfomance, query, triage
>
> Hi,
> So we are developing a system that computes profile of things that it
> observes. The observation comes in form of events. Each thing that it
> observe has an id and each thing has a set of subthings in it which has
> measurement of some kind. Roughly there are about 500 subthings within each
> thing. We receive events containing measurements of these 500 subthings every
> 10 seconds or so.
> So as we receive events, we read the old profile value, calculate the new
> profile based on the new value and save it back.
> One of the things we observe are the processes running on the server.
> We use the following schema to hold the profile.
> {noformat}
> CREATE TABLE processinfometric_profile (
> profilecontext text,
> id text,
> month text,
> day text,
> hour text,
> minute text,
> command text,
> cpu map,
> majorfaults map,
> minorfaults map,
> nice map,
> pagefaults map,
> pid map,
> ppid map,
> priority map,
> resident map,
> rss map,
> sharesize map,
> size map,
> starttime map,
> state map,
> threads map,
> user map,
> vsize map,
> PRIMARY KEY ((profilecontext, id, month, day, hour, minute), command)
> ) WITH CLUSTERING ORDER BY (command ASC)
> AND bloom_filter_fp_chance = 0.1
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {noformat}
> This profile will then be use for certain analytics that can use in the
> context of the ‘thing’ or in the context of specific thing and subthing.
> A profile can be defined as monthly, daily, hourly. So in case of monthly the
> month will be set to the current month (i.e. ‘Oct’) and the day and hour will
> be set to empty ‘’ string.
> The problem that we have observed is that over time (actually in just a
> matter of hours) we will see a huge degradation of query response for the
> monthly profile. At the start it will be respinding in 10-100 ms and after a
> couple of hours it will go to 2000-3000 ms . If you leave it for a couple of
> days you will start experiencing readtimeouts . The query is basically just :
> {noformat}
> select * from myprofile where id=‘1’ and month=‘Oct’ and day=‘’ and hour=‘'
> and minute=''
> {noformat}
> This will have only about 500 rows or so.
> We were using Cassandra 2.2.1 , but upgraded to 2.2.2 to see if it fixed the
> issue to no avail. And since this is a test, we are running on a single node.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-10510) Compacted SSTables failing to get removed, overflowing disk
[ https://issues.apache.org/jira/browse/CASSANDRA-10510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault resolved CASSANDRA-10510. - Resolution: Won't Fix > Compacted SSTables failing to get removed, overflowing disk > --- > > Key: CASSANDRA-10510 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10510 > Project: Cassandra > Issue Type: Bug >Reporter: Björn Hegerfors > Attachments: nonReleasedSSTables.txt > > > Short version: it appears that if the resulting SSTable of a compaction > enters another compaction soon after, the SSTables participating in the > former compaction don't get deleted from disk until Cassandra is restarted. > We have run into a big problem after applying CASSANDRA-10276 and > CASSANDRA-10280, backported to 2.0.14. But the bug we're seeing is not > introduced by these patches, it has just made itself very apparent and > harmful. > Here's what has happened. We had repair running on our table that is a time > series and uses DTCS. The ring was split into 5016 small ranges being > repaired one after the other (using parallel repair, i.e. not snapshot > repair). This causes a flood of tiny SSTables to get streamed into all nodes > (we don't use vnodes), with timestamp ranges similar to existing SSTables on > disk. The problem with that is the sheer number of SSTables, disk usage is > not affected. This has been reported before, see CASSANDRA-9644. These > SSTables are streamed continuously for up to a couple of days. > The patches were applied to fix the problem of ending up with tens of > thousands of SSTables that would never get touched by DTCS. But now that DTCS > does touch them, we have run into a new problem instead. While disk usage was > in the 25-30% neighborhood before repairs began, disk usage started growing > fast when these continuous streams started coming in. Eventually, a couple of > nodes ran out of disk, which led us to stop all the repairing on the cluster. > This didn't reduce the disk usage. Compactions were of course very active. > More than doubling disk usage should not be possible, regardless of the > choices your compaction strategy makes. And we were not getting magnitudes of > data streamed in. Large quantities of SSTables, yes, but this was the nodes > creating more data out of thin air. > We have a tool to show timestamp and size metadata of SSTables. What we > found, looking at all non-tmp data files, was something akin to duplicates of > almost all the largest SSTables. Not quite exact replicas, but there were > these multi-gigabyte SSTables covering exactly the same range of timestamps > and differing in size by mere kilobytes. There were typically 3 of each of > the largest SSTables, sometimes even more. > Here's what I suspect: DTCS is the only compaction strategy that would > commonly finish compacting a really large SSTable and on the very next run of > the compaction strategy nominate the result for yet another compaction. Even > together with tiny SSTables, which certainly happens in our scenario. > Potentially, the large SSTable that participated in the first compaction > might even get nominated again by DTCS, if for some reason it can be returned > by getUncompactingSSTables. > Whatever the reason, I have collected evidence showing that these large > "duplicate" SSTables are of the same "lineage". Only one should remain on > disk: the latest one. The older ones have already been compacted, resulting > in the newer ones. But for some reason, they never got deleted from disk. And > this was really harmful when combining DTCS with continuously streaming in > tiny SSTables. The same but worse would happen without the patches and > uncapped max_sstable_age_days. > Attached is one occurrence of 3 duplicated SSTables, their metadata and log > lines about their compactions. You can see how similar they were to each > other. SSTable generations 374277, 374249, 373702 (the large one), 374305, > 374231 and 374333 completed compaction at 04:05:26,878, yet they were all > still on disk over 6 hours later. At 04:05:26,898 the result, 374373, entered > another compaction with 375174. They also stayed around after that compaction > finished. Literally all SSTables named in these log lines were still on disk > when I checked! Only one should have remained: 375189. > Now this was just one random example from the data I collected. This happened > everywhere. Some SSTables should probably have been deleted a day before. > However, once we restarted the nodes, all of the duplicates were suddenly > gone! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10502) Cassandra query degradation with high frequency updated tables
[
https://issues.apache.org/jira/browse/CASSANDRA-10502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10502:
Labels: perfomance query (was: )
Priority: Minor (was: Major)
> Cassandra query degradation with high frequency updated tables
> --
>
> Key: CASSANDRA-10502
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10502
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Dodong Juan
>Priority: Minor
> Labels: perfomance, query
>
> Hi,
> So we are developing a system that computes profile of things that it
> observes. The observation comes in form of events. Each thing that it
> observe has an id and each thing has a set of subthings in it which has
> measurement of some kind. Roughly there are about 500 subthings within each
> thing. We receive events containing measurements of these 500 subthings every
> 10 seconds or so.
> So as we receive events, we read the old profile value, calculate the new
> profile based on the new value and save it back.
> One of the things we observe are the processes running on the server.
> We use the following schema to hold the profile.
> {noformat}
> CREATE TABLE processinfometric_profile (
> profilecontext text,
> id text,
> month text,
> day text,
> hour text,
> minute text,
> command text,
> cpu map,
> majorfaults map,
> minorfaults map,
> nice map,
> pagefaults map,
> pid map,
> ppid map,
> priority map,
> resident map,
> rss map,
> sharesize map,
> size map,
> starttime map,
> state map,
> threads map,
> user map,
> vsize map,
> PRIMARY KEY ((profilecontext, id, month, day, hour, minute), command)
> ) WITH CLUSTERING ORDER BY (command ASC)
> AND bloom_filter_fp_chance = 0.1
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {noformat}
> This profile will then be use for certain analytics that can use in the
> context of the ‘thing’ or in the context of specific thing and subthing.
> A profile can be defined as monthly, daily, hourly. So in case of monthly the
> month will be set to the current month (i.e. ‘Oct’) and the day and hour will
> be set to empty ‘’ string.
> The problem that we have observed is that over time (actually in just a
> matter of hours) we will see a huge degradation of query response for the
> monthly profile. At the start it will be respinding in 10-100 ms and after a
> couple of hours it will go to 2000-3000 ms . If you leave it for a couple of
> days you will start experiencing readtimeouts . The query is basically just :
> {noformat}
> select * from myprofile where id=‘1’ and month=‘Oct’ and day=‘’ and hour=‘'
> and minute=''
> {noformat}
> This will have only about 500 rows or so.
> We were using Cassandra 2.2.1 , but upgraded to 2.2.2 to see if it fixed the
> issue to no avail. And since this is a test, we are running on a single node.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10501) Failure to start up Cassandra when temporary compaction files are not all renamed after kill/crash (FSReadError)
[ https://issues.apache.org/jira/browse/CASSANDRA-10501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-10501: Assignee: Marcus Eriksson Reproduced In: 2.1.6 Labels: compaction triage (was: ) > Failure to start up Cassandra when temporary compaction files are not all > renamed after kill/crash (FSReadError) > > > Key: CASSANDRA-10501 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10501 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Cassandra 2.1.6 > Redhat Linux >Reporter: Mathieu Roy >Assignee: Marcus Eriksson > Labels: compaction, triage > > We have seen an issue intermittently but repeatedly over the last few months > where, after exiting the Cassandra process, it fails to start with an > FSReadError (stack trace below). The FSReadError refers to a 'statistics' > file for a that doesn't exist, though a corresponding temporary file does > exist (eg. there is no > /media/data/cassandraDB/data/clusteradmin/singleton_token-01a92ed069b511e59b2c53679a538c14/clusteradmin-singleton_token-ka-9-Statistics.db > file, but there is a > /media/data/cassandraDB/data/clusteradmin/singleton_token-01a92ed069b511e59b2c53679a538c14/clusteradmin-singleton_token-tmp-ka-9-Statistics.db > file.) > We tracked down the issue to the fact that the process exited with leftover > compactions and some of the 'tmp' files for the SSTable had been renamed to > final files, but not all of them - the issue happens if the 'Statistics' file > is not renamed but others are. The scenario we've seen on the last two > occurrences involves the 'CompressionInfo' file being a final file while all > other files for the SSTable generation were left with 'tmp' names. > When this occurs, Cassandra cannot start until the file issue is resolved; > we've worked around it by deleting the SSTable files from the same > generation, both final and tmp, which at least allows Cassandra to start. > Renaming all files to either tmp or final names would also work. > We've done some debugging in Cassandra and have been unable to cause the > issue without renaming the files manually. The rename code at > SSTableWriter.rename() looks like it could result in this if the process > exits in the middle of the rename, but in every occurrence we've debugged > through, the Set of components is ordered and Statistics is the first file > renamed. > However the comments in SSTableWriter.rename() suggest that the 'Data' file > is meant to be used as meaning the files were completely renamed. The method > ColumnFamilyStore. removeUnfinishedCompactionLeftovers(), however, will > proceed assuming the compaction is complete if any of the component files has > a final name, and will skip temporary files when reading the list. If the > 'Statistics' file is temporary then it won't be read, and the defaults does > not include a list of ancestors, leading to the NullPointerException. > It appears that ColumnFamilyStore. removeUnfinishedCompactionLeftovers() > should perhaps either ensure that all 'tmp' files are properly renamed before > it uses them, or skip SSTable files that don't have either the 'Data' or > 'Statistics' file in final form. > Stack trace: > FSReadError in Failed to remove unfinished compaction leftovers (file: > /media/data/cassandraDB/data/clusteradmin/singleton_token-01a92ed069b511e59b2c53679a538c14/clusteradmin-singleton_token-ka-9-Statistics.db). > See log for details. > at > org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:617) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:302) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:536) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:625) > Caused by: java.lang.NullPointerException > at > org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:609) > ... 3 more > Exception encountered during startup: java.lang.NullPointerException -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10515) Commit logs back up with move to 2.1.10
[
https://issues.apache.org/jira/browse/CASSANDRA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10515:
Labels: commitlog triage (was: )
> Commit logs back up with move to 2.1.10
> ---
>
> Key: CASSANDRA-10515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10515
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: redhat 6.5, cassandra 2.1.10
>Reporter: Jeff Griffith
>Assignee: Branimir Lambov
>Priority: Critical
> Labels: commitlog, triage
> Attachments: CommitLogProblem.jpg
>
>
> After upgrading from cassandra 2.0.x to 2.1.10, we began seeing problems
> where some nodes break the 12G commit log max we configured and go as high as
> 65G or more before it restarts. Once it reaches the state of more than 12G
> commit log files, "nodetool compactionstats" hangs. Eventually C* restarts
> without errors (not sure yet whether it is crashing but I'm checking into it)
> and the cleanup occurs and the commit logs shrink back down again. Here is
> the nodetool compactionstats immediately after restart.
> {code}
> jgriffith@prod1xc1.c2.bf1:~$ ndc
> pending tasks: 2185
>compaction type keyspace table completed
> totalunit progress
> Compaction SyncCore *cf1* 61251208033
> 170643574558 bytes 35.89%
> Compaction SyncCore *cf2* 19262483904
> 19266079916 bytes 99.98%
> Compaction SyncCore *cf3*6592197093
> 6592316682 bytes100.00%
> Compaction SyncCore *cf4*3411039555
> 3411039557 bytes100.00%
> Compaction SyncCore *cf5*2879241009
> 2879487621 bytes 99.99%
> Compaction SyncCore *cf6* 21252493623
> 21252635196 bytes100.00%
> Compaction SyncCore *cf7* 81009853587
> 81009854438 bytes100.00%
> Compaction SyncCore *cf8*3005734580
> 3005768582 bytes100.00%
> Active compaction remaining time :n/a
> {code}
> I was also doing periodic "nodetool tpstats" which were working but not being
> logged in system.log on the StatusLogger thread until after the compaction
> started working again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10490) DTCS historic compaction, possibly with major compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-10490: Labels: compaction (was: ) > DTCS historic compaction, possibly with major compaction > > > Key: CASSANDRA-10490 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10490 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Shook > Labels: compaction > Fix For: 3.x, 2.2.x > > > Presently, it's simply painful to run a major compaction with DTCS. It > doesn't really serve a useful purpose. Instead, a DTCS major compaction > should allow for a DTCS-style compaction to go back before > max_sstable_age_days. We can call this a historic compaction, for lack of a > better term. > Such a compaction should not take precedence over normal compaction work, but > should be considered a background task. By default there should be a cap on > the number of these tasks running. It would be nice to have a separate > "max_historic_compaction_tasks" and possibly a > "max_historic_compaction_throughput" in the compaction settings to allow for > separate throttles on this. I would set these at 1 and 20% of the usual > compaction throughput if they aren't set explicitly. > It may also be desirable to allow historic compaction to run apart from > running a major compaction, and to simply disable major compaction altogether > for DTCS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10515) Commit logs back up with move to 2.1.10
[
https://issues.apache.org/jira/browse/CASSANDRA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10515:
Assignee: Branimir Lambov
Reproduced In: 2.1.10
> Commit logs back up with move to 2.1.10
> ---
>
> Key: CASSANDRA-10515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10515
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: redhat 6.5, cassandra 2.1.10
>Reporter: Jeff Griffith
>Assignee: Branimir Lambov
>Priority: Critical
> Attachments: CommitLogProblem.jpg
>
>
> After upgrading from cassandra 2.0.x to 2.1.10, we began seeing problems
> where some nodes break the 12G commit log max we configured and go as high as
> 65G or more before it restarts. Once it reaches the state of more than 12G
> commit log files, "nodetool compactionstats" hangs. Eventually C* restarts
> without errors (not sure yet whether it is crashing but I'm checking into it)
> and the cleanup occurs and the commit logs shrink back down again. Here is
> the nodetool compactionstats immediately after restart.
> {code}
> jgriffith@prod1xc1.c2.bf1:~$ ndc
> pending tasks: 2185
>compaction type keyspace table completed
> totalunit progress
> Compaction SyncCore *cf1* 61251208033
> 170643574558 bytes 35.89%
> Compaction SyncCore *cf2* 19262483904
> 19266079916 bytes 99.98%
> Compaction SyncCore *cf3*6592197093
> 6592316682 bytes100.00%
> Compaction SyncCore *cf4*3411039555
> 3411039557 bytes100.00%
> Compaction SyncCore *cf5*2879241009
> 2879487621 bytes 99.99%
> Compaction SyncCore *cf6* 21252493623
> 21252635196 bytes100.00%
> Compaction SyncCore *cf7* 81009853587
> 81009854438 bytes100.00%
> Compaction SyncCore *cf8*3005734580
> 3005768582 bytes100.00%
> Active compaction remaining time :n/a
> {code}
> I was also doing periodic "nodetool tpstats" which were working but not being
> logged in system.log on the StatusLogger thread until after the compaction
> started working again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10490) DTCS historic compaction, possibly with major compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-10490: Labels: compaction triage (was: compaction) > DTCS historic compaction, possibly with major compaction > > > Key: CASSANDRA-10490 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10490 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Shook > Labels: compaction, triage > Fix For: 3.x, 2.2.x > > > Presently, it's simply painful to run a major compaction with DTCS. It > doesn't really serve a useful purpose. Instead, a DTCS major compaction > should allow for a DTCS-style compaction to go back before > max_sstable_age_days. We can call this a historic compaction, for lack of a > better term. > Such a compaction should not take precedence over normal compaction work, but > should be considered a background task. By default there should be a cap on > the number of these tasks running. It would be nice to have a separate > "max_historic_compaction_tasks" and possibly a > "max_historic_compaction_throughput" in the compaction settings to allow for > separate throttles on this. I would set these at 1 and 20% of the usual > compaction throughput if they aren't set explicitly. > It may also be desirable to allow historic compaction to run apart from > running a major compaction, and to simply disable major compaction altogether > for DTCS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10480) CFS.truncateBlocking relies on sleep
[
https://issues.apache.org/jira/browse/CASSANDRA-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10480:
Labels: triage windows (was: windows)
> CFS.truncateBlocking relies on sleep
>
>
> Key: CASSANDRA-10480
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10480
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
>Priority: Minor
> Labels: triage, windows
>
> {{ColumnFamilyStore.truncateBlocking}} uses
> {{Uninterruptibles.sleepUninterruptibly(1, TimeUnit.MILLISECONDS);}} to
> ensure that _truncatedAt_ does not compete with a _deletedAt_.
> {{System.currentTimeMillis()}} _usually_ (it's not guaranteed) works as
> expected on Linux and OSX - but especially on Windows it may stall (update
> the {{System.currentTimeMillis()}} at non-regular intervals).
> I don't think this is a major issue as it just affects the truncate
> operation, which is a DDL operation, which should never be used in
> application code (IMO).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10490) DTCS historic compaction, possibly with major compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-10490: Issue Type: Improvement (was: Bug) > DTCS historic compaction, possibly with major compaction > > > Key: CASSANDRA-10490 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10490 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Shook > Fix For: 3.x, 2.2.x > > > Presently, it's simply painful to run a major compaction with DTCS. It > doesn't really serve a useful purpose. Instead, a DTCS major compaction > should allow for a DTCS-style compaction to go back before > max_sstable_age_days. We can call this a historic compaction, for lack of a > better term. > Such a compaction should not take precedence over normal compaction work, but > should be considered a background task. By default there should be a cap on > the number of these tasks running. It would be nice to have a separate > "max_historic_compaction_tasks" and possibly a > "max_historic_compaction_throughput" in the compaction settings to allow for > separate throttles on this. I would set these at 1 and 20% of the usual > compaction throughput if they aren't set explicitly. > It may also be desirable to allow historic compaction to run apart from > running a major compaction, and to simply disable major compaction altogether > for DTCS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10407) Benchmark and evaluate CASSANDRA-8894 improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14941719#comment-14941719 ] Alan Boudreault commented on CASSANDRA-10407: - [~Stefania] Here are all the graphs from our tests yesterday. I tested the 3.0.0-rc1 this morning. As you already know, I've not been able to clearly demonstrate the improvement. All tests have been run on a single node cluster (ec2, i2.4xlarge) Revisions: * without path: a59be2693e33e899264bc29b37b669d4bb645a74 * with path: 6422e34769e42b4eef5d9c073301c9791e9ca8b2 without path - 64k readahead: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-readahead-64k.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 without patch - readhead disabled: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-no-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 with path - 64k readahead: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-with-path-64k-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 witht path - readahead disabled: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-with-path-no-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 3.0.0-rc1 64k readahead: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-3.0.0-rc1-64k-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 3.0.0-rc1 readahead disabled: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-3.0.0-rc1-no-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 > Benchmark and evaluate CASSANDRA-8894 improvements > -- > > Key: CASSANDRA-10407 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10407 > Project: Cassandra > Issue Type: Test >Reporter: Aleksey Yeschenko >Assignee: Alan Boudreault > Fix For: 3.0.0 rc2 > > > The original ticket (CASSANDRA-8894) was committed to 3.0 alpha1 two months > ago. We need to get proper performance tests before GA. > See [~benedict]'s > [comment|https://issues.apache.org/jira/browse/CASSANDRA-8894?focusedCommentId=14631203=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14631203] > for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10407) Benchmark and evaluate CASSANDRA-8894 improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-10407: Attachment: flight-recordings.tar.gz Here are the flight recordings [^flight-recordings.tar.gz] * with path revision - readahead disabled - disk access standard * cassandra-3.0.0-rc1 - readahead disabled - disk access standard > Benchmark and evaluate CASSANDRA-8894 improvements > -- > > Key: CASSANDRA-10407 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10407 > Project: Cassandra > Issue Type: Test >Reporter: Aleksey Yeschenko >Assignee: Alan Boudreault > Fix For: 3.0.0 rc2 > > Attachments: flight-recordings.tar.gz > > > The original ticket (CASSANDRA-8894) was committed to 3.0 alpha1 two months > ago. We need to get proper performance tests before GA. > See [~benedict]'s > [comment|https://issues.apache.org/jira/browse/CASSANDRA-8894?focusedCommentId=14631203=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14631203] > for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10407) Benchmark and evaluate CASSANDRA-8894 improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-10407: Attachment: 8894_tiny.yaml test-without-8894-tiny.json test-with-8894-tiny.json For the records, I'm attaching the stress profile and the cstar_perf job definitions used. The disk readahead has to be set in the ~/.cstar_perf/cluster_config.json file. > Benchmark and evaluate CASSANDRA-8894 improvements > -- > > Key: CASSANDRA-10407 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10407 > Project: Cassandra > Issue Type: Test >Reporter: Aleksey Yeschenko >Assignee: Alan Boudreault > Fix For: 3.0.0 rc2 > > Attachments: 8894_tiny.yaml, flight-recordings.tar.gz, > test-with-8894-tiny.json, test-without-8894-tiny.json > > > The original ticket (CASSANDRA-8894) was committed to 3.0 alpha1 two months > ago. We need to get proper performance tests before GA. > See [~benedict]'s > [comment|https://issues.apache.org/jira/browse/CASSANDRA-8894?focusedCommentId=14631203=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14631203] > for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10407) Benchmark and evaluate CASSANDRA-8894 improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14941719#comment-14941719 ] Alan Boudreault edited comment on CASSANDRA-10407 at 10/2/15 9:48 PM: -- [~Stefania] Here are all the graphs from our tests yesterday. I tested the 3.0.0-rc1 this morning. As you already know, I've not been able to clearly demonstrate the improvement. All tests have been run on a single node cluster (ec2, i2.4xlarge) using ONLY non-striped ssd disk (so the other disks had been removed from the data_directory setting of cstar_perf) Revisions: * without path: a59be2693e33e899264bc29b37b669d4bb645a74 * with path: 6422e34769e42b4eef5d9c073301c9791e9ca8b2 without path - 64k readahead: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-readahead-64k.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 without patch - readhead disabled: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-no-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 with path - 64k readahead: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-with-path-64k-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 witht path - readahead disabled: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-with-path-no-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 3.0.0-rc1 64k readahead: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-3.0.0-rc1-64k-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 3.0.0-rc1 readahead disabled: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-3.0.0-rc1-no-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 was (Author: aboudreault): [~Stefania] Here are all the graphs from our tests yesterday. I tested the 3.0.0-rc1 this morning. As you already know, I've not been able to clearly demonstrate the improvement. All tests have been run on a single node cluster (ec2, i2.4xlarge) Revisions: * without path: a59be2693e33e899264bc29b37b669d4bb645a74 * with path: 6422e34769e42b4eef5d9c073301c9791e9ca8b2 without path - 64k readahead: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-readahead-64k.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 without patch - readhead disabled: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-no-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 with path - 64k readahead: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-with-path-64k-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 witht path - readahead disabled: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-with-path-no-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 3.0.0-rc1 64k readahead: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-3.0.0-rc1-64k-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 3.0.0-rc1 readahead disabled: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=10407%2Ftiny-3.0.0-rc1-no-readahead.log=op_rate=2_user=1_aggregates=true=0=209.11=0=74949.6 > Benchmark and evaluate CASSANDRA-8894 improvements > -- > > Key: CASSANDRA-10407 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10407 > Project: Cassandra > Issue Type: Test >Reporter: Aleksey Yeschenko >Assignee: Alan Boudreault > Fix For: 3.0.0 rc2 > > Attachments: 8894_tiny.yaml, flight-recordings.tar.gz, > test-with-8894-tiny.json, test-without-8894-tiny.json > > > The original ticket (CASSANDRA-8894) was committed to 3.0 alpha1 two months > ago. We need to get proper performance tests before GA. > See [~benedict]'s > [comment|https://issues.apache.org/jira/browse/CASSANDRA-8894?focusedCommentId=14631203=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14631203] > for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10407) Benchmark and evaluate CASSANDRA-8894 improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-10407: Tester: Alan Boudreault > Benchmark and evaluate CASSANDRA-8894 improvements > -- > > Key: CASSANDRA-10407 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10407 > Project: Cassandra > Issue Type: Test >Reporter: Aleksey Yeschenko >Assignee: Alan Boudreault > Fix For: 3.0.0 rc2 > > > The original ticket (CASSANDRA-8894) was committed to 3.0 alpha1 two months > ago. We need to get proper performance tests before GA. > See [~benedict]'s > [comment|https://issues.apache.org/jira/browse/CASSANDRA-8894?focusedCommentId=14631203=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14631203] > for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10407) Benchmark and evaluate CASSANDRA-8894 improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14935962#comment-14935962
]
Alan Boudreault commented on CASSANDRA-10407:
-
[~benedict] What yaml profile should I benchmark? All attached in #8894?
Is there any other setting required than the following?
{code}
disk_access_mode: standard
{code}
I will use the same commands than Stefania mentioned (with 65M as n parameter):
{code}
user profile=https://dl.dropboxusercontent.com/u/15683245/8894_tiny.yaml
ops\(insert=1,\) n=10 -rate threads=50
user profile=https://dl.dropboxusercontent.com/u/15683245/8894_tiny.yaml
ops\(singleblob=1,\) n=10 -rate threads=50
{code}
so will benchmark both *insert* and *singleblob (read)* operations. Anything
else?
> Benchmark and evaluate CASSANDRA-8894 improvements
> --
>
> Key: CASSANDRA-10407
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10407
> Project: Cassandra
> Issue Type: Test
>Reporter: Aleksey Yeschenko
>Assignee: Alan Boudreault
> Fix For: 3.0.0 rc2
>
>
> The original ticket (CASSANDRA-8894) was committed to 3.0 alpha1 two months
> ago. We need to get proper performance tests before GA.
> See [~benedict]'s
> [comment|https://issues.apache.org/jira/browse/CASSANDRA-8894?focusedCommentId=14631203=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14631203]
> for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10407) Benchmark and evaluate CASSANDRA-8894 improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936050#comment-14936050 ] Alan Boudreault commented on CASSANDRA-10407: - [~enigmacurry] Is that already done? > Benchmark and evaluate CASSANDRA-8894 improvements > -- > > Key: CASSANDRA-10407 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10407 > Project: Cassandra > Issue Type: Test >Reporter: Aleksey Yeschenko >Assignee: Alan Boudreault > Fix For: 3.0.0 rc2 > > > The original ticket (CASSANDRA-8894) was committed to 3.0 alpha1 two months > ago. We need to get proper performance tests before GA. > See [~benedict]'s > [comment|https://issues.apache.org/jira/browse/CASSANDRA-8894?focusedCommentId=14631203=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14631203] > for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
