[jira] [Commented] (CASSANDRA-13935) Indexes creation should have IF EXISTS on its String representation
[
https://issues.apache.org/jira/browse/CASSANDRA-13935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509089#comment-16509089
]
Kurt Greaves commented on CASSANDRA-13935:
--
[~Kani] that was on the trunk unit tests I had run above, but it appears flaky
after multiple re-runs. I've rebased the trunk patch and kicked off the tests
again
[utests|https://circleci.com/gh/kgreav/workflows/cassandra/tree/13935-trunk]
> Indexes creation should have IF EXISTS on its String representation
> ---
>
> Key: CASSANDRA-13935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13935
> Project: Cassandra
> Issue Type: Bug
> Components: CQL, Secondary Indexes
> Environment: Ubuntu 16.04.2 LTS
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
>Reporter: Javier Canillas
>Assignee: Javier Canillas
>Priority: Trivial
> Fix For: 3.11.x
>
> Attachments: 13935-3.0.txt, 13935-3.11.txt, 13935-trunk.txt
>
>
> I came across something that bothers me a lot. I'm using snapshots to backup
> data from my Cassandra cluster in case something really bad happens (like
> dropping a table or a keyspace).
> Exercising the recovery actions from those backups, I discover that the
> schema put on the file "schema.cql" as a result of the snapshot has the
> "CREATE IF NOT EXISTS" for the table, but not for the indexes.
> When restoring from snapshots, and relying on the execution of these schemas
> to build up the table structure, everything seems fine for tables without
> secondary indexes, but for the ones that make use of them, the execution of
> these statements fail miserably.
> Here I paste a generated schema.cql content for a table with indexes:
> CREATE TABLE IF NOT EXISTS keyspace1.table1 (
> id text PRIMARY KEY,
> content text,
> last_update_date date,
> last_update_date_time timestamp)
> WITH ID = f1045fc0-2f59-11e7-95ec-295c3c064920
> AND bloom_filter_fp_chance = 0.01
> AND dclocal_read_repair_chance = 0.1
> AND crc_check_chance = 1.0
> AND default_time_to_live = 864
> AND gc_grace_seconds = 864000
> AND min_index_interval = 128
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE'
> AND caching = { 'keys': 'NONE', 'rows_per_partition': 'NONE' }
> AND compaction = { 'max_threshold': '32', 'min_threshold': '4',
> 'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy' }
> AND compression = { 'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor' }
> AND cdc = false
> AND extensions = { };
> CREATE INDEX table1_last_update_date_idx ON keyspace1.table1
> (last_update_date);
> I think the last part should be:
> CREATE INDEX IF NOT EXISTS table1_last_update_date_idx ON keyspace1.table1
> (last_update_date);
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14459) DynamicEndpointSnitch should never prefer latent nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509026#comment-16509026
]
Joseph Lynch commented on CASSANDRA-14459:
--
I've uploaded a patch which I think goes in the right direction. The Gossiper
only sends an extra {{PingMessage}} if the latency subscribers need latency
information. The DES only needs latency information if it has 2 or fewer
measurements in the preceding 10 minute period. This would mean a maximum of
one additional message to a host per 10 minute period under the assumption of
no traffic (may want this to be Echo if we want to support mixed mode clusters,
I'm somewhat ambivalent if it's an {{EchoMessage}} coming in on
{{REQUEST_RESPONSE}} or a {{PingMessage}} coming in on READ, but it sounds like
you dislike the latter). This avoids extra traffic to seeds as well as most
hosts under most conditions.
I think at this point I'm pretty convinced that we need some active
healthchecks _somewhere_, and at the start of this I was hoping to piggy back
on the existing ones in the Gossiper. If the {{Ack}} used {{sendReply}} then I
could get away with no additional traffic by adding callback handling in
{{GossipDigestAckVerbHandler}} but I don't see how to do that in a backwards
compatible way (where old hosts are not using reply leading to a 10s
measurement). I can also take a different approach and have a task that runs
separately from gossiper and sends out {{PingMessages}} from a different thread
all together. I could even unify that with the {{StartupConnectivityChecker}}
to form a {{HealthcheckService}}. This would be consistent with how e.g. load
balancers like HAProxy work (where they have background healthchecks that run
all at once on startup and then periodically after that).
> DynamicEndpointSnitch should never prefer latent nodes
> --
>
> Key: CASSANDRA-14459
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14459
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
> Fix For: 4.x
>
>
> The DynamicEndpointSnitch has two unfortunate behaviors that allow it to
> provide latent hosts as replicas:
> # Loses all latency information when Cassandra restarts
> # Clears latency information entirely every ten minutes (by default),
> allowing global queries to be routed to _other datacenters_ (and local
> queries cross racks/azs)
> This means that the first few queries after restart/reset could be quite slow
> compared to average latencies. I propose we solve this by resetting to the
> minimum observed latency instead of completely clearing the samples and
> extending the {{isLatencyForSnitch}} idea to a three state variable instead
> of two, in particular {{YES}}, {{NO}}, {{MAYBE}}. This extension allows
> {{EchoMessages}} and {{PingMessages}} to send {{MAYBE}} indicating that the
> DS should use those measurements if it only has one or fewer samples for a
> host. This fixes both problems because on process restart we send out
> {{PingMessages}} / {{EchoMessages}} as part of startup, and we would reset to
> effectively the RTT of the hosts (also at that point normal gossip
> {{EchoMessages}} have an opportunity to add an additional latency
> measurement).
> This strategy also nicely deals with the "a host got slow but now it's fine"
> problem that the DS resets were (afaik) designed to stop because the
> {{EchoMessage}} ping latency will count only after the reset for that host.
> Ping latency is a more reasonable lower bound on host latency (as opposed to
> status quo of zero).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14510) Flaky uTest: RemoveTest.testRemoveHostId
[
https://issues.apache.org/jira/browse/CASSANDRA-14510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509025#comment-16509025
]
Jay Zhuang commented on CASSANDRA-14510:
It's a duplication for CASSANDRA-14509
Basically, there's a message storm which causes the test timeout.
But it could still happen in production if there's any timeout here:
[{{AsyncOneResponse.java:51}}
|https://github.com/apache/cassandra/blob/6da5fb56c8e0777843e88359a45a461a9f9eb639/src/java/org/apache/cassandra/net/AsyncOneResponse.java#L51].
The next
[{{get()}}|https://github.com/apache/cassandra/blob/5dc55e715eba6667c388da9f8f1eb7a46489b35c/src/java/org/apache/cassandra/service/StorageService.java#L2736]
will timeout immediately and retry again and again:
[{{StorageService.java:2731}}|https://github.com/apache/cassandra/blob/5dc55e715eba6667c388da9f8f1eb7a46489b35c/src/java/org/apache/cassandra/service/StorageService.java#L2731].
Should we reset
[{{start}}|https://github.com/apache/cassandra/blob/6da5fb56c8e0777843e88359a45a461a9f9eb639/src/java/org/apache/cassandra/net/AsyncOneResponse.java#L33]
here:
[{{AsyncOneResponse.java:50}}|https://github.com/apache/cassandra/blob/6da5fb56c8e0777843e88359a45a461a9f9eb639/src/java/org/apache/cassandra/net/AsyncOneResponse.java#L50]
or remove the {{start}} and just use:
{noformat}
return super.get(timeout, unit);
{noformat}
cc [~bdeggleston], [~djoshi3], [~krummas]
> Flaky uTest: RemoveTest.testRemoveHostId
>
>
> Key: CASSANDRA-14510
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14510
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Major
>
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test/619/testReport/org.apache.cassandra.service/RemoveTest/testRemoveHostId/
> {noformat}
> Failed 13 times in the last 30 runs. Flakiness: 31%, Stability: 56%
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14510) Flaky uTest: RemoveTest.testRemoveHostId
[
https://issues.apache.org/jira/browse/CASSANDRA-14510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Zhuang reassigned CASSANDRA-14510:
--
Assignee: Jay Zhuang
> Flaky uTest: RemoveTest.testRemoveHostId
>
>
> Key: CASSANDRA-14510
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14510
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Major
>
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test/619/testReport/org.apache.cassandra.service/RemoveTest/testRemoveHostId/
> {noformat}
> Failed 13 times in the last 30 runs. Flakiness: 31%, Stability: 56%
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14499) node-level disk quota
[
https://issues.apache.org/jira/browse/CASSANDRA-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508955#comment-16508955
]
Jordan West commented on CASSANDRA-14499:
-
Nothing would disallow truncate, although if one node is at quota, is dropping
all data what is desired? In some use-cases perhaps. Since deletes temporarily
inflate storage use, for a node level quota I don't think they should be
allowed (for a keyspace-level quota that would be different perhaps). The
client also can't be expected to know exactly which keys live on the node(s)
that are at quota which makes remediation by delete less viable. The most
likely remediations are adding more nodes or truncation. A correct
implementation would prevent neither of these.
I agree that this could/should live in the management process
> node-level disk quota
> -
>
> Key: CASSANDRA-14499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14499
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Major
>
> Operators should be able to specify, via YAML, the amount of usable disk
> space on a node as a percentage of the total available or as an absolute
> value. If both are specified, the absolute value should take precedence. This
> allows operators to reserve space available to the database for background
> tasks -- primarily compaction. When a node reaches its quota, gossip should
> be disabled to prevent it taking further writes (which would increase the
> amount of data stored), being involved in reads (which are likely to be more
> inconsistent over time), or participating in repair (which may increase the
> amount of space used on the machine). The node re-enables gossip when the
> amount of data it stores is below the quota.
> The proposed option differs from {{min_free_space_per_drive_in_mb}}, which
> reserves some amount of space on each drive that is not usable by the
> database.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14499) node-level disk quota
[
https://issues.apache.org/jira/browse/CASSANDRA-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508939#comment-16508939
]
sankalp kohli commented on CASSANDRA-14499:
---
Why not add nodetool stop insert/delete/select and have the side car call these
when required?
> node-level disk quota
> -
>
> Key: CASSANDRA-14499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14499
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Major
>
> Operators should be able to specify, via YAML, the amount of usable disk
> space on a node as a percentage of the total available or as an absolute
> value. If both are specified, the absolute value should take precedence. This
> allows operators to reserve space available to the database for background
> tasks -- primarily compaction. When a node reaches its quota, gossip should
> be disabled to prevent it taking further writes (which would increase the
> amount of data stored), being involved in reads (which are likely to be more
> inconsistent over time), or participating in repair (which may increase the
> amount of space used on the machine). The node re-enables gossip when the
> amount of data it stores is below the quota.
> The proposed option differs from {{min_free_space_per_drive_in_mb}}, which
> reserves some amount of space on each drive that is not usable by the
> database.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14499) node-level disk quota
[
https://issues.apache.org/jira/browse/CASSANDRA-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508936#comment-16508936
]
sankalp kohli commented on CASSANDRA-14499:
---
We should give a way to recover after an application has filled a node. Can we
allow deletes or truncate?
> node-level disk quota
> -
>
> Key: CASSANDRA-14499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14499
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Major
>
> Operators should be able to specify, via YAML, the amount of usable disk
> space on a node as a percentage of the total available or as an absolute
> value. If both are specified, the absolute value should take precedence. This
> allows operators to reserve space available to the database for background
> tasks -- primarily compaction. When a node reaches its quota, gossip should
> be disabled to prevent it taking further writes (which would increase the
> amount of data stored), being involved in reads (which are likely to be more
> inconsistent over time), or participating in repair (which may increase the
> amount of space used on the machine). The node re-enables gossip when the
> amount of data it stores is below the quota.
> The proposed option differs from {{min_free_space_per_drive_in_mb}}, which
> reserves some amount of space on each drive that is not usable by the
> database.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13078) Increase unittest test.runners to speed up the test
[
https://issues.apache.org/jira/browse/CASSANDRA-13078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-13078:
Attachment: image.png
> Increase unittest test.runners to speed up the test
> ---
>
> Key: CASSANDRA-13078
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13078
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 3.0.15, 3.11.1, 4.0
>
> Attachments: unittest.png, unittest_time.png
>
>
> The unittest takes very long time to run (about 40 minutes on a macbook). By
> overriding the
> [{{test.runners}}|https://github.com/apache/cassandra/blob/cassandra-3.0/build.xml#L62],
> it could speed up the test, especially on powerful servers. Currently, it's
> set to 1 by default. I would like to propose to set the {{test.runners}} by
> the [number of CPUs
> dynamically|http://www.iliachemodanov.ru/en/blog-en/15-tools/ant/48-get-number-of-processors-in-ant-en].
> For example, {{runners = num_cores / 4}}. What do you guys think?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13078) Increase unittest test.runners to speed up the test
[
https://issues.apache.org/jira/browse/CASSANDRA-13078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-13078:
Attachment: (was: image.png)
> Increase unittest test.runners to speed up the test
> ---
>
> Key: CASSANDRA-13078
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13078
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 3.0.15, 3.11.1, 4.0
>
> Attachments: unittest.png, unittest_time.png
>
>
> The unittest takes very long time to run (about 40 minutes on a macbook). By
> overriding the
> [{{test.runners}}|https://github.com/apache/cassandra/blob/cassandra-3.0/build.xml#L62],
> it could speed up the test, especially on powerful servers. Currently, it's
> set to 1 by default. I would like to propose to set the {{test.runners}} by
> the [number of CPUs
> dynamically|http://www.iliachemodanov.ru/en/blog-en/15-tools/ant/48-get-number-of-processors-in-ant-en].
> For example, {{runners = num_cores / 4}}. What do you guys think?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14499) node-level disk quota
[
https://issues.apache.org/jira/browse/CASSANDRA-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508496#comment-16508496
]
Jeff Jirsa commented on CASSANDRA-14499:
{quote}
You need to recognize as an operator that your nodes are full and add more
nodes to your cluster, or add more disk space to your cluster.
{quote}
Even the best operators can't always add disk/instances fast enough to avoid
running a cluster 100% out of space in some cases, and if/when that happens,
you end up in a situation where you have no good options, perhaps a few "ok"
options (block client connections and expand, maybe), but plenty of bad options
(removing commitlogs, etc).
This isn't entirely new ground here, there's precedent elsewhere. Hadoop has
{{dfs.datanode.du.reserved}} per disk, for example.
> node-level disk quota
> -
>
> Key: CASSANDRA-14499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14499
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Major
>
> Operators should be able to specify, via YAML, the amount of usable disk
> space on a node as a percentage of the total available or as an absolute
> value. If both are specified, the absolute value should take precedence. This
> allows operators to reserve space available to the database for background
> tasks -- primarily compaction. When a node reaches its quota, gossip should
> be disabled to prevent it taking further writes (which would increase the
> amount of data stored), being involved in reads (which are likely to be more
> inconsistent over time), or participating in repair (which may increase the
> amount of space used on the machine). The node re-enables gossip when the
> amount of data it stores is below the quota.
> The proposed option differs from {{min_free_space_per_drive_in_mb}}, which
> reserves some amount of space on each drive that is not usable by the
> database.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14499) node-level disk quota
[
https://issues.apache.org/jira/browse/CASSANDRA-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508484#comment-16508484
]
Jordan West commented on CASSANDRA-14499:
-
Since the goal isn't to strictly enforce the quota (its ok if its violated but
once noticed action should be taken) the code isn't invasive. Its a small
amount of new code with the only change being to schedule the check on optional
tasks. That being said, if the concern is complexity, one potential place for
this (and I think it may be better home regardless) is
[CASSANDRA-14395|https://issues.apache.org/jira/browse/CASSANDRA-14395].
While this may seem like a small bandaid, and there are cases where multiple
nodes can go down at once, it is exactly meant to give some headroom. This
headroom makes it considerably easier to get the cluster into a healthy state
again.
> node-level disk quota
> -
>
> Key: CASSANDRA-14499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14499
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Major
>
> Operators should be able to specify, via YAML, the amount of usable disk
> space on a node as a percentage of the total available or as an absolute
> value. If both are specified, the absolute value should take precedence. This
> allows operators to reserve space available to the database for background
> tasks -- primarily compaction. When a node reaches its quota, gossip should
> be disabled to prevent it taking further writes (which would increase the
> amount of data stored), being involved in reads (which are likely to be more
> inconsistent over time), or participating in repair (which may increase the
> amount of space used on the machine). The node re-enables gossip when the
> amount of data it stores is below the quota.
> The proposed option differs from {{min_free_space_per_drive_in_mb}}, which
> reserves some amount of space on each drive that is not usable by the
> database.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14485) Optimize internode messaging protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508319#comment-16508319
]
Jason Brown commented on CASSANDRA-14485:
-
heh, good call with separating out the logic for pre.40 and 4.0 logic in
{{MessageInHandler}}. I was originally worried that the logic in {{decode()}}
would become more complicated if it was duplicated across two classes, but now
i see that most of the logic is different and just order of elements in the
header remains the same.
Pushed an extra commit to the same branch, and addressed the other items, as
well.
> Optimize internode messaging protocol
> -
>
> Key: CASSANDRA-14485
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14485
> Project: Cassandra
> Issue Type: Improvement
> Components: Streaming and Messaging
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Major
> Fix For: 4.0.x
>
>
> There's some dead wood and places for optimization in the internode messaging
> protocol. Currently, we include the sender's \{{IPAddressAndPort}} in *every*
> internode message, even though we already sent that in the handshake that
> established the connection/session. Further, there are several places where
> we can use vints instead of a fixed, 4-byte integer value- especially as
> those values will almost always be less than one byte.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14509) AsyncOneResponse uses the incorrect timeout
[
https://issues.apache.org/jira/browse/CASSANDRA-14509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-14509:
Resolution: Fixed
Fix Version/s: (was: 4.x)
4.0
Status: Resolved (was: Patch Available)
and committed as {{6da5fb56c8e0777843e88359a45a461a9f9eb639}} without the
test.runners change - if that is a problem we should probably fix that in a
separate ticket, thanks!
> AsyncOneResponse uses the incorrect timeout
> ---
>
> Key: CASSANDRA-14509
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14509
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
> Fix For: 4.0
>
>
> {{AsyncOneResponse}} has a bug where it uses the initial timeout value
> instead of the adjustedTimeout. Combined with passing in the wrong
> {{TimeUnit}}, it leads to a shorter timeout than expected. This can have
> unintended consequences for example, in
> {{StorageService::sendReplicationNotification}} instead of waiting 10 seconds
> ({{request_timeout_in_ms}}), we wait for {{1}} Nano Seconds.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra git commit: Make AsyncOneResponse use the correct timeout
Repository: cassandra
Updated Branches:
refs/heads/trunk 958e13d16 -> 6da5fb56c
Make AsyncOneResponse use the correct timeout
Patch by Dinesh Joshi; reviewed by marcuse for CASSANDRA-14509
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6da5fb56
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6da5fb56
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6da5fb56
Branch: refs/heads/trunk
Commit: 6da5fb56c8e0777843e88359a45a461a9f9eb639
Parents: 958e13d
Author: Dinesh A. Joshi
Authored: Fri Jun 8 16:38:00 2018 -0700
Committer: Marcus Eriksson
Committed: Mon Jun 11 08:55:55 2018 -0700
--
CHANGES.txt | 1 +
.../apache/cassandra/net/AsyncOneResponse.java | 2 +-
.../cassandra/net/AsyncOneResponseTest.java | 63
3 files changed, 65 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6da5fb56/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 27c9561..6819711 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0
+ * Make AsyncOneResponse use the correct timeout (CASSANDRA-14509)
* Add option to sanity check tombstones on reads/compactions (CASSANDRA-14467)
* Add a virtual table to expose all running sstable tasks (CASSANDRA-14457)
* Let nodetool import take a list of directories (CASSANDRA-14442)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6da5fb56/src/java/org/apache/cassandra/net/AsyncOneResponse.java
--
diff --git a/src/java/org/apache/cassandra/net/AsyncOneResponse.java
b/src/java/org/apache/cassandra/net/AsyncOneResponse.java
index 4e004d2..3fe0a2a 100644
--- a/src/java/org/apache/cassandra/net/AsyncOneResponse.java
+++ b/src/java/org/apache/cassandra/net/AsyncOneResponse.java
@@ -52,7 +52,7 @@ public class AsyncOneResponse extends AbstractFuture
implements IAsyncCall
}
try
{
-return super.get(timeout, TimeUnit.NANOSECONDS);
+return super.get(adjustedTimeout, TimeUnit.NANOSECONDS);
}
catch (InterruptedException | ExecutionException e)
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6da5fb56/test/unit/org/apache/cassandra/net/AsyncOneResponseTest.java
--
diff --git a/test/unit/org/apache/cassandra/net/AsyncOneResponseTest.java
b/test/unit/org/apache/cassandra/net/AsyncOneResponseTest.java
new file mode 100644
index 000..15b3327
--- /dev/null
+++ b/test/unit/org/apache/cassandra/net/AsyncOneResponseTest.java
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.net;
+
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+import org.junit.Test;
+
+import static org.junit.Assert.assertTrue;
+
+public class AsyncOneResponseTest
+{
+@Test(expected = TimeoutException.class)
+public void getThrowsExceptionAfterTimeout() throws InterruptedException,
TimeoutException
+{
+AsyncOneResponse response = new AsyncOneResponse<>();
+Thread.sleep(2000);
+response.get(1, TimeUnit.SECONDS);
+}
+
+@Test
+public void getThrowsExceptionAfterCorrectTimeout()
+{
+AsyncOneResponse response = new AsyncOneResponse<>();
+
+final long expectedTimeoutMillis = 1000; // Should time out after
roughly this time
+final long schedulingError = 10; // Scheduling is imperfect
+boolean hitException = false; // Ensure we actually hit the
TimeoutException
+
+long startTime = System.currentTimeMillis();
+
+try
+{
+response.get(expectedTimeoutMillis, TimeUnit.MILLISECONDS);
+}
+catch(TimeoutException e)
+{
+hitException = true;
+}
+
+long endTime = System.currentTimeMillis();
+
[jira] [Commented] (CASSANDRA-13938) Default repair is broken, crashes other nodes participating in repair (in trunk)
[
https://issues.apache.org/jira/browse/CASSANDRA-13938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508237#comment-16508237
]
Jason Brown commented on CASSANDRA-13938:
-
The problem is that when {{CompressedInputStream#position()}} is called, the
new position might be in the middle of a buffer. We need to remember that
offset, and subtract that value when updating {{current}} in
{{#reBuffer(boolean)}}. The resaon why is that those offset bytes get double
counted on the first call to {{#reBuffer()}} after {{#position()}} as we add
the {{buffer.position()}} to {{current}}. {{current}} already accounts for
those offset bytes when {{#position()}} was called.
Patch below fixes the problem:
||13938||
|[branch|https://github.com/jasobrown/cassandra/tree/13938]|
|[utests &
dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/13938]|
Also, since I was in this code, I've cleaned it up a bit:
- renamed {{currentOffset}} to {{streamOffset}}
- reorganized static and member fields at top of class, and added more
comments.
- capture a reference to the background thread that will execute
{{CompressedInputStream#Reader}}, so we can close it properly.
- made {{close()}} more responsible for shutting down/cleaning up resources
- cleaned up {{decompress()}} by moving the CRC validation into a sub-method
(got rid of a needless local boolean)
I also have a dtest based on [~zznate]'s repro steps and schema (slightly
modified as we no longer support COMPACT_STORAGE), and it produced the error
every time (before the patch). I also tried [~pauloricardomg] 's dtest from
CASSANDRA-14394, but it did not repro for me. I'm happy with either dtest (or
both), but would definitely like to include one of them.
||13938-dtest||
|[jasobrown dtest|https://github.com/jasobrown/cassandra-dtest/tree/13938]|
|[paulo
dtest|https://github.com/pauloricardomg/cassandra-dtest/tree/CASSANDRA-14394]|
> Default repair is broken, crashes other nodes participating in repair (in
> trunk)
>
>
> Key: CASSANDRA-13938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13938
> Project: Cassandra
> Issue Type: Bug
> Components: Repair
>Reporter: Nate McCall
>Assignee: Jason Brown
>Priority: Critical
> Attachments: 13938.yaml, test.sh
>
>
> Running through a simple scenario to test some of the new repair features, I
> was not able to make a repair command work. Further, the exception seemed to
> trigger a nasty failure state that basically shuts down the netty connections
> for messaging *and* CQL on the nodes transferring back data to the node being
> repaired. The following steps reproduce this issue consistently.
> Cassandra stress profile (probably not necessary, but this one provides a
> really simple schema and consistent data shape):
> {noformat}
> keyspace: standard_long
> keyspace_definition: |
> CREATE KEYSPACE standard_long WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> table: test_data
> table_definition: |
> CREATE TABLE test_data (
> key text,
> ts bigint,
> val text,
> PRIMARY KEY (key, ts)
> ) WITH COMPACT STORAGE AND
> CLUSTERING ORDER BY (ts DESC) AND
> bloom_filter_fp_chance=0.01 AND
> caching={'keys':'ALL', 'rows_per_partition':'NONE'} AND
> comment='' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.00 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> columnspec:
> - name: key
> population: uniform(1..5000) # 50 million records available
> - name: ts
> cluster: gaussian(1..50) # Up to 50 inserts per record
> - name: val
> population: gaussian(128..1024) # varrying size of value data
> insert:
> partitions: fixed(1) # only one insert per batch for individual partitions
> select: fixed(1)/1 # each insert comes in one at a time
> batchtype: UNLOGGED
> queries:
> single:
> cql: select * from test_data where key = ? and ts = ? limit 1;
> series:
> cql: select key,ts,val from test_data where key = ? limit 10;
> {noformat}
> The commands to build and run:
> {noformat}
> ccm create 4_0_test -v git:trunk -n 3 -s
> ccm stress user profile=./histo-test-schema.yml
> ops\(insert=20,single=1,series=1\) duration=15s -rate threads=4
> # flush the memtable just to get everything on disk
> ccm node1 nodetool flush
> ccm node2 nodetool flush
> ccm node3 nodetool flush
> # disable hints for nodes 2 and 3
> ccm node2 nodetool disablehandoff
> ccm node3 nodetool disablehandoff
> # stop node1
> ccm node1 stop
> ccm stress user profile=./histo-test-schema.yml
> ops\(insert=20,single=1,series=1\) duration=45s -rate threads=4
> # wait
[jira] [Resolved] (CASSANDRA-14511) Trigger for TTL
[ https://issues.apache.org/jira/browse/CASSANDRA-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi resolved CASSANDRA-14511. -- Resolution: Invalid > Trigger for TTL > --- > > Key: CASSANDRA-14511 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14511 > Project: Cassandra > Issue Type: Wish >Reporter: Ghazi Ennacer >Priority: Minor > > Hello, > I am asking if there is a trigger that detects the TTL mechanism ? > I appreciate your help. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14511) Trigger for TTL
[ https://issues.apache.org/jira/browse/CASSANDRA-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508178#comment-16508178 ] Dinesh Joshi commented on CASSANDRA-14511: -- This belongs on the users list. Could you ask this question on the mailing list? > Trigger for TTL > --- > > Key: CASSANDRA-14511 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14511 > Project: Cassandra > Issue Type: Wish >Reporter: Ghazi Ennacer >Priority: Minor > > Hello, > I am asking if there is a trigger that detects the TTL mechanism ? > I appreciate your help. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14511) Trigger for TTL
Ghazi Ennacer created CASSANDRA-14511: - Summary: Trigger for TTL Key: CASSANDRA-14511 URL: https://issues.apache.org/jira/browse/CASSANDRA-14511 Project: Cassandra Issue Type: Wish Reporter: Ghazi Ennacer Hello, I am asking if there is a trigger that detects the TTL mechanism ? I appreciate your help. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
