[jira] [Comment Edited] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683716#comment-17683716
]
Berenguer Blasi edited comment on CASSANDRA-14227 at 2/3/23 7:50 AM:
-
Putting this back for a first pass review with the new Uint approach. I have
done a final round of perf testing:
* Perf testing revolved around 10M rows and runs with random/fixed TTL for
periods of 2m, 5m and 10m and averaging 10 runs. The exception being the much
longer latency test.
* Longer tests tend to give less noisy results. 14227 vs trunk may randomly
appear one slightly better than the other which I attribute to env noise and
lack of dedicated perf testing HW.
* Latency seems to be aligned as per comment above
* JFRs available on request
* Disk size, flushes etc seem to be aligned. Here is some table stats where
sometimes 14227 is slightly better and sometimes the other way around
!unnamed-1.png!
The PR has several commits to make it easier to review. Some refactoring of an
earlier commit might be found in a later one but nothing too big and quite
straightforward. I have left a bunch of 14227 TODO comments to bring the
attention of the reviewer just for feedback. CI can be found in the PR, yet
repeat tests haven't been run yet due to the size. We can do it at a later time
once we're happy with the approach
The only thing to note is the addition of the NONE policy where it's later
removed. That is the only bit it can't be easily collapsed, apologies in
advance.
was (Author: bereng):
Putting this back for a first pass review with the new Uint approach. I have
done a final round of perf testing:
* Perf testing revolved around 10M rows and runs with random/fixed TTL for
period of 2m, 5m and 10m. The exception being the much longer latency test.
* Longer tests tend to give less noisy results. 14227 vs trunk may randomly
appear one slightly better than the other which I attribute to env noise and
lack of dedicated perf testing HW.
* Latency seems to be aligned as per comment above
* JFRs available on request
* Disk size, flushes etc seem to be aligned. Here is some table stats where
sometimes 14227 is slightly better and sometimes the other way around
!unnamed-1.png!
The PR has several commits to make it easier to review. Some refactoring of an
earlier commit might be found in a later one but nothing too big and quite
straightforward. I have left a bunch of 14227 TODO comments to bring the
attention of the reviewer just for feedback. CI can be found in the PR, yet
repeat tests haven't been run yet due to the size. We can do it at a later time
once we're happy with the approach
The only thing to note is the addition of the NONE policy where it's later
removed. That is the only bit it can't be easily collapsed, apologies in
advance.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683716#comment-17683716
]
Berenguer Blasi edited comment on CASSANDRA-14227 at 2/3/23 7:49 AM:
-
Putting this back for a first pass review with the new Uint approach. I have
done a final round of perf testing:
* Perf testing revolved around 10M rows and runs with random/fixed TTL for
period of 2m, 5m and 10m. The exception being the much longer latency test.
* Longer tests tend to give less noisy results. 14227 vs trunk may randomly
appear one slightly better than the other which I attribute to env noise and
lack of dedicated perf testing HW.
* Latency seems to be aligned as per comment above
* JFRs available on request
* Disk size, flushes etc seem to be aligned. Here is some table stats where
sometimes 14227 is slightly better and sometimes the other way around
!unnamed-1.png!
The PR has several commits to make it easier to review. Some refactoring of an
earlier commit might be found in a later one but nothing too big and quite
straightforward. I have left a bunch of 14227 TODO comments to bring the
attention of the reviewer just for feedback. CI can be found in the PR, yet
repeat tests haven't been run yet due to the size. We can do it at a later time
once we're happy with the approach
The only thing to note is the addition of the NONE policy where it's later
removed. That is the only bit it can't be easily collapsed, apologies in
advance.
was (Author: bereng):
Putting this back for a first pass review with the new Uint approach. I have
done a final round of perf testing:
* Perf testing revolved around 10M rows and runs with random/fixed TTL for
period of 2m, 5m and 10m. The exception being the much longer latency test.
* Longer tests tend to give less noisy results. 14227 vs trunk may randomly
appear one slightly better than the other which I attribute to env noise and
lack of dedicated perf testing HW.
* Latency seems to be aligned as per comment above
* Disk size, flushes etc seem to be aligned. Here is some table stats where
sometimes 14227 is slightly better and sometimes the other way around
!unnamed-1.png!
The PR has several commits to make it easier to review. Some refactoring of an
earlier commit might be found in a later one but nothing too big and quite
straightforward. I have left a bunch of 14227 TODO comments to bring the
attention of the reviewer just for feedback. CI can be found in the PR, yet
repeat tests haven't been run yet due to the size. We can do it at a later time
once we're happy with the approach
The only thing to note is the addition of the NONE policy where it's later
removed. That is the only bit it can't be easily collapsed, apologies in
advance.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-14227:
Status: Patch Available (was: In Progress)
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683716#comment-17683716
]
Berenguer Blasi edited comment on CASSANDRA-14227 at 2/3/23 7:48 AM:
-
Putting this back for a first pass review with the new Uint approach. I have
done a final round of perf testing:
* Perf testing revolved around 10M rows and runs with random/fixed TTL for
period of 2m, 5m and 10m. The exception being the much longer latency test.
* Longer tests tend to give less noisy results. 14227 vs trunk may randomly
appear one slightly better than the other which I attribute to env noise and
lack of dedicated perf testing HW.
* Latency seems to be aligned as per comment above
* Disk size, flushes etc seem to be aligned. Here is some table stats where
sometimes 14227 is slightly better and sometimes the other way around
!unnamed-1.png!
The PR has several commits to make it easier to review. Some refactoring of an
earlier commit might be found in a later one but nothing too big and quite
straightforward. I have left a bunch of 14227 TODO comments to bring the
attention of the reviewer just for feedback. CI can be found in the PR, yet
repeat tests haven't been run yet due to the size. We can do it at a later time
once we're happy with the approach
The only thing to note is the addition of the NONE policy where it's later
removed. That is the only bit it can't be easily collapsed, apologies in
advance.
was (Author: bereng):
Putting this back into review with the new Uint approach. I have done a final
round of perf testing:
* Perf testing revolved around 10M rows and runs with random/fixed TTL for
period of 2m, 5m and 10m. The exception being the much longer latency test.
* Longer tests tend to give less noisy results. 14227 vs trunk may randomly
appear one slightly better than the other which I attribute to env noise and
lack of dedicated perf testing HW.
* Latency seems to be aligned as per comment above
* Disk size, flushes etc seem to be aligned. Here is some table stats where
sometimes 14227 is slightly better and sometimes the other way around
!unnamed-1.png!
The PR has several commits to make it easier to review. Some refactoring of an
earlier commit might be found in a later one but nothing too big and quite
straightforward. I have left a bunch of 14227 TODO comments to bring the
attention of the reviewer just for feedback. CI can be found in the PR.
The only thing to note is the addition of the NONE policy where it's later
removed. That is the only bit it can't be easily collapsed, apologies in
advance.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683716#comment-17683716
]
Berenguer Blasi edited comment on CASSANDRA-14227 at 2/3/23 7:46 AM:
-
Putting this back into review with the new Uint approach. I have done a final
round of perf testing:
* Perf testing revolved around 10M rows and runs with random/fixed TTL for
period of 2m, 5m and 10m. The exception being the much longer latency test.
* Longer tests tend to give less noisy results. 14227 vs trunk may randomly
appear one slightly better than the other which I attribute to env noise and
lack of dedicated perf testing HW.
* Latency seems to be aligned as per comment above
* Disk size, flushes etc seem to be aligned. Here is some table stats where
sometimes 14227 is slightly better and sometimes the other way around
!unnamed-1.png!
The PR has several commits to make it easier to review. Some refactoring of an
earlier commit might be found in a later one but nothing too big and quite
straightforward. I have left a bunch of 14227 TODO comments to bring the
attention of the reviewer just for feedback. CI can be found in the PR.
The only thing to note is the addition of the NONE policy where it's later
removed. That is the only bit it can't be easily collapsed, apologies in
advance.
was (Author: bereng):
Putting this back into review with the new Uint approach. I have done a final
round of perf testing:
* Perf testing revolved around 10M rows and runs with random/fixed TTL for
period of 2m, 5m and 10m. The exception being the much longer latency test.
* Longer tests tend to give less noisy results. 14227 vs trunk may randomly
appear one slightly better than the other which I attribute to env noise and
lack of dedicated perf testing HW.
* Latency seems to be aligned as per comment above
* Disk size, flushes etc seem to be aligned. Here is some table stats where
sometimes 14227 is slightly better and sometimes the other way around
!unnamed.png!
The PR has several commits to make it easier to review. Some refactoring of an
earlier commit might be found in a later one but nothing too big and quite
straightforward. I have left a bunch of 14227 TODO comments to bring the
attention of the reviewer just for feedback. CI can be found in the PR.
The only thing to note is the addition of the NONE policy where it's later
removed. That is the only bit it can't be easily collapsed, apologies in
advance.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683716#comment-17683716
]
Berenguer Blasi commented on CASSANDRA-14227:
-
Putting this back into review with the new Uint approach. I have done a final
round of perf testing:
* Perf testing revolved around 10M rows and runs with random/fixed TTL for
period of 2m, 5m and 10m. The exception being the much longer latency test.
* Longer tests tend to give less noisy results. 14227 vs trunk may randomly
appear one slightly better than the other which I attribute to env noise and
lack of dedicated perf testing HW.
* Latency seems to be aligned as per comment above
* Disk size, flushes etc seem to be aligned. Here is some table stats where
sometimes 14227 is slightly better and sometimes the other way around
!unnamed.png!
The PR has several commits to make it easier to review. Some refactoring of an
earlier commit might be found in a later one but nothing too big and quite
straightforward. I have left a bunch of 14227 TODO comments to bring the
attention of the reviewer just for feedback. CI can be found in the PR.
The only thing to note is the addition of the NONE policy where it's later
removed. That is the only bit it can't be easily collapsed, apologies in
advance.
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png, unnamed.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-14227:
Attachment: (was: unnamed.png)
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-14227:
Attachment: unnamed-1.png
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed-1.png, unnamed.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-14227:
Attachment: unnamed.png
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta (Deprecated)
>Assignee: Berenguer Blasi
>Priority: Urgent
> Fix For: 4.x
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, unnamed.png
>
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17949) Update CCM post CASSANDRA-17379
[ https://issues.apache.org/jira/browse/CASSANDRA-17949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683674#comment-17683674 ] Berenguer Blasi commented on CASSANDRA-17949: - Hi, Is the link broken? > Update CCM post CASSANDRA-17379 > --- > > Key: CASSANDRA-17949 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17949 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > > We updated in CASSANDRA-15234 CCM updateconf to enable it to handle > overloading of parameters Python upgrade tests. > This also allows us not to update for 4.1 the Python DTests but exercise the > backward compatibility we have in C* in our tests. > Now CASSANDRA-17379 added a few flags to opt in or out for overloading of > parameters. > The default points to -Dcassandra.allow_new_old_config_keys=false but this > does not affect our tests as CCM handles the overloading in the yaml. > But this is confusing for users and might lead to issues. > We need to bring on par one way or another ccm with the new flags introduced > in CASSANDRA-17379 > *Until then the recommendation is to use > `-Dcassandra.allow_new_old_config_keys=false` and > `-Dcassandra.allow_duplicate_config_keys=false` to prohibit any kind of > overloading when using CCM master and CCM released to prevent any confusion.* -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18051) Optimize test_network_topology_strategy and test_network_topology_strategy_each_quorum for large containers in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683672#comment-17683672
]
Berenguer Blasi commented on CASSANDRA-18051:
-
+1
> Optimize test_network_topology_strategy and
> test_network_topology_strategy_each_quorum for large containers in CircleCI
> ---
>
> Key: CASSANDRA-18051
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18051
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>
> As pointed in CASSANDRA-18001, there are two Python Large DTests that fail
> because of not enough resources with large containers. [~adelapena] looked
> into them and suggested that the tests themselves can be optimized for
> running on Large containers instead of using XLarge containers for only two
> tests from the whole suite.
> {quote}As for the two tests starting nine nodes each,
> {{test_network_topology_strategy}} and
> {{{}test_network_topology_strategy_each_quorum{}}}, I think they were added
> by CASSANDRA-10584 to test multi-DC consistency levels. I wonder if they
> actually need to start 3 data centers with 3 nodes each. Would two data
> centers with 3 nodes be enough to test multi-DC consistency levels with
> multiple data centers? 6 nodes should be doable on Circle with the current
> resources.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18215) Fix the output of FQL dump tool to properly separate entries
[
https://issues.apache.org/jira/browse/CASSANDRA-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683653#comment-17683653
]
n.v.harikrishna commented on CASSANDRA-18215:
-
Thanks! I will update the patch details soon.
> Fix the output of FQL dump tool to properly separate entries
>
>
> Key: CASSANDRA-18215
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18215
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Stefan Miklosovic
>Assignee: n.v.harikrishna
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
>
> This was reported in (1)
> (1) https://github.com/apache/cassandra/pull/2050
> If I create a table something like this:
> {code}
> CREATE TABLE t1 ( id int PRIMARY KEY , v1 int, v2 int, v3 int) ;
> {code}
> and inserted a row using:
> {code}
> try (CqlSession cqlSession = CqlSession.builder().build()) {
> PreparedStatement preparedStatement = cqlSession.prepare("INSERT INTO
> ks1.t1 (id, v1, v2, v3) VALUES (?, ?, ?, ?)");
> cqlSession.execute(preparedStatement.bind(1, 1, 1, 1));
> }
> {code}
> The output of fqltool looks something like this:
> {code}
> Type: single-query
> Query start time: 1673373829119
> Protocol version: 5
> Generated timestamp:-9223372036854775808
> Generated nowInSeconds:1673373829
> Query: INSERT INTO ks1.t1 (id, v1, v2, v3) VALUES (?, ?, ?, ?)
> Values:
> 00 00 00 01
> 00 00 00 01
> -
> 00 00 00 01
> -
> 00 00 00 01
> -
> {code}
> - is not printed between the first and second value
> {code}
> Values:
> 00 00 00 01
> 00 00 00 01
> {code}
> We are normally very cautious about changing the output of the tooling but in
> this case I think this is a legit bug which should be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18215) Fix the output of FQL dump tool to properly separate entries
[
https://issues.apache.org/jira/browse/CASSANDRA-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
n.v.harikrishna reassigned CASSANDRA-18215:
---
Assignee: n.v.harikrishna
> Fix the output of FQL dump tool to properly separate entries
>
>
> Key: CASSANDRA-18215
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18215
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Stefan Miklosovic
>Assignee: n.v.harikrishna
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
>
> This was reported in (1)
> (1) https://github.com/apache/cassandra/pull/2050
> If I create a table something like this:
> {code}
> CREATE TABLE t1 ( id int PRIMARY KEY , v1 int, v2 int, v3 int) ;
> {code}
> and inserted a row using:
> {code}
> try (CqlSession cqlSession = CqlSession.builder().build()) {
> PreparedStatement preparedStatement = cqlSession.prepare("INSERT INTO
> ks1.t1 (id, v1, v2, v3) VALUES (?, ?, ?, ?)");
> cqlSession.execute(preparedStatement.bind(1, 1, 1, 1));
> }
> {code}
> The output of fqltool looks something like this:
> {code}
> Type: single-query
> Query start time: 1673373829119
> Protocol version: 5
> Generated timestamp:-9223372036854775808
> Generated nowInSeconds:1673373829
> Query: INSERT INTO ks1.t1 (id, v1, v2, v3) VALUES (?, ?, ?, ?)
> Values:
> 00 00 00 01
> 00 00 00 01
> -
> 00 00 00 01
> -
> 00 00 00 01
> -
> {code}
> - is not printed between the first and second value
> {code}
> Values:
> 00 00 00 01
> 00 00 00 01
> {code}
> We are normally very cautious about changing the output of the tooling but in
> this case I think this is a legit bug which should be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18050) Update JNA
[
https://issues.apache.org/jira/browse/CASSANDRA-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683632#comment-17683632
]
Ekaterina Dimitrova commented on CASSANDRA-18050:
-
{quote}Setting up a jdk17 circleci config (or just hacking the config in the
throwaway commit) might be the productive way to go here.
{quote}
I can do that at any time (I have CircleCI J17 config), it won't test arm though
> Update JNA
> --
>
> Key: CASSANDRA-18050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18050
> Project: Cassandra
> Issue Type: Task
>Reporter: Ekaterina Dimitrova
>Priority: Normal
>
> JNA needs to be updated for JDK 17 support. There was an update pre-4.0 but
> not to the latest version (confirmed with the people involved, there was no
> particular reason not to update to the latest version, just missed on rebase
> of older PR)
> More details to come, opening this as a placeholder for now
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18184) 'Maximum memory usage reached' chunk cache log message doesn't specify which cache is exhausted
[
https://issues.apache.org/jira/browse/CASSANDRA-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683630#comment-17683630
]
Yong Jiang commented on CASSANDRA-18184:
Great! Thank you everyone for the help! [~brandon.williams] , [~maxwellguo] ,
[~smiklosovic] !
> 'Maximum memory usage reached' chunk cache log message doesn't specify which
> cache is exhausted
> ---
>
> Key: CASSANDRA-18184
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18184
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Yong Jiang
>Priority: Normal
> Fix For: 4.0.8, 4.1.1, 4.2
>
> Attachments: 18184-trunk.txt
>
>
> With Cassandra 4.0.x, we are seeing this cassandra.log message very
> frequently on the majority of our Cassandra 4.0 clusters:
> _[INFO ] [epollEventLoopGroup-5-3] cluster_id=99 ip_address=127.0.0.50
> NoSpamLogger.java:92 - Maximum memory usage reached (128.000MiB), cannot
> allocate chunk of 8.000MiB_
> It took me several weeks to track down what it means, until I saw this
> start-up message
> _BufferPools.java:49 - Global buffer pool limit is 2.000GiB for
> chunk-cache and 128.000MiB for networking_
> This maximum memory usage warning would benefit from clarifying that its the
> {*}network cache{*}, not the *off-heap chunk* *cache* which is exhausted.
> With 'chunk cache' in both warning messages, they're too easily confused.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18050) Update JNA
[ https://issues.apache.org/jira/browse/CASSANDRA-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683623#comment-17683623 ] Michael Semb Wever commented on CASSANDRA-18050: bq. we will need to test also potentially with the JDK17 branches too but at this point JDK17 in Jenkins is tested not in ASF Jenkins. Let's catch up with Mick on that tomorrow maybe? I'm not sure how best to tackle this. My suggestion for now is that we deal with JDK17 testing separately later. Setting up a jdk17 circleci config (or just hacking the config in the throwaway commit) might be the productive way to go here. > Update JNA > -- > > Key: CASSANDRA-18050 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18050 > Project: Cassandra > Issue Type: Task >Reporter: Ekaterina Dimitrova >Priority: Normal > > JNA needs to be updated for JDK 17 support. There was an update pre-4.0 but > not to the latest version (confirmed with the people involved, there was no > particular reason not to update to the latest version, just missed on rebase > of older PR) > More details to come, opening this as a placeholder for now -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18050) Update JNA
[ https://issues.apache.org/jira/browse/CASSANDRA-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683621#comment-17683621 ] Michael Semb Wever commented on CASSANDRA-18050: bq. I think Michael Semb Wever mentioned something about the need to update snappy anyway, if I am not mistaken Was already done in CASSANDRA-17040 (for >4.1 we're now on the latest snappy) > Update JNA > -- > > Key: CASSANDRA-18050 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18050 > Project: Cassandra > Issue Type: Task >Reporter: Ekaterina Dimitrova >Priority: Normal > > JNA needs to be updated for JDK 17 support. There was an update pre-4.0 but > not to the latest version (confirmed with the people involved, there was no > particular reason not to update to the latest version, just missed on rebase > of older PR) > More details to come, opening this as a placeholder for now -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17040) Upgrade Snappy version to support Apple M1
[
https://issues.apache.org/jira/browse/CASSANDRA-17040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-17040:
---
Platform: ARM (was: All)
> Upgrade Snappy version to support Apple M1
> --
>
> Key: CASSANDRA-17040
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17040
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Compression
>Reporter: Yuqi Gu
>Assignee: Yuqi Gu
>Priority: Normal
> Fix For: 4.1-alpha1, 4.1
>
> Attachments: UTs.txt, UTs_2.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Some Unit test cases were failed in Apple M1:
>
> {code:java}
> [junit-timeout] Testcase:
> testTableOptions(org.apache.cassandra.cql3.validation.miscellaneous.OverflowTest):
> Caused an ERROR
> [junit-timeout] SnappyCompressor.create() threw an error:
> java.lang.NoClassDefFoundError Could not initialize class
> org.xerial.snappy.Snappy
> [junit-timeout] org.apache.cassandra.exceptions.ConfigurationException:
> SnappyCompressor.create() threw an error: java.lang.NoClassDefFoundError
> Could not initialize class org.xerial.snappy.Snappy
> [junit-timeout] at
> org.apache.cassandra.schema.CompressionParams.createCompressor(CompressionParams.java:344)
> [junit-timeout] at
> org.apache.cassandra.schema.CompressionParams.(CompressionParams.java:211)
> [junit-timeout] at
> org.apache.cassandra.schema.CompressionParams.fromMap(CompressionParams.java:124)
> [junit-timeout] at
> org.apache.cassandra.cql3.statements.schema.TableAttributes.build(TableAttributes.java:110)
> [junit-timeout] at
> org.apache.cassandra.cql3.statements.schema.TableAttributes.validate(TableAttributes.java:58)
> [junit-timeout] at
>
> ...
> ..
>
> {code}
>
> Snappy-java added M1 support since
> 1.1.8.2.([https://github.com/xerial/snappy-java/pull/268).]
> So let's upgrade snappy-java dependency to the latest release 1.1.8.4.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17608) Fix testMetricsWithRebuildAndStreamingToTwoNodes
[
https://issues.apache.org/jira/browse/CASSANDRA-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dan jatnieks updated CASSANDRA-17608:
-
Resolution: (was: Cannot Reproduce)
Status: Open (was: Resolved)
{{testMetricsWithRebuildAndStreamingToTwoNodes}} failed on {{trunk}} in [build
1445|https://butler.cassandra.apache.org/#/ci/upstream/workflow/Cassandra-trunk/failure/org.apache.cassandra.distributed.test.metrics/StreamingMetricsTest/testMetricsWithRebuildAndStreamingToTwoNodes]
> Fix testMetricsWithRebuildAndStreamingToTwoNodes
>
>
> Key: CASSANDRA-17608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17608
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta1
>
>
> [https://ci-cassandra.apache.org/job/Cassandra-4.1/4/testReport/org.apache.cassandra.distributed.test.metrics/StreamingMetricsTest/testMetricsWithRebuildAndStreamingToTwoNodes/]
>
> h3.
> {code:java}
> Error Message
> [The amount of files received by node2 from node3 is not the expected one.
> [expected: 5, actual: 6]] expected:<[5]L> but was:<[6]L>
> Stacktrace
> junit.framework.AssertionFailedError: [The amount of files received by node2
> from node3 is not the expected one. [expected: 5, actual: 6]] expected:<[5]L>
> but was:<[6]L> at
> org.apache.cassandra.distributed.test.metrics.StreamingMetricsTest.lambda$checkDataReceived$e7abd6ea$1(StreamingMetricsTest.java:368)
> at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:81) at
> org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:47) at
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:57) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18130) Log hardware and container params during test runs to help troubleshoot intermittent failures
[
https://issues.apache.org/jira/browse/CASSANDRA-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683591#comment-17683591
]
Michael Semb Wever edited comment on CASSANDRA-18130 at 2/3/23 12:37 AM:
-
brain-fart. this patch only logs on the built-in where the pipeline runs, not
the stage jobs. the patch (before CASSANDRA-18133) needs to go in
cassandra-builds/jenkins-dsl/
was (Author: michaelsembwever):
brain-fart. this patch only logs on the built-in where the pipeline runs, not
the stage jobs. the patch needs to go in cassandra-builds/jenkins-dsl/
> Log hardware and container params during test runs to help troubleshoot
> intermittent failures
> -
>
> Key: CASSANDRA-18130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18130
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/java, Test/dtest/python, Test/unit
>Reporter: Josh McKenzie
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
>
> {color:#00}We’ve long had flakiness in our containerized ASF CI
> environment that we don’t see in circleci. The environment itself is both
> containerized and heterogenous, so there are differences in both the hardware
> environment and the software environment in which it executes. For reference,
> see:
> [https://github.com/apache/cassandra-builds/blob/trunk/ASF-jenkins-agents.md#current-agents]{color}
> {color:#00} {color}
> {color:#00}We should log a variety of hardware, container, and software
> environment details to help get to the bottom of where some test failures may
> be occurring. As we don’t have shell access to the machines it’ll be easier
> to have this information logged / retrieved during test runs than to try and
> profile each host independently.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (22c4f960e -> 394fee0a4)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 22c4f960e generate docs for 11d2d4b3 new 394fee0a4 generate docs for 11d2d4b3 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (22c4f960e) \ N -- N -- N refs/heads/asf-staging (394fee0a4) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4970898 -> 4970898 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18051) Optimize test_network_topology_strategy and test_network_topology_strategy_each_quorum for large containers in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683593#comment-17683593
]
Michael Semb Wever commented on CASSANDRA-18051:
WFM too.
> Optimize test_network_topology_strategy and
> test_network_topology_strategy_each_quorum for large containers in CircleCI
> ---
>
> Key: CASSANDRA-18051
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18051
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>
> As pointed in CASSANDRA-18001, there are two Python Large DTests that fail
> because of not enough resources with large containers. [~adelapena] looked
> into them and suggested that the tests themselves can be optimized for
> running on Large containers instead of using XLarge containers for only two
> tests from the whole suite.
> {quote}As for the two tests starting nine nodes each,
> {{test_network_topology_strategy}} and
> {{{}test_network_topology_strategy_each_quorum{}}}, I think they were added
> by CASSANDRA-10584 to test multi-DC consistency levels. I wonder if they
> actually need to start 3 data centers with 3 nodes each. Would two data
> centers with 3 nodes be enough to test multi-DC consistency levels with
> multiple data centers? 6 nodes should be doable on Circle with the current
> resources.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18130) Log hardware and container params during test runs to help troubleshoot intermittent failures
[
https://issues.apache.org/jira/browse/CASSANDRA-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683591#comment-17683591
]
Michael Semb Wever commented on CASSANDRA-18130:
brain-fart. this patch only logs on the built-in where the pipeline runs, not
the stage jobs. the patch needs to go in cassandra-builds/jenkins-dsl/
> Log hardware and container params during test runs to help troubleshoot
> intermittent failures
> -
>
> Key: CASSANDRA-18130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18130
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/java, Test/dtest/python, Test/unit
>Reporter: Josh McKenzie
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
>
> {color:#00}We’ve long had flakiness in our containerized ASF CI
> environment that we don’t see in circleci. The environment itself is both
> containerized and heterogenous, so there are differences in both the hardware
> environment and the software environment in which it executes. For reference,
> see:
> [https://github.com/apache/cassandra-builds/blob/trunk/ASF-jenkins-agents.md#current-agents]{color}
> {color:#00} {color}
> {color:#00}We should log a variety of hardware, container, and software
> environment details to help get to the bottom of where some test failures may
> be occurring. As we don’t have shell access to the machines it’ll be easier
> to have this information logged / retrieved during test runs than to try and
> profile each host independently.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (4c5d246f3 -> 22c4f960e)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 4c5d246f3 generate docs for 11d2d4b3 new 22c4f960e generate docs for 11d2d4b3 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (4c5d246f3) \ N -- N -- N refs/heads/asf-staging (22c4f960e) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4970898 -> 4970898 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15254) Allow UPDATE on settings virtual table to change running configurations
[ https://issues.apache.org/jira/browse/CASSANDRA-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683585#comment-17683585 ] David Capwell commented on CASSANDRA-15254: --- so PR was thrown up, marking myself as reviewer for when its ready > Allow UPDATE on settings virtual table to change running configurations > --- > > Key: CASSANDRA-15254 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15254 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Virtual Tables >Reporter: Chris Lohfink >Assignee: Maxim Muzafarov >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > Allow using UPDATE on the system_views.settings virtual table to update > configs at runtime for the equivalent of the dispersed JMX > attributes/operations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15254) Allow UPDATE on settings virtual table to change running configurations
[ https://issues.apache.org/jira/browse/CASSANDRA-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15254: -- Reviewers: David Capwell > Allow UPDATE on settings virtual table to change running configurations > --- > > Key: CASSANDRA-15254 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15254 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Virtual Tables >Reporter: Chris Lohfink >Assignee: Maxim Muzafarov >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > Allow using UPDATE on the system_views.settings virtual table to update > configs at runtime for the equivalent of the dispersed JMX > attributes/operations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18225) CEP-15: (C*) Accord adds a accord.messages.Defer which isn't supported for persisting, causing the operation to fail
David Capwell created CASSANDRA-18225:
-
Summary: CEP-15: (C*) Accord adds a accord.messages.Defer which
isn't supported for persisting, causing the operation to fail
Key: CASSANDRA-18225
URL: https://issues.apache.org/jira/browse/CASSANDRA-18225
Project: Cassandra
Issue Type: Bug
Components: Accord
Reporter: David Capwell
See
https://app.circleci.com/pipelines/github/maedhroz/cassandra/688/workflows/84cb0f28-d010-43c6-97cb-55af4a31a3ce/jobs/6554/tests#failed-test-0
{code}
ERROR [CommandStore[25]:1] node1 CS:[25] OP:0xf01a5a4b 2023-02-02 19:46:44,701
Operation AsyncOperation{RUNNING}-0xf01a5a4b failed
java.lang.RuntimeException: Unhandled non-transient listener:
accord.messages.Defer@3d36718e
at
org.apache.cassandra.service.accord.AccordCommand.maybeWrapListener(AccordCommand.java:711)
at
org.apache.cassandra.service.accord.AccordCommand.addListener(AccordCommand.java:717)
at accord.messages.Defer.add(Defer.java:67)
at accord.messages.Commit.apply$unsync(Commit.java:163)
at accord.messages.Commit.apply(Commit.java)
at accord.messages.Commit.apply(Commit.java:40)
at
org.apache.cassandra.service.accord.async.AsyncOperation$ForFunction.apply(AsyncOperation.java:239)
at
org.apache.cassandra.service.accord.async.AsyncOperation$ForFunction.apply(AsyncOperation.java:226)
at
org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
at
org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
at
org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
at
org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18224) CEP-15 (C*) No commandsForKey in context for key PartitionKey
David Capwell created CASSANDRA-18224:
-
Summary: CEP-15 (C*) No commandsForKey in context for key
PartitionKey
Key: CASSANDRA-18224
URL: https://issues.apache.org/jira/browse/CASSANDRA-18224
Project: Cassandra
Issue Type: Bug
Components: Accord
Reporter: David Capwell
See
https://app.circleci.com/pipelines/github/maedhroz/cassandra/688/workflows/84cb0f28-d010-43c6-97cb-55af4a31a3ce/jobs/6554/tests#failed-test-0
{code}
ERROR [CommandStore[25]:1] node1 CS:[25] OP:0xe17e624f 2023-02-02 19:46:44,764
Operation AsyncOperation{RUNNING}-0xe17e624f failed
java.lang.IllegalArgumentException: No commandsForKey in context for key
PartitionKey{tableId=dbb35709-2b11-38b3-9fcb-9a197512080e,
key=DecoratedKey(5414964737888519087, 242cad05)}
at
org.apache.cassandra.service.accord.AccordCommandStore.getCommandsForKeyInternal(AccordCommandStore.java:442)
at
org.apache.cassandra.service.accord.AccordCommandStore$SafeAccordCommandStore.commandsForKey(AccordCommandStore.java:204)
at
org.apache.cassandra.service.accord.AccordCommandStore$SafeAccordCommandStore.lambda$register$1(AccordCommandStore.java:190)
at accord.primitives.Routables$Helper.foldl(Routables.java:282)
at accord.primitives.Routables.foldl(Routables.java:134)
at
org.apache.cassandra.service.accord.AccordCommandStore$SafeAccordCommandStore.register(AccordCommandStore.java:190)
at accord.local.Command.set(Command.java:1015)
at accord.local.Command.commit(Command.java:377)
at accord.messages.Commit.apply$unsync(Commit.java:152)
at accord.messages.Commit.apply(Commit.java)
at accord.messages.Commit.apply(Commit.java:40)
at
org.apache.cassandra.service.accord.async.AsyncOperation$ForFunction.apply(AsyncOperation.java:239)
at
org.apache.cassandra.service.accord.async.AsyncOperation$ForFunction.apply(AsyncOperation.java:226)
at
org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
at
org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
at
org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
at
org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cep-15-accord updated: Ninja: .build/include-accord.sh was trying to do pull --rebase on a SHA and fail as git doesnt know how to do that. Switched to git fetch then checkout SHA
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch cep-15-accord
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cep-15-accord by this push:
new a052e062d2 Ninja: .build/include-accord.sh was trying to do pull
--rebase on a SHA and fail as git doesnt know how to do that. Switched to git
fetch then checkout SHA
a052e062d2 is described below
commit a052e062d22c3363dfd12f9cf35efe278fe194a7
Author: David Capwell
AuthorDate: Thu Feb 2 14:01:33 2023 -0800
Ninja: .build/include-accord.sh was trying to do pull --rebase on a SHA and
fail as git doesnt know how to do that. Switched to git fetch then checkout SHA
---
.build/include-accord.sh | 24 +---
1 file changed, 9 insertions(+), 15 deletions(-)
diff --git a/.build/include-accord.sh b/.build/include-accord.sh
index a1e3a580af..5609ea0585 100755
--- a/.build/include-accord.sh
+++ b/.build/include-accord.sh
@@ -25,31 +25,25 @@ set -o nounset
bin="$(cd "$(dirname "$0")" > /dev/null; pwd)"
accord_repo='https://github.com/apache/cassandra-accord.git'
-accord_branch='0cc9e273b2eaa37d82a1ae1ac2681aec65aa0f6d'
+accord_sha='0cc9e273b2eaa37d82a1ae1ac2681aec65aa0f6d'
accord_src="$bin/cassandra-accord"
-checkout() {
- cd "$accord_src"
-git checkout "$accord_branch"
-echo "$accord_branch" > .BRANCH
- cd -
-}
-
_main() {
# have we already cloned?
if [[ ! -e "$accord_src" ]] || [[ $(cat "$accord_src/.REPO" || true) !=
"$accord_repo" ]]; then
rm -rf "$accord_src" || true
git clone "$accord_repo" "$accord_src"
echo "$accord_repo" > "$accord_src/.REPO"
-checkout
- fi
- if [[ $(cat "$accord_src"/.BRANCH || true) != "$accord_branch" ]]; then
-checkout
fi
cd "$accord_src"
- # are there changes?
- git pull --rebase origin "$accord_branch"
- if [[ $(git rev-parse HEAD) != $(cat .SHA || true) ]]; then
+ # switch to target SHA
+ git fetch origin # check for changes
+ local current_sha
+ current_sha="$(git rev-parse HEAD)"
+ if [[ "$current_sha" != "$accord_sha" ]]; then
+git checkout "$accord_sha"
+ fi
+ if [[ "$accord_sha" != $(cat .SHA || true) ]]; then
./gradlew clean install -x test -x rat
git rev-parse HEAD > .SHA
fi
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18204) CEP-15: (C*) Add git submodule for Accord
[
https://issues.apache.org/jira/browse/CASSANDRA-18204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683517#comment-17683517
]
David Capwell edited comment on CASSANDRA-18204 at 2/2/23 9:44 PM:
---
To avoid cache issues when testing, I am doing the following
{code}
cd /tmp
git clone --depth 1 --branch accord-submodules
https://github.com/dcapwell/cassandra.git deleteme
cd deleteme
rm -rf ~/.m2/repository/org/apache/cassandra/cassandra-accord/
{code}
This lets me make sure a fresh copy "just works", and also I can delete .git to
simulator source tarball (can always build source tar, but this takes a while)
To simulate source tar, can do
{code}
git clone --depth 1 --branch accord-submodules
https://github.com/dcapwell/cassandra.git deleteme
cd deleteme/
git submodule update --init
rm -rf .git
rm -rf ~/.m2/repository/org/apache/cassandra/cassandra-accord/
ant jar
{code}
This does not mean I am not testing source tar, its just a quick way when I
make changes to test early
was (Author: dcapwell):
To avoid cache issues when testing, I am doing the following
{code}
cd /tmp
git clone --depth 1 --branch accord-submodules
https://github.com/dcapwell/cassandra.git deleteme
cd deleteme
{code}
This lets me make sure a fresh copy "just works", and also I can delete .git to
simulator source tarball (can always build source tar, but this takes a while)
To simulate source tar, can do
{code}
git clone --depth 1 --branch accord-submodules
https://github.com/dcapwell/cassandra.git deleteme
cd deleteme/
git submodule update --init
rm -rf .git
ant jar
{code}
This does not mean I am not testing source tar, its just a quick way when I
make changes to test early
> CEP-15: (C*) Add git submodule for Accord
> -
>
> Key: CASSANDRA-18204
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18204
> Project: Cassandra
> Issue Type: Task
> Components: Accord
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> As talked about in dev@ thread "Intra-project dependencies”, we talked about
> adding git submodules but before doing this had to work out a few issues
> first; this ticket is to track this work.
> Goals
> * when checking out an older commit, or pulling in newer commits, the
> submodule should also be updated automatically
> * release artifact must include the submodule and must be able to build
> without issue
> * build.xml must be updated to build the submodule
> * build.xml must be updated to release the submodule jar
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18218) OOM while running ShortAccordSimulationTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-18218:
Authors: Benedict Elliott Smith (was: Caleb Rackliffe)
> OOM while running ShortAccordSimulationTest
> ---
>
> Key: CASSANDRA-18218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18218
> Project: Cassandra
> Issue Type: Bug
> Components: Accord, Test/burn
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: simulator
> Fix For: NA
>
> Attachments: java_pid73497.hprof.zip, java_pid75762.hprof.zip
>
>
> {{ShortAccordSimulationTest}} seems to consistently run out of heap when run
> locally. This is true even when we scale the number of threads and duration
> down...
> {noformat}
> AccordSimulationRunner.main(new String[] { "run", "-n", "3", "-t", "10",
> "--cluster-action-limit", "-1", "-c", "2", "-s", "10"});
> {noformat}
> {noformat}
> ERROR [CommandStore[0]:1] node1 CS:[0] OP:0x8fb4f8f1 2023-01-31 17:16:27,143
> Operation AsyncOperation{RUNNING}-0x8fb4f8f1 failed
> java.lang.OutOfMemoryError: Java heap space
>at java.util.Arrays.copyOf(Arrays.java:3181)
>at accord.local.Command$NotifyWaitingOn.push(Command.java:794)
>at accord.local.Command$NotifyWaitingOn.accept(Command.java:757)
>at accord.local.Command.maybeExecute(Command.java:600)
>at accord.local.Command.onChange(Command.java:546)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy.lambda$onChange$0(ListenerProxy.java:148)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy$$Lambda$4606/1457728285.accept(Unknown
> Source)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:261)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:248)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
>at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>at
> org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus$$Lambda$768/827906088.run(Unknown
> Source)
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>at java.lang.Thread.run(Thread.java:748)
> {noformat}
> JVM args make it seem like we’re passing both a 1 GiB and 8 GiB heap size,
> although that doesn’t seem to have any bearing on the result. Setting only 8
> GiB just takes longer to hit the same problem.
> {noformat}
> INFO [isolatedExecutor:2] node1 2023-01-31 17:15:56,940 JVM Arguments:
> [-Dstorage-config=/Users/maedhroz/Forks/cassandra/test/conf,
> -Djava.awt.headless=true,
> -javaagent:/Users/maedhroz/Forks/cassandra/lib/jamm-0.3.2.jar, -ea,
> -Djava.io.tmpdir=/var/folders/4d/zfjs7m7s6x5_l93k33r5k668gn/T/,
> -Dcassandra.debugrefcount=true, -Xss384k, -XX:SoftRefLRUPolicyMSPerMB=0,
> -XX:HeapDumpPath=build/test,
> -Dcassandra.test.driver.connection_timeout_ms=1,
> -Dcassandra.test.driver.read_timeout_ms=24000,
> -Dcassandra.memtable_row_overhead_computation_step=100,
> -Dcassandra.test.use_prepared=true,
> -Dcassandra.test.sstableformatdevelopment=true,
> -Djava.security.egd=file:/dev/urandom, -Dcassandra.testtag=,
> -Dcassandra.keepBriefBrief=${cassandra.keepBriefBrief},
> -Dcassandra.strict.runtime.checks=true,
> -Dcassandra.reads.thresholds.coordinator.defensive_checks_enabled=true,
> -DQT_SHRINKS=0, -Dlogback.configurationFile=test/conf/logback-simulator.xml,
> -Dcassandra.ring_delay_ms=1, -Dcassandra.tolerate_sstable_size=true,
> -Dcassandra.skip_sync=true, -Dcassandra.debugrefcount=false,
> -Dcassandra.test.simulator.determinismcheck=strict,
> -Dcassandra.test.simulator.print_asm=none,
> -javaagent:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-asm.jar,
>
> -Xbootclasspath/a:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-bootstrap.jar,
> -XX:ActiveProcessorCount=4, -XX:-TieredCompilation,
> -XX:-BackgroundCompilation, -XX:CICompilerCount=1,
> -XX:Tier4CompileThreshold=1000, -XX:ReservedCodeCacheSize=256M, -Xmx8G,
> -Xmx1024m]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail:
[jira] [Updated] (CASSANDRA-18218) OOM while running ShortAccordSimulationTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-18218:
Since Version: NA
Source Control Link:
https://github.com/apache/cassandra-accord/commit/0cc9e273b2eaa37d82a1ae1ac2681aec65aa0f6d
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed to Accord
[here|https://github.com/apache/cassandra-accord/commit/0cc9e273b2eaa37d82a1ae1ac2681aec65aa0f6d]
and C*
[here|https://github.com/apache/cassandra/commit/7a85178a3d081d36c4c3de55d4a49ec368cc8c20].
> OOM while running ShortAccordSimulationTest
> ---
>
> Key: CASSANDRA-18218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18218
> Project: Cassandra
> Issue Type: Bug
> Components: Accord, Test/burn
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: simulator
> Fix For: NA
>
> Attachments: java_pid73497.hprof.zip, java_pid75762.hprof.zip
>
>
> {{ShortAccordSimulationTest}} seems to consistently run out of heap when run
> locally. This is true even when we scale the number of threads and duration
> down...
> {noformat}
> AccordSimulationRunner.main(new String[] { "run", "-n", "3", "-t", "10",
> "--cluster-action-limit", "-1", "-c", "2", "-s", "10"});
> {noformat}
> {noformat}
> ERROR [CommandStore[0]:1] node1 CS:[0] OP:0x8fb4f8f1 2023-01-31 17:16:27,143
> Operation AsyncOperation{RUNNING}-0x8fb4f8f1 failed
> java.lang.OutOfMemoryError: Java heap space
>at java.util.Arrays.copyOf(Arrays.java:3181)
>at accord.local.Command$NotifyWaitingOn.push(Command.java:794)
>at accord.local.Command$NotifyWaitingOn.accept(Command.java:757)
>at accord.local.Command.maybeExecute(Command.java:600)
>at accord.local.Command.onChange(Command.java:546)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy.lambda$onChange$0(ListenerProxy.java:148)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy$$Lambda$4606/1457728285.accept(Unknown
> Source)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:261)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:248)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
>at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>at
> org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus$$Lambda$768/827906088.run(Unknown
> Source)
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>at java.lang.Thread.run(Thread.java:748)
> {noformat}
> JVM args make it seem like we’re passing both a 1 GiB and 8 GiB heap size,
> although that doesn’t seem to have any bearing on the result. Setting only 8
> GiB just takes longer to hit the same problem.
> {noformat}
> INFO [isolatedExecutor:2] node1 2023-01-31 17:15:56,940 JVM Arguments:
> [-Dstorage-config=/Users/maedhroz/Forks/cassandra/test/conf,
> -Djava.awt.headless=true,
> -javaagent:/Users/maedhroz/Forks/cassandra/lib/jamm-0.3.2.jar, -ea,
> -Djava.io.tmpdir=/var/folders/4d/zfjs7m7s6x5_l93k33r5k668gn/T/,
> -Dcassandra.debugrefcount=true, -Xss384k, -XX:SoftRefLRUPolicyMSPerMB=0,
> -XX:HeapDumpPath=build/test,
> -Dcassandra.test.driver.connection_timeout_ms=1,
> -Dcassandra.test.driver.read_timeout_ms=24000,
> -Dcassandra.memtable_row_overhead_computation_step=100,
> -Dcassandra.test.use_prepared=true,
> -Dcassandra.test.sstableformatdevelopment=true,
> -Djava.security.egd=file:/dev/urandom, -Dcassandra.testtag=,
> -Dcassandra.keepBriefBrief=${cassandra.keepBriefBrief},
> -Dcassandra.strict.runtime.checks=true,
> -Dcassandra.reads.thresholds.coordinator.defensive_checks_enabled=true,
> -DQT_SHRINKS=0, -Dlogback.configurationFile=test/conf/logback-simulator.xml,
> -Dcassandra.ring_delay_ms=1, -Dcassandra.tolerate_sstable_size=true,
> -Dcassandra.skip_sync=true, -Dcassandra.debugrefcount=false,
> -Dcassandra.test.simulator.determinismcheck=strict,
> -Dcassandra.test.simulator.print_asm=none,
> -javaagent:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-asm.jar,
>
> -Xbootclasspath/a:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-bootstrap.jar,
>
[cassandra] branch cep-15-accord updated: use equals() rather than reference equality for excluding ourselves from the dependency builder
This is an automated email from the ASF dual-hosted git repository.
maedhroz pushed a commit to branch cep-15-accord
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cep-15-accord by this push:
new 7a85178a3d use equals() rather than reference equality for excluding
ourselves from the dependency builder
7a85178a3d is described below
commit 7a85178a3d081d36c4c3de55d4a49ec368cc8c20
Author: Caleb Rackliffe
AuthorDate: Thu Feb 2 11:43:47 2023 -0600
use equals() rather than reference equality for excluding ourselves from
the dependency builder
patch by Benedict; reviewed by Caleb Rackliffe for CASSANDRA-18218
---
.build/include-accord.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.build/include-accord.sh b/.build/include-accord.sh
index 84bf993d32..a1e3a580af 100755
--- a/.build/include-accord.sh
+++ b/.build/include-accord.sh
@@ -25,7 +25,7 @@ set -o nounset
bin="$(cd "$(dirname "$0")" > /dev/null; pwd)"
accord_repo='https://github.com/apache/cassandra-accord.git'
-accord_branch='1230eceb077c928123e6bf848a103964fe90c9f7'
+accord_branch='0cc9e273b2eaa37d82a1ae1ac2681aec65aa0f6d'
accord_src="$bin/cassandra-accord"
checkout() {
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-accord] branch trunk updated: use equals() rather than reference equality for excluding ourselves from the dependency builder
This is an automated email from the ASF dual-hosted git repository.
maedhroz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-accord.git
The following commit(s) were added to refs/heads/trunk by this push:
new 0cc9e27 use equals() rather than reference equality for excluding
ourselves from the dependency builder
0cc9e27 is described below
commit 0cc9e273b2eaa37d82a1ae1ac2681aec65aa0f6d
Author: Caleb Rackliffe
AuthorDate: Thu Feb 2 11:33:30 2023 -0600
use equals() rather than reference equality for excluding ourselves from
the dependency builder
patch by Benedict; reviewed by Caleb Rackliffe for CASSANDRA-18218
---
accord-core/src/main/java/accord/messages/PreAccept.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/accord-core/src/main/java/accord/messages/PreAccept.java
b/accord-core/src/main/java/accord/messages/PreAccept.java
index 9b171f0..8426cb7 100644
--- a/accord-core/src/main/java/accord/messages/PreAccept.java
+++ b/accord-core/src/main/java/accord/messages/PreAccept.java
@@ -232,7 +232,7 @@ public class PreAccept extends
WithUnsynced
commandStore.mapReduce(keys, ranges, testKind, STARTED_BEFORE,
executeAt, ANY_DEPS, null, null, null,
(keyOrRange, testTxnId, testExecuteAt, in) -> {
// TODO (easy, efficiency): either pass txnId as parameter
or encode this behaviour in a specialised builder to avoid extra allocations
-if (testTxnId != txnId)
+if (!testTxnId.equals(txnId))
in.add(keyOrRange, testTxnId);
return in;
}, builder, null);
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18218) OOM while running ShortAccordSimulationTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-18218:
Status: Ready to Commit (was: Review In Progress)
The good news is that the repeat job doesn't surface this OOM now. The bad news
is that it surfaces other problems we knew about before CASSANDRA-18174. In any
case, time to commit.
> OOM while running ShortAccordSimulationTest
> ---
>
> Key: CASSANDRA-18218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18218
> Project: Cassandra
> Issue Type: Bug
> Components: Accord, Test/burn
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: simulator
> Fix For: NA
>
> Attachments: java_pid73497.hprof.zip, java_pid75762.hprof.zip
>
>
> {{ShortAccordSimulationTest}} seems to consistently run out of heap when run
> locally. This is true even when we scale the number of threads and duration
> down...
> {noformat}
> AccordSimulationRunner.main(new String[] { "run", "-n", "3", "-t", "10",
> "--cluster-action-limit", "-1", "-c", "2", "-s", "10"});
> {noformat}
> {noformat}
> ERROR [CommandStore[0]:1] node1 CS:[0] OP:0x8fb4f8f1 2023-01-31 17:16:27,143
> Operation AsyncOperation{RUNNING}-0x8fb4f8f1 failed
> java.lang.OutOfMemoryError: Java heap space
>at java.util.Arrays.copyOf(Arrays.java:3181)
>at accord.local.Command$NotifyWaitingOn.push(Command.java:794)
>at accord.local.Command$NotifyWaitingOn.accept(Command.java:757)
>at accord.local.Command.maybeExecute(Command.java:600)
>at accord.local.Command.onChange(Command.java:546)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy.lambda$onChange$0(ListenerProxy.java:148)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy$$Lambda$4606/1457728285.accept(Unknown
> Source)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:261)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:248)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
>at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>at
> org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus$$Lambda$768/827906088.run(Unknown
> Source)
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>at java.lang.Thread.run(Thread.java:748)
> {noformat}
> JVM args make it seem like we’re passing both a 1 GiB and 8 GiB heap size,
> although that doesn’t seem to have any bearing on the result. Setting only 8
> GiB just takes longer to hit the same problem.
> {noformat}
> INFO [isolatedExecutor:2] node1 2023-01-31 17:15:56,940 JVM Arguments:
> [-Dstorage-config=/Users/maedhroz/Forks/cassandra/test/conf,
> -Djava.awt.headless=true,
> -javaagent:/Users/maedhroz/Forks/cassandra/lib/jamm-0.3.2.jar, -ea,
> -Djava.io.tmpdir=/var/folders/4d/zfjs7m7s6x5_l93k33r5k668gn/T/,
> -Dcassandra.debugrefcount=true, -Xss384k, -XX:SoftRefLRUPolicyMSPerMB=0,
> -XX:HeapDumpPath=build/test,
> -Dcassandra.test.driver.connection_timeout_ms=1,
> -Dcassandra.test.driver.read_timeout_ms=24000,
> -Dcassandra.memtable_row_overhead_computation_step=100,
> -Dcassandra.test.use_prepared=true,
> -Dcassandra.test.sstableformatdevelopment=true,
> -Djava.security.egd=file:/dev/urandom, -Dcassandra.testtag=,
> -Dcassandra.keepBriefBrief=${cassandra.keepBriefBrief},
> -Dcassandra.strict.runtime.checks=true,
> -Dcassandra.reads.thresholds.coordinator.defensive_checks_enabled=true,
> -DQT_SHRINKS=0, -Dlogback.configurationFile=test/conf/logback-simulator.xml,
> -Dcassandra.ring_delay_ms=1, -Dcassandra.tolerate_sstable_size=true,
> -Dcassandra.skip_sync=true, -Dcassandra.debugrefcount=false,
> -Dcassandra.test.simulator.determinismcheck=strict,
> -Dcassandra.test.simulator.print_asm=none,
> -javaagent:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-asm.jar,
>
> -Xbootclasspath/a:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-bootstrap.jar,
> -XX:ActiveProcessorCount=4, -XX:-TieredCompilation,
> -XX:-BackgroundCompilation, -XX:CICompilerCount=1,
> -XX:Tier4CompileThreshold=1000, -XX:ReservedCodeCacheSize=256M, -Xmx8G,
> -Xmx1024m]
> {noformat}
--
[jira] [Commented] (CASSANDRA-18051) Optimize test_network_topology_strategy and test_network_topology_strategy_each_quorum for large containers in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683556#comment-17683556
]
Ekaterina Dimitrova commented on CASSANDRA-18051:
-
{quote}Yep, fine with me.
{quote}
I will post it later today as part of the final patch for CASSANDRA-18001.
> Optimize test_network_topology_strategy and
> test_network_topology_strategy_each_quorum for large containers in CircleCI
> ---
>
> Key: CASSANDRA-18051
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18051
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>
> As pointed in CASSANDRA-18001, there are two Python Large DTests that fail
> because of not enough resources with large containers. [~adelapena] looked
> into them and suggested that the tests themselves can be optimized for
> running on Large containers instead of using XLarge containers for only two
> tests from the whole suite.
> {quote}As for the two tests starting nine nodes each,
> {{test_network_topology_strategy}} and
> {{{}test_network_topology_strategy_each_quorum{}}}, I think they were added
> by CASSANDRA-10584 to test multi-DC consistency levels. I wonder if they
> actually need to start 3 data centers with 3 nodes each. Would two data
> centers with 3 nodes be enough to test multi-DC consistency levels with
> multiple data centers? 6 nodes should be doable on Circle with the current
> resources.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18050) Update JNA
[ https://issues.apache.org/jira/browse/CASSANDRA-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683555#comment-17683555 ] Ekaterina Dimitrova commented on CASSANDRA-18050: - I'm thinking probably of updating it again with snappy as before. I think [~mck] mentioned something about the need to update snappy anyway, if I am not mistaken > Update JNA > -- > > Key: CASSANDRA-18050 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18050 > Project: Cassandra > Issue Type: Task >Reporter: Ekaterina Dimitrova >Priority: Normal > > JNA needs to be updated for JDK 17 support. There was an update pre-4.0 but > not to the latest version (confirmed with the people involved, there was no > particular reason not to update to the latest version, just missed on rebase > of older PR) > More details to come, opening this as a placeholder for now -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17949) Update CCM post CASSANDRA-17379
[ https://issues.apache.org/jira/browse/CASSANDRA-17949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683540#comment-17683540 ] Ekaterina Dimitrova edited comment on CASSANDRA-17949 at 2/2/23 7:08 PM: - Committed [[4609ac4..c51c655|https://github.com/riptano/ccm/commit/c51c655973e6873f272f77b95185235002a2c0a9]4609ac4..c51c655], also fixed the formatting a bit as it was off (too long lines). Thanks. I didn't retag CCM as it is only a README note was (Author: e.dimitrova): Committed [[4609ac4..c51c655|https://github.com/riptano/ccm/commit/c51c655973e6873f272f77b95185235002a2c0a9]4609ac4..c51c655], also fixed the formatting a bit as it was off (too long lines). Thanks. > Update CCM post CASSANDRA-17379 > --- > > Key: CASSANDRA-17949 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17949 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > > We updated in CASSANDRA-15234 CCM updateconf to enable it to handle > overloading of parameters Python upgrade tests. > This also allows us not to update for 4.1 the Python DTests but exercise the > backward compatibility we have in C* in our tests. > Now CASSANDRA-17379 added a few flags to opt in or out for overloading of > parameters. > The default points to -Dcassandra.allow_new_old_config_keys=false but this > does not affect our tests as CCM handles the overloading in the yaml. > But this is confusing for users and might lead to issues. > We need to bring on par one way or another ccm with the new flags introduced > in CASSANDRA-17379 > *Until then the recommendation is to use > `-Dcassandra.allow_new_old_config_keys=false` and > `-Dcassandra.allow_duplicate_config_keys=false` to prohibit any kind of > overloading when using CCM master and CCM released to prevent any confusion.* -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17949) Update CCM post CASSANDRA-17379
[ https://issues.apache.org/jira/browse/CASSANDRA-17949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683540#comment-17683540 ] Ekaterina Dimitrova commented on CASSANDRA-17949: - Committed [[4609ac4..c51c655|https://github.com/riptano/ccm/commit/c51c655973e6873f272f77b95185235002a2c0a9]4609ac4..c51c655], also fixed the formatting a bit as it was off (too long lines). Thanks. > Update CCM post CASSANDRA-17379 > --- > > Key: CASSANDRA-17949 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17949 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > > We updated in CASSANDRA-15234 CCM updateconf to enable it to handle > overloading of parameters Python upgrade tests. > This also allows us not to update for 4.1 the Python DTests but exercise the > backward compatibility we have in C* in our tests. > Now CASSANDRA-17379 added a few flags to opt in or out for overloading of > parameters. > The default points to -Dcassandra.allow_new_old_config_keys=false but this > does not affect our tests as CCM handles the overloading in the yaml. > But this is confusing for users and might lead to issues. > We need to bring on par one way or another ccm with the new flags introduced > in CASSANDRA-17379 > *Until then the recommendation is to use > `-Dcassandra.allow_new_old_config_keys=false` and > `-Dcassandra.allow_duplicate_config_keys=false` to prohibit any kind of > overloading when using CCM master and CCM released to prevent any confusion.* -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17199) Provide summary of failed SessionInfo's in StreamResultFuture
[
https://issues.apache.org/jira/browse/CASSANDRA-17199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683538#comment-17683538
]
David Capwell commented on CASSANDRA-17199:
---
Have 1 comment still not addressed, but mostly LGTM. Will wait for [~djoshi]
review
> Provide summary of failed SessionInfo's in StreamResultFuture
> -
>
> Key: CASSANDRA-17199
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17199
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brendan Cicchi
>Assignee: Natnael Adere
>Priority: Normal
> Labels: AdventCalendar2021, lhf
> Fix For: 4.0.x
>
>
> Currently, we warn about the presence of one or more failed sessions existing
> in the final state and then an operator/user traces back through the logs to
> find any failed streams for troubleshooting.
> {code:java}
> private synchronized void maybeComplete()
> {
> if (finishedAllSessions())
> {
> StreamState finalState = getCurrentState();
> if (finalState.hasFailedSession())
> {
> logger.warn("[Stream #{}] Stream failed", planId);
> tryFailure(new StreamException(finalState, "Stream failed"));
> }
> else
> {
> logger.info("[Stream #{}] All sessions completed", planId);
> trySuccess(finalState);
> }
> }
> } {code}
> It would be helpful to log out a summary of the SessionInfo for each failed
> session since that should be accessible via the StreamState.
>
> This can be especially helpful for longer streaming operations like bootstrap
> where the failure could have been a long time back and all recent streams
> leading up to the warning actually are successful.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18218) OOM while running ShortAccordSimulationTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683518#comment-17683518
]
David Capwell edited comment on CASSANDRA-18218 at 2/2/23 6:30 PM:
---
patch LGTM, can we attempt to run the simulator in a repeat job so we see if
this doesn't come back?
was (Author: dcapwell):
patch LGTM, can we get circle ci runs? Can we attempt to run the simulator in
a repeat job so we see if this doesn't come back?
> OOM while running ShortAccordSimulationTest
> ---
>
> Key: CASSANDRA-18218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18218
> Project: Cassandra
> Issue Type: Bug
> Components: Accord, Test/burn
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: simulator
> Fix For: NA
>
> Attachments: java_pid73497.hprof.zip, java_pid75762.hprof.zip
>
>
> {{ShortAccordSimulationTest}} seems to consistently run out of heap when run
> locally. This is true even when we scale the number of threads and duration
> down...
> {noformat}
> AccordSimulationRunner.main(new String[] { "run", "-n", "3", "-t", "10",
> "--cluster-action-limit", "-1", "-c", "2", "-s", "10"});
> {noformat}
> {noformat}
> ERROR [CommandStore[0]:1] node1 CS:[0] OP:0x8fb4f8f1 2023-01-31 17:16:27,143
> Operation AsyncOperation{RUNNING}-0x8fb4f8f1 failed
> java.lang.OutOfMemoryError: Java heap space
>at java.util.Arrays.copyOf(Arrays.java:3181)
>at accord.local.Command$NotifyWaitingOn.push(Command.java:794)
>at accord.local.Command$NotifyWaitingOn.accept(Command.java:757)
>at accord.local.Command.maybeExecute(Command.java:600)
>at accord.local.Command.onChange(Command.java:546)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy.lambda$onChange$0(ListenerProxy.java:148)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy$$Lambda$4606/1457728285.accept(Unknown
> Source)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:261)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:248)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
>at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>at
> org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus$$Lambda$768/827906088.run(Unknown
> Source)
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>at java.lang.Thread.run(Thread.java:748)
> {noformat}
> JVM args make it seem like we’re passing both a 1 GiB and 8 GiB heap size,
> although that doesn’t seem to have any bearing on the result. Setting only 8
> GiB just takes longer to hit the same problem.
> {noformat}
> INFO [isolatedExecutor:2] node1 2023-01-31 17:15:56,940 JVM Arguments:
> [-Dstorage-config=/Users/maedhroz/Forks/cassandra/test/conf,
> -Djava.awt.headless=true,
> -javaagent:/Users/maedhroz/Forks/cassandra/lib/jamm-0.3.2.jar, -ea,
> -Djava.io.tmpdir=/var/folders/4d/zfjs7m7s6x5_l93k33r5k668gn/T/,
> -Dcassandra.debugrefcount=true, -Xss384k, -XX:SoftRefLRUPolicyMSPerMB=0,
> -XX:HeapDumpPath=build/test,
> -Dcassandra.test.driver.connection_timeout_ms=1,
> -Dcassandra.test.driver.read_timeout_ms=24000,
> -Dcassandra.memtable_row_overhead_computation_step=100,
> -Dcassandra.test.use_prepared=true,
> -Dcassandra.test.sstableformatdevelopment=true,
> -Djava.security.egd=file:/dev/urandom, -Dcassandra.testtag=,
> -Dcassandra.keepBriefBrief=${cassandra.keepBriefBrief},
> -Dcassandra.strict.runtime.checks=true,
> -Dcassandra.reads.thresholds.coordinator.defensive_checks_enabled=true,
> -DQT_SHRINKS=0, -Dlogback.configurationFile=test/conf/logback-simulator.xml,
> -Dcassandra.ring_delay_ms=1, -Dcassandra.tolerate_sstable_size=true,
> -Dcassandra.skip_sync=true, -Dcassandra.debugrefcount=false,
> -Dcassandra.test.simulator.determinismcheck=strict,
> -Dcassandra.test.simulator.print_asm=none,
> -javaagent:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-asm.jar,
>
> -Xbootclasspath/a:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-bootstrap.jar,
> -XX:ActiveProcessorCount=4, -XX:-TieredCompilation,
> -XX:-BackgroundCompilation, -XX:CICompilerCount=1,
>
[jira] [Updated] (CASSANDRA-18218) OOM while running ShortAccordSimulationTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-18218:
--
Reviewers: David Capwell, David Capwell
David Capwell, David Capwell (was: David Capwell)
Status: Review In Progress (was: Patch Available)
patch LGTM, can we get circle ci runs? Can we attempt to run the simulator in
a repeat job so we see if this doesn't come back?
> OOM while running ShortAccordSimulationTest
> ---
>
> Key: CASSANDRA-18218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18218
> Project: Cassandra
> Issue Type: Bug
> Components: Accord, Test/burn
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: simulator
> Fix For: NA
>
> Attachments: java_pid73497.hprof.zip, java_pid75762.hprof.zip
>
>
> {{ShortAccordSimulationTest}} seems to consistently run out of heap when run
> locally. This is true even when we scale the number of threads and duration
> down...
> {noformat}
> AccordSimulationRunner.main(new String[] { "run", "-n", "3", "-t", "10",
> "--cluster-action-limit", "-1", "-c", "2", "-s", "10"});
> {noformat}
> {noformat}
> ERROR [CommandStore[0]:1] node1 CS:[0] OP:0x8fb4f8f1 2023-01-31 17:16:27,143
> Operation AsyncOperation{RUNNING}-0x8fb4f8f1 failed
> java.lang.OutOfMemoryError: Java heap space
>at java.util.Arrays.copyOf(Arrays.java:3181)
>at accord.local.Command$NotifyWaitingOn.push(Command.java:794)
>at accord.local.Command$NotifyWaitingOn.accept(Command.java:757)
>at accord.local.Command.maybeExecute(Command.java:600)
>at accord.local.Command.onChange(Command.java:546)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy.lambda$onChange$0(ListenerProxy.java:148)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy$$Lambda$4606/1457728285.accept(Unknown
> Source)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:261)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:248)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
>at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>at
> org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus$$Lambda$768/827906088.run(Unknown
> Source)
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>at java.lang.Thread.run(Thread.java:748)
> {noformat}
> JVM args make it seem like we’re passing both a 1 GiB and 8 GiB heap size,
> although that doesn’t seem to have any bearing on the result. Setting only 8
> GiB just takes longer to hit the same problem.
> {noformat}
> INFO [isolatedExecutor:2] node1 2023-01-31 17:15:56,940 JVM Arguments:
> [-Dstorage-config=/Users/maedhroz/Forks/cassandra/test/conf,
> -Djava.awt.headless=true,
> -javaagent:/Users/maedhroz/Forks/cassandra/lib/jamm-0.3.2.jar, -ea,
> -Djava.io.tmpdir=/var/folders/4d/zfjs7m7s6x5_l93k33r5k668gn/T/,
> -Dcassandra.debugrefcount=true, -Xss384k, -XX:SoftRefLRUPolicyMSPerMB=0,
> -XX:HeapDumpPath=build/test,
> -Dcassandra.test.driver.connection_timeout_ms=1,
> -Dcassandra.test.driver.read_timeout_ms=24000,
> -Dcassandra.memtable_row_overhead_computation_step=100,
> -Dcassandra.test.use_prepared=true,
> -Dcassandra.test.sstableformatdevelopment=true,
> -Djava.security.egd=file:/dev/urandom, -Dcassandra.testtag=,
> -Dcassandra.keepBriefBrief=${cassandra.keepBriefBrief},
> -Dcassandra.strict.runtime.checks=true,
> -Dcassandra.reads.thresholds.coordinator.defensive_checks_enabled=true,
> -DQT_SHRINKS=0, -Dlogback.configurationFile=test/conf/logback-simulator.xml,
> -Dcassandra.ring_delay_ms=1, -Dcassandra.tolerate_sstable_size=true,
> -Dcassandra.skip_sync=true, -Dcassandra.debugrefcount=false,
> -Dcassandra.test.simulator.determinismcheck=strict,
> -Dcassandra.test.simulator.print_asm=none,
> -javaagent:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-asm.jar,
>
> -Xbootclasspath/a:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-bootstrap.jar,
> -XX:ActiveProcessorCount=4, -XX:-TieredCompilation,
> -XX:-BackgroundCompilation, -XX:CICompilerCount=1,
> -XX:Tier4CompileThreshold=1000,
[jira] [Comment Edited] (CASSANDRA-18204) CEP-15: (C*) Add git submodule for Accord
[
https://issues.apache.org/jira/browse/CASSANDRA-18204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683517#comment-17683517
]
David Capwell edited comment on CASSANDRA-18204 at 2/2/23 6:18 PM:
---
To avoid cache issues when testing, I am doing the following
{code}
cd /tmp
git clone --depth 1 --branch accord-submodules
https://github.com/dcapwell/cassandra.git deleteme
cd deleteme
{code}
This lets me make sure a fresh copy "just works", and also I can delete .git to
simulator source tarball (can always build source tar, but this takes a while)
To simulate source tar, can do
{code}
git clone --depth 1 --branch accord-submodules
https://github.com/dcapwell/cassandra.git deleteme
cd deleteme/
git submodule update --init
rm -rf .git
ant jar
{code}
This does not mean I am not testing source tar, its just a quick way when I
make changes to test early
was (Author: dcapwell):
To avoid cache issues when testing, I am doing the following
{code}
cd /tmp
git clone --depth 1 --branch accord-submodules
https://github.com/dcapwell/cassandra.git deleteme
cd deleteme
{code}
This lets me make sure a fresh copy "just works", and also I can delete .git to
simulator source tarball (can always build source tar, but this takes a while)
> CEP-15: (C*) Add git submodule for Accord
> -
>
> Key: CASSANDRA-18204
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18204
> Project: Cassandra
> Issue Type: Task
> Components: Accord
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> As talked about in dev@ thread "Intra-project dependencies”, we talked about
> adding git submodules but before doing this had to work out a few issues
> first; this ticket is to track this work.
> Goals
> * when checking out an older commit, or pulling in newer commits, the
> submodule should also be updated automatically
> * release artifact must include the submodule and must be able to build
> without issue
> * build.xml must be updated to build the submodule
> * build.xml must be updated to release the submodule jar
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18204) CEP-15: (C*) Add git submodule for Accord
[
https://issues.apache.org/jira/browse/CASSANDRA-18204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683517#comment-17683517
]
David Capwell commented on CASSANDRA-18204:
---
To avoid cache issues when testing, I am doing the following
{code}
cd /tmp
git clone --depth 1 --branch accord-submodules
https://github.com/dcapwell/cassandra.git deleteme
cd deleteme
{code}
This lets me make sure a fresh copy "just works", and also I can delete .git to
simulator source tarball (can always build source tar, but this takes a while)
> CEP-15: (C*) Add git submodule for Accord
> -
>
> Key: CASSANDRA-18204
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18204
> Project: Cassandra
> Issue Type: Task
> Components: Accord
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> As talked about in dev@ thread "Intra-project dependencies”, we talked about
> adding git submodules but before doing this had to work out a few issues
> first; this ticket is to track this work.
> Goals
> * when checking out an older commit, or pulling in newer commits, the
> submodule should also be updated automatically
> * release artifact must include the submodule and must be able to build
> without issue
> * build.xml must be updated to build the submodule
> * build.xml must be updated to release the submodule jar
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18218) OOM while running ShortAccordSimulationTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-18218:
Test and Documentation Plan: n/a (test fix)
Status: Patch Available (was: In Progress)
accord: [https://github.com/apache/cassandra-accord/pull/32]
C*: [https://github.com/apache/cassandra/pull/2132]
CircleCI:
[https://app.circleci.com/pipelines/github/maedhroz/cassandra?branch=CASSANDRA-18218]
> OOM while running ShortAccordSimulationTest
> ---
>
> Key: CASSANDRA-18218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18218
> Project: Cassandra
> Issue Type: Bug
> Components: Accord, Test/burn
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: simulator
> Fix For: NA
>
> Attachments: java_pid73497.hprof.zip, java_pid75762.hprof.zip
>
>
> {{ShortAccordSimulationTest}} seems to consistently run out of heap when run
> locally. This is true even when we scale the number of threads and duration
> down...
> {noformat}
> AccordSimulationRunner.main(new String[] { "run", "-n", "3", "-t", "10",
> "--cluster-action-limit", "-1", "-c", "2", "-s", "10"});
> {noformat}
> {noformat}
> ERROR [CommandStore[0]:1] node1 CS:[0] OP:0x8fb4f8f1 2023-01-31 17:16:27,143
> Operation AsyncOperation{RUNNING}-0x8fb4f8f1 failed
> java.lang.OutOfMemoryError: Java heap space
>at java.util.Arrays.copyOf(Arrays.java:3181)
>at accord.local.Command$NotifyWaitingOn.push(Command.java:794)
>at accord.local.Command$NotifyWaitingOn.accept(Command.java:757)
>at accord.local.Command.maybeExecute(Command.java:600)
>at accord.local.Command.onChange(Command.java:546)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy.lambda$onChange$0(ListenerProxy.java:148)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy$$Lambda$4606/1457728285.accept(Unknown
> Source)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:261)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:248)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
>at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>at
> org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus$$Lambda$768/827906088.run(Unknown
> Source)
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>at java.lang.Thread.run(Thread.java:748)
> {noformat}
> JVM args make it seem like we’re passing both a 1 GiB and 8 GiB heap size,
> although that doesn’t seem to have any bearing on the result. Setting only 8
> GiB just takes longer to hit the same problem.
> {noformat}
> INFO [isolatedExecutor:2] node1 2023-01-31 17:15:56,940 JVM Arguments:
> [-Dstorage-config=/Users/maedhroz/Forks/cassandra/test/conf,
> -Djava.awt.headless=true,
> -javaagent:/Users/maedhroz/Forks/cassandra/lib/jamm-0.3.2.jar, -ea,
> -Djava.io.tmpdir=/var/folders/4d/zfjs7m7s6x5_l93k33r5k668gn/T/,
> -Dcassandra.debugrefcount=true, -Xss384k, -XX:SoftRefLRUPolicyMSPerMB=0,
> -XX:HeapDumpPath=build/test,
> -Dcassandra.test.driver.connection_timeout_ms=1,
> -Dcassandra.test.driver.read_timeout_ms=24000,
> -Dcassandra.memtable_row_overhead_computation_step=100,
> -Dcassandra.test.use_prepared=true,
> -Dcassandra.test.sstableformatdevelopment=true,
> -Djava.security.egd=file:/dev/urandom, -Dcassandra.testtag=,
> -Dcassandra.keepBriefBrief=${cassandra.keepBriefBrief},
> -Dcassandra.strict.runtime.checks=true,
> -Dcassandra.reads.thresholds.coordinator.defensive_checks_enabled=true,
> -DQT_SHRINKS=0, -Dlogback.configurationFile=test/conf/logback-simulator.xml,
> -Dcassandra.ring_delay_ms=1, -Dcassandra.tolerate_sstable_size=true,
> -Dcassandra.skip_sync=true, -Dcassandra.debugrefcount=false,
> -Dcassandra.test.simulator.determinismcheck=strict,
> -Dcassandra.test.simulator.print_asm=none,
> -javaagent:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-asm.jar,
>
> -Xbootclasspath/a:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-bootstrap.jar,
> -XX:ActiveProcessorCount=4, -XX:-TieredCompilation,
> -XX:-BackgroundCompilation, -XX:CICompilerCount=1,
>
[jira] [Commented] (CASSANDRA-18219) Warning message on aggregation queries doesn't specify table name or aggregate type
[
https://issues.apache.org/jira/browse/CASSANDRA-18219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683513#comment-17683513
]
Brandon Williams commented on CASSANDRA-18219:
--
I don't see any reason to not put this in 4.0.x.
> Warning message on aggregation queries doesn't specify table name or
> aggregate type
> ---
>
> Key: CASSANDRA-18219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18219
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The existing aggregation query warning messages in cassandra.log from
> SelectStatement.java:
> _'[WARN] Aggregation query used without partition key'_
> {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys
> (IN restriction)'_
> are missing the helpful details which would assist users in quickly
> identifying the offending query. The table name and type of aggregation would
> be useful additions in the WARN message.
> In addition, following the example of the slow query logger and printing out
> the query string at DEBUG level would be especially helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18184) 'Maximum memory usage reached' chunk cache log message doesn't specify which cache is exhausted
[
https://issues.apache.org/jira/browse/CASSANDRA-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683511#comment-17683511
]
Brandon Williams commented on CASSANDRA-18184:
--
Committed, thanks.
> 'Maximum memory usage reached' chunk cache log message doesn't specify which
> cache is exhausted
> ---
>
> Key: CASSANDRA-18184
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18184
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Yong Jiang
>Priority: Normal
> Fix For: 4.0.8, 4.1.1, 4.2
>
> Attachments: 18184-trunk.txt
>
>
> With Cassandra 4.0.x, we are seeing this cassandra.log message very
> frequently on the majority of our Cassandra 4.0 clusters:
> _[INFO ] [epollEventLoopGroup-5-3] cluster_id=99 ip_address=127.0.0.50
> NoSpamLogger.java:92 - Maximum memory usage reached (128.000MiB), cannot
> allocate chunk of 8.000MiB_
> It took me several weeks to track down what it means, until I saw this
> start-up message
> _BufferPools.java:49 - Global buffer pool limit is 2.000GiB for
> chunk-cache and 128.000MiB for networking_
> This maximum memory usage warning would benefit from clarifying that its the
> {*}network cache{*}, not the *off-heap chunk* *cache* which is exhausted.
> With 'chunk cache' in both warning messages, they're too easily confused.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18184) 'Maximum memory usage reached' chunk cache log message doesn't specify which cache is exhausted
[
https://issues.apache.org/jira/browse/CASSANDRA-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18184:
-
Fix Version/s: 4.0.8
4.1.1
4.2
(was: 4.x)
(was: 4.0.x)
(was: 4.1.x)
Since Version: 4.0-alpha4
Source Control Link:
https://github.com/apache/cassandra/commit/0c58fbb8dd25beab4b4a81650be1ed0ec888ff66
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> 'Maximum memory usage reached' chunk cache log message doesn't specify which
> cache is exhausted
> ---
>
> Key: CASSANDRA-18184
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18184
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Yong Jiang
>Priority: Normal
> Fix For: 4.0.8, 4.1.1, 4.2
>
> Attachments: 18184-trunk.txt
>
>
> With Cassandra 4.0.x, we are seeing this cassandra.log message very
> frequently on the majority of our Cassandra 4.0 clusters:
> _[INFO ] [epollEventLoopGroup-5-3] cluster_id=99 ip_address=127.0.0.50
> NoSpamLogger.java:92 - Maximum memory usage reached (128.000MiB), cannot
> allocate chunk of 8.000MiB_
> It took me several weeks to track down what it means, until I saw this
> start-up message
> _BufferPools.java:49 - Global buffer pool limit is 2.000GiB for
> chunk-cache and 128.000MiB for networking_
> This maximum memory usage warning would benefit from clarifying that its the
> {*}network cache{*}, not the *off-heap chunk* *cache* which is exhausted.
> With 'chunk cache' in both warning messages, they're too easily confused.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated: Add cache type information for maximum memory usage warning message
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-4.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-4.0 by this push:
new 0c58fbb8dd Add cache type information for maximum memory usage warning
message
0c58fbb8dd is described below
commit 0c58fbb8dd25beab4b4a81650be1ed0ec888ff66
Author: yongj
AuthorDate: Wed Feb 1 07:52:35 2023 +
Add cache type information for maximum memory usage warning message
Patch by Yong Jiang; reviewed by brandonwilliams and smiklosovic for
CASSANDRA-18184
---
CHANGES.txt| 1 +
src/java/org/apache/cassandra/utils/memory/BufferPool.java | 4 ++--
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 5fa7bc481d..ae1135b049 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0.8
+ * Add cache type information for maximum memory usage warning message
(CASSANDRA-18184)
* Fix NPE in fqltool dump on null value (CASSANDRA-18113)
* Improve unit tests performance (CASSANDRA-17427)
* Connect to listen address when own broadcast address is requested
(CASSANDRA-18200)
diff --git a/src/java/org/apache/cassandra/utils/memory/BufferPool.java
b/src/java/org/apache/cassandra/utils/memory/BufferPool.java
index cfa588cacb..443e9d04a1 100644
--- a/src/java/org/apache/cassandra/utils/memory/BufferPool.java
+++ b/src/java/org/apache/cassandra/utils/memory/BufferPool.java
@@ -427,8 +427,8 @@ public class BufferPool
{
if (memoryUsageThreshold > 0)
{
-noSpamLogger.info("Maximum memory usage reached ({}),
cannot allocate chunk of {}",
- readableMemoryUsageThreshold,
READABLE_MACRO_CHUNK_SIZE);
+noSpamLogger.info("Maximum memory usage reached ({})
for {} buffer pool, cannot allocate chunk of {}",
+ readableMemoryUsageThreshold, name,
READABLE_MACRO_CHUNK_SIZE);
}
return null;
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.1' into trunk
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit a07c15d72d575845624746b828067ed8d2dc3d9d Merge: 1e685219da 209ba28b4d Author: Brandon Williams AuthorDate: Thu Feb 2 11:46:40 2023 -0600 Merge branch 'cassandra-4.1' into trunk CHANGES.txt| 1 + src/java/org/apache/cassandra/utils/memory/BufferPool.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (1e685219da -> a07c15d72d)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 1e685219da Add unit tests for per-row TTL and Timestamp usage in CQLSSTableWriter new 0c58fbb8dd Add cache type information for maximum memory usage warning message new 209ba28b4d Merge branch 'cassandra-4.0' into cassandra-4.1 new a07c15d72d Merge branch 'cassandra-4.1' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + src/java/org/apache/cassandra/utils/memory/BufferPool.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.0' into cassandra-4.1
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 209ba28b4dc5abb32d053d2d9d2605076163d32c Merge: bede10f36c 0c58fbb8dd Author: Brandon Williams AuthorDate: Thu Feb 2 11:46:31 2023 -0600 Merge branch 'cassandra-4.0' into cassandra-4.1 CHANGES.txt| 1 + src/java/org/apache/cassandra/utils/memory/BufferPool.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --cc CHANGES.txt index aeee3abf4d,ae1135b049..6d7c25f620 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,9 -1,5 +1,10 @@@ -4.0.8 +4.1.1 + * PaxosPrepare may add instances to the Electorate that are not in gossip (CASSANDRA-18194) + * Fix PAXOS2_COMMIT_AND_PREPARE_RSP serialisation AssertionError (CASSANDRA-18164) + * Streaming progress virtual table lock contention can trigger TCP_USER_TIMEOUT and fail streaming (CASSANDRA-18110) + * Fix perpetual load of denylist on read in cases where denylist can never be loaded (CASSANDRA-18116) +Merged from 4.0: + * Add cache type information for maximum memory usage warning message (CASSANDRA-18184) * Fix NPE in fqltool dump on null value (CASSANDRA-18113) * Improve unit tests performance (CASSANDRA-17427) * Connect to listen address when own broadcast address is requested (CASSANDRA-18200) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.1 updated (bede10f36c -> 209ba28b4d)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from bede10f36c Merge branch 'cassandra-4.0' into cassandra-4.1 new 0c58fbb8dd Add cache type information for maximum memory usage warning message new 209ba28b4d Merge branch 'cassandra-4.0' into cassandra-4.1 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + src/java/org/apache/cassandra/utils/memory/BufferPool.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18219) Warning message on aggregation queries doesn't specify table name or aggregate type
[
https://issues.apache.org/jira/browse/CASSANDRA-18219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683505#comment-17683505
]
Stefan Miklosovic commented on CASSANDRA-18219:
---
They are improving logging recently in CASSANDRA-18184 so I think this might to
4.0.x up as well.
> Warning message on aggregation queries doesn't specify table name or
> aggregate type
> ---
>
> Key: CASSANDRA-18219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18219
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The existing aggregation query warning messages in cassandra.log from
> SelectStatement.java:
> _'[WARN] Aggregation query used without partition key'_
> {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys
> (IN restriction)'_
> are missing the helpful details which would assist users in quickly
> identifying the offending query. The table name and type of aggregation would
> be useful additions in the WARN message.
> In addition, following the example of the slow query logger and printing out
> the query string at DEBUG level would be especially helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18184) 'Maximum memory usage reached' chunk cache log message doesn't specify which cache is exhausted
[
https://issues.apache.org/jira/browse/CASSANDRA-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18184:
--
Status: Ready to Commit (was: Review In Progress)
Straightforward patch! +1. Thanks everybody.
> 'Maximum memory usage reached' chunk cache log message doesn't specify which
> cache is exhausted
> ---
>
> Key: CASSANDRA-18184
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18184
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Yong Jiang
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
> Attachments: 18184-trunk.txt
>
>
> With Cassandra 4.0.x, we are seeing this cassandra.log message very
> frequently on the majority of our Cassandra 4.0 clusters:
> _[INFO ] [epollEventLoopGroup-5-3] cluster_id=99 ip_address=127.0.0.50
> NoSpamLogger.java:92 - Maximum memory usage reached (128.000MiB), cannot
> allocate chunk of 8.000MiB_
> It took me several weeks to track down what it means, until I saw this
> start-up message
> _BufferPools.java:49 - Global buffer pool limit is 2.000GiB for
> chunk-cache and 128.000MiB for networking_
> This maximum memory usage warning would benefit from clarifying that its the
> {*}network cache{*}, not the *off-heap chunk* *cache* which is exhausted.
> With 'chunk cache' in both warning messages, they're too easily confused.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18184) 'Maximum memory usage reached' chunk cache log message doesn't specify which cache is exhausted
[
https://issues.apache.org/jira/browse/CASSANDRA-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18184:
--
Reviewers: Brandon Williams, Stefan Miklosovic (was: Brandon Williams)
Status: Review In Progress (was: Needs Committer)
> 'Maximum memory usage reached' chunk cache log message doesn't specify which
> cache is exhausted
> ---
>
> Key: CASSANDRA-18184
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18184
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Yong Jiang
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
> Attachments: 18184-trunk.txt
>
>
> With Cassandra 4.0.x, we are seeing this cassandra.log message very
> frequently on the majority of our Cassandra 4.0 clusters:
> _[INFO ] [epollEventLoopGroup-5-3] cluster_id=99 ip_address=127.0.0.50
> NoSpamLogger.java:92 - Maximum memory usage reached (128.000MiB), cannot
> allocate chunk of 8.000MiB_
> It took me several weeks to track down what it means, until I saw this
> start-up message
> _BufferPools.java:49 - Global buffer pool limit is 2.000GiB for
> chunk-cache and 128.000MiB for networking_
> This maximum memory usage warning would benefit from clarifying that its the
> {*}network cache{*}, not the *off-heap chunk* *cache* which is exhausted.
> With 'chunk cache' in both warning messages, they're too easily confused.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18184) 'Maximum memory usage reached' chunk cache log message doesn't specify which cache is exhausted
[
https://issues.apache.org/jira/browse/CASSANDRA-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18184:
-
Status: Needs Committer (was: Review In Progress)
> 'Maximum memory usage reached' chunk cache log message doesn't specify which
> cache is exhausted
> ---
>
> Key: CASSANDRA-18184
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18184
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Yong Jiang
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
> Attachments: 18184-trunk.txt
>
>
> With Cassandra 4.0.x, we are seeing this cassandra.log message very
> frequently on the majority of our Cassandra 4.0 clusters:
> _[INFO ] [epollEventLoopGroup-5-3] cluster_id=99 ip_address=127.0.0.50
> NoSpamLogger.java:92 - Maximum memory usage reached (128.000MiB), cannot
> allocate chunk of 8.000MiB_
> It took me several weeks to track down what it means, until I saw this
> start-up message
> _BufferPools.java:49 - Global buffer pool limit is 2.000GiB for
> chunk-cache and 128.000MiB for networking_
> This maximum memory usage warning would benefit from clarifying that its the
> {*}network cache{*}, not the *off-heap chunk* *cache* which is exhausted.
> With 'chunk cache' in both warning messages, they're too easily confused.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18184) 'Maximum memory usage reached' chunk cache log message doesn't specify which cache is exhausted
[
https://issues.apache.org/jira/browse/CASSANDRA-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683498#comment-17683498
]
Brandon Williams commented on CASSANDRA-18184:
--
Everything looks good to me, +1.
> 'Maximum memory usage reached' chunk cache log message doesn't specify which
> cache is exhausted
> ---
>
> Key: CASSANDRA-18184
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18184
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Yong Jiang
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
> Attachments: 18184-trunk.txt
>
>
> With Cassandra 4.0.x, we are seeing this cassandra.log message very
> frequently on the majority of our Cassandra 4.0 clusters:
> _[INFO ] [epollEventLoopGroup-5-3] cluster_id=99 ip_address=127.0.0.50
> NoSpamLogger.java:92 - Maximum memory usage reached (128.000MiB), cannot
> allocate chunk of 8.000MiB_
> It took me several weeks to track down what it means, until I saw this
> start-up message
> _BufferPools.java:49 - Global buffer pool limit is 2.000GiB for
> chunk-cache and 128.000MiB for networking_
> This maximum memory usage warning would benefit from clarifying that its the
> {*}network cache{*}, not the *off-heap chunk* *cache* which is exhausted.
> With 'chunk cache' in both warning messages, they're too easily confused.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18218) OOM while running ShortAccordSimulationTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-18218:
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Normal
Discovered By: Fuzz Test
Severity: Normal
Status: Open (was: Triage Needed)
> OOM while running ShortAccordSimulationTest
> ---
>
> Key: CASSANDRA-18218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18218
> Project: Cassandra
> Issue Type: Bug
> Components: Accord, Test/burn
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: simulator
> Fix For: NA
>
> Attachments: java_pid73497.hprof.zip, java_pid75762.hprof.zip
>
>
> {{ShortAccordSimulationTest}} seems to consistently run out of heap when run
> locally. This is true even when we scale the number of threads and duration
> down...
> {noformat}
> AccordSimulationRunner.main(new String[] { "run", "-n", "3", "-t", "10",
> "--cluster-action-limit", "-1", "-c", "2", "-s", "10"});
> {noformat}
> {noformat}
> ERROR [CommandStore[0]:1] node1 CS:[0] OP:0x8fb4f8f1 2023-01-31 17:16:27,143
> Operation AsyncOperation{RUNNING}-0x8fb4f8f1 failed
> java.lang.OutOfMemoryError: Java heap space
>at java.util.Arrays.copyOf(Arrays.java:3181)
>at accord.local.Command$NotifyWaitingOn.push(Command.java:794)
>at accord.local.Command$NotifyWaitingOn.accept(Command.java:757)
>at accord.local.Command.maybeExecute(Command.java:600)
>at accord.local.Command.onChange(Command.java:546)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy.lambda$onChange$0(ListenerProxy.java:148)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy$$Lambda$4606/1457728285.accept(Unknown
> Source)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:261)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:248)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
>at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>at
> org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus$$Lambda$768/827906088.run(Unknown
> Source)
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>at java.lang.Thread.run(Thread.java:748)
> {noformat}
> JVM args make it seem like we’re passing both a 1 GiB and 8 GiB heap size,
> although that doesn’t seem to have any bearing on the result. Setting only 8
> GiB just takes longer to hit the same problem.
> {noformat}
> INFO [isolatedExecutor:2] node1 2023-01-31 17:15:56,940 JVM Arguments:
> [-Dstorage-config=/Users/maedhroz/Forks/cassandra/test/conf,
> -Djava.awt.headless=true,
> -javaagent:/Users/maedhroz/Forks/cassandra/lib/jamm-0.3.2.jar, -ea,
> -Djava.io.tmpdir=/var/folders/4d/zfjs7m7s6x5_l93k33r5k668gn/T/,
> -Dcassandra.debugrefcount=true, -Xss384k, -XX:SoftRefLRUPolicyMSPerMB=0,
> -XX:HeapDumpPath=build/test,
> -Dcassandra.test.driver.connection_timeout_ms=1,
> -Dcassandra.test.driver.read_timeout_ms=24000,
> -Dcassandra.memtable_row_overhead_computation_step=100,
> -Dcassandra.test.use_prepared=true,
> -Dcassandra.test.sstableformatdevelopment=true,
> -Djava.security.egd=file:/dev/urandom, -Dcassandra.testtag=,
> -Dcassandra.keepBriefBrief=${cassandra.keepBriefBrief},
> -Dcassandra.strict.runtime.checks=true,
> -Dcassandra.reads.thresholds.coordinator.defensive_checks_enabled=true,
> -DQT_SHRINKS=0, -Dlogback.configurationFile=test/conf/logback-simulator.xml,
> -Dcassandra.ring_delay_ms=1, -Dcassandra.tolerate_sstable_size=true,
> -Dcassandra.skip_sync=true, -Dcassandra.debugrefcount=false,
> -Dcassandra.test.simulator.determinismcheck=strict,
> -Dcassandra.test.simulator.print_asm=none,
> -javaagent:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-asm.jar,
>
> -Xbootclasspath/a:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-bootstrap.jar,
> -XX:ActiveProcessorCount=4, -XX:-TieredCompilation,
> -XX:-BackgroundCompilation, -XX:CICompilerCount=1,
> -XX:Tier4CompileThreshold=1000, -XX:ReservedCodeCacheSize=256M, -Xmx8G,
> -Xmx1024m]
> {noformat}
--
This message was sent by
[jira] [Assigned] (CASSANDRA-18218) OOM while running ShortAccordSimulationTest
[
https://issues.apache.org/jira/browse/CASSANDRA-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe reassigned CASSANDRA-18218:
---
Assignee: Caleb Rackliffe (was: Benedict Elliott Smith)
> OOM while running ShortAccordSimulationTest
> ---
>
> Key: CASSANDRA-18218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18218
> Project: Cassandra
> Issue Type: Bug
> Components: Accord, Test/burn
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: simulator
> Fix For: NA
>
> Attachments: java_pid73497.hprof.zip, java_pid75762.hprof.zip
>
>
> {{ShortAccordSimulationTest}} seems to consistently run out of heap when run
> locally. This is true even when we scale the number of threads and duration
> down...
> {noformat}
> AccordSimulationRunner.main(new String[] { "run", "-n", "3", "-t", "10",
> "--cluster-action-limit", "-1", "-c", "2", "-s", "10"});
> {noformat}
> {noformat}
> ERROR [CommandStore[0]:1] node1 CS:[0] OP:0x8fb4f8f1 2023-01-31 17:16:27,143
> Operation AsyncOperation{RUNNING}-0x8fb4f8f1 failed
> java.lang.OutOfMemoryError: Java heap space
>at java.util.Arrays.copyOf(Arrays.java:3181)
>at accord.local.Command$NotifyWaitingOn.push(Command.java:794)
>at accord.local.Command$NotifyWaitingOn.accept(Command.java:757)
>at accord.local.Command.maybeExecute(Command.java:600)
>at accord.local.Command.onChange(Command.java:546)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy.lambda$onChange$0(ListenerProxy.java:148)
>at
> org.apache.cassandra.service.accord.ListenerProxy$CommandListenerProxy$$Lambda$4606/1457728285.accept(Unknown
> Source)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:261)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation$ForConsumer.apply(AsyncOperation.java:248)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
>at
> org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
>at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
>at
> org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
>at
> org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus$$Lambda$768/827906088.run(Unknown
> Source)
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>at java.lang.Thread.run(Thread.java:748)
> {noformat}
> JVM args make it seem like we’re passing both a 1 GiB and 8 GiB heap size,
> although that doesn’t seem to have any bearing on the result. Setting only 8
> GiB just takes longer to hit the same problem.
> {noformat}
> INFO [isolatedExecutor:2] node1 2023-01-31 17:15:56,940 JVM Arguments:
> [-Dstorage-config=/Users/maedhroz/Forks/cassandra/test/conf,
> -Djava.awt.headless=true,
> -javaagent:/Users/maedhroz/Forks/cassandra/lib/jamm-0.3.2.jar, -ea,
> -Djava.io.tmpdir=/var/folders/4d/zfjs7m7s6x5_l93k33r5k668gn/T/,
> -Dcassandra.debugrefcount=true, -Xss384k, -XX:SoftRefLRUPolicyMSPerMB=0,
> -XX:HeapDumpPath=build/test,
> -Dcassandra.test.driver.connection_timeout_ms=1,
> -Dcassandra.test.driver.read_timeout_ms=24000,
> -Dcassandra.memtable_row_overhead_computation_step=100,
> -Dcassandra.test.use_prepared=true,
> -Dcassandra.test.sstableformatdevelopment=true,
> -Djava.security.egd=file:/dev/urandom, -Dcassandra.testtag=,
> -Dcassandra.keepBriefBrief=${cassandra.keepBriefBrief},
> -Dcassandra.strict.runtime.checks=true,
> -Dcassandra.reads.thresholds.coordinator.defensive_checks_enabled=true,
> -DQT_SHRINKS=0, -Dlogback.configurationFile=test/conf/logback-simulator.xml,
> -Dcassandra.ring_delay_ms=1, -Dcassandra.tolerate_sstable_size=true,
> -Dcassandra.skip_sync=true, -Dcassandra.debugrefcount=false,
> -Dcassandra.test.simulator.determinismcheck=strict,
> -Dcassandra.test.simulator.print_asm=none,
> -javaagent:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-asm.jar,
>
> -Xbootclasspath/a:/Users/maedhroz/Forks/cassandra/build/test/lib/jars/simulator-bootstrap.jar,
> -XX:ActiveProcessorCount=4, -XX:-TieredCompilation,
> -XX:-BackgroundCompilation, -XX:CICompilerCount=1,
> -XX:Tier4CompileThreshold=1000, -XX:ReservedCodeCacheSize=256M, -Xmx8G,
> -Xmx1024m]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail:
[jira] [Updated] (CASSANDRA-18222) Fix flaky test org.apache.cassandra.distributed.test.accord.AccordIntegrationTest
[ https://issues.apache.org/jira/browse/CASSANDRA-18222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-18222: -- Description: The accord integration tests are currently flaky due to Preempt happening (we convert that to a timeout). We need to address this to make sure the tests are stable. One solution may be make SimpleProgressLog’s frequency run different for jvm-dtest (likely to depend on CASSANDRA-18221 ), this would be similar to what we already do as timeouts maybe larger in jvm-dtest due to the extra complexity running multiple processes in the same JVM. Another solution may be to have the client coordinator “attach” to the new coordinator, this would avoid the user being impacted by this preempt when the coordinator is healthy. Example: https://app.circleci.com/pipelines/github/dcapwell/cassandra/1822/workflows/0d92898f-ad7d-4a7d-9198-3a7ce844ee93/jobs/16464 was: The accord integration tests are currently flaky due to Preempt happening (we convert that to a timeout). We need to address this to make sure the tests are stable. One solution may be make SimpleProgressLog’s frequency run different for jvm-dtest (likely to depend on CASSANDRA-18221 ), this would be similar to what we already do as timeouts maybe larger in jvm-dtest due to the extra complexity running multiple processes in the same JVM. Another solution may be to have the client coordinator “attach” to the new coordinator, this would avoid the user being impacted by this preempt when the coordinator is healthy. > Fix flaky test > org.apache.cassandra.distributed.test.accord.AccordIntegrationTest > - > > Key: CASSANDRA-18222 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18222 > Project: Cassandra > Issue Type: Bug > Components: Accord, CI >Reporter: David Capwell >Priority: Normal > Fix For: 4.x > > > The accord integration tests are currently flaky due to Preempt happening (we > convert that to a timeout). We need to address this to make sure the tests > are stable. > One solution may be make SimpleProgressLog’s frequency run different for > jvm-dtest (likely to depend on CASSANDRA-18221 ), this would be similar to > what we already do as timeouts maybe larger in jvm-dtest due to the extra > complexity running multiple processes in the same JVM. Another solution may > be to have the client coordinator “attach” to the new coordinator, this would > avoid the user being impacted by this preempt when the coordinator is > healthy. > Example: > https://app.circleci.com/pipelines/github/dcapwell/cassandra/1822/workflows/0d92898f-ad7d-4a7d-9198-3a7ce844ee93/jobs/16464 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18223) Byteman rule in stop_data_reads.btm cannot compile against accord.messages.ReplyContext
Caleb Rackliffe created CASSANDRA-18223:
---
Summary: Byteman rule in stop_data_reads.btm cannot compile
against accord.messages.ReplyContext
Key: CASSANDRA-18223
URL: https://issues.apache.org/jira/browse/CASSANDRA-18223
Project: Cassandra
Issue Type: Bug
Components: Accord
Reporter: Caleb Rackliffe
The Python {{read_repair_test}} relies on a Byteman rule on the {{doVerb()}}
method in {{ReadCommandVerbHandler}}, but {{accord.messages.ReplyContext}}
isn’t on the classpath. This is probably because we don't include it in the
list of jars created in {{byteman_validate}}.
{noformat}
AssertionError: byteman script didn't compile
Checking rule disable data reads against class
org.apache.cassandra.db.ReadCommandVerbHandler
Parsed rule "disable data reads" for class
org.apache.cassandra.db.ReadCommandVerbHandler
ERROR : Failed to check rule "disable data reads" loaded from
/home/cassandra/cassandra-dtest/byteman/read_repair/stop_data_reads.btm line 8
against method doVerb(org.apache.cassandra.net.Message) void
java.lang.NoClassDefFoundError: accord/messages/ReplyContext
{noformat}
ex.
https://app.circleci.com/pipelines/github/maedhroz/cassandra/686/workflows/ffd1e528-b8ec-4534-a333-ab450e110e89/jobs/6481/tests#failed-test-0
It might make sense to fix this after CASSANDRA-18204 wraps up, so we know
exactly how the Accord library is pulled into C*. Then, once we do fix it, we
should fix in a way that still works w/ 4.0 and 4.1, etc. (i.e. Don't assume
the Accord library must be present.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18223) Byteman rule in stop_data_reads.btm cannot compile against accord.messages.ReplyContext
[
https://issues.apache.org/jira/browse/CASSANDRA-18223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-18223:
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Normal
Discovered By: DTest
Fix Version/s: NA
Severity: Normal
Status: Open (was: Triage Needed)
> Byteman rule in stop_data_reads.btm cannot compile against
> accord.messages.ReplyContext
> ---
>
> Key: CASSANDRA-18223
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18223
> Project: Cassandra
> Issue Type: Bug
> Components: Accord
>Reporter: Caleb Rackliffe
>Priority: Normal
> Fix For: NA
>
>
> The Python {{read_repair_test}} relies on a Byteman rule on the {{doVerb()}}
> method in {{ReadCommandVerbHandler}}, but {{accord.messages.ReplyContext}}
> isn’t on the classpath. This is probably because we don't include it in the
> list of jars created in {{byteman_validate}}.
> {noformat}
> AssertionError: byteman script didn't compile
> Checking rule disable data reads against class
> org.apache.cassandra.db.ReadCommandVerbHandler
> Parsed rule "disable data reads" for class
> org.apache.cassandra.db.ReadCommandVerbHandler
> ERROR : Failed to check rule "disable data reads" loaded from
> /home/cassandra/cassandra-dtest/byteman/read_repair/stop_data_reads.btm line
> 8 against method doVerb(org.apache.cassandra.net.Message) void
> java.lang.NoClassDefFoundError: accord/messages/ReplyContext
> {noformat}
> ex.
> https://app.circleci.com/pipelines/github/maedhroz/cassandra/686/workflows/ffd1e528-b8ec-4534-a333-ab450e110e89/jobs/6481/tests#failed-test-0
> It might make sense to fix this after CASSANDRA-18204 wraps up, so we know
> exactly how the Accord library is pulled into C*. Then, once we do fix it, we
> should fix in a way that still works w/ 4.0 and 4.1, etc. (i.e. Don't assume
> the Accord library must be present.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18223) Byteman rule in stop_data_reads.btm cannot compile against accord.messages.ReplyContext
[
https://issues.apache.org/jira/browse/CASSANDRA-18223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-18223:
Labels: byteman dtest python (was: )
> Byteman rule in stop_data_reads.btm cannot compile against
> accord.messages.ReplyContext
> ---
>
> Key: CASSANDRA-18223
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18223
> Project: Cassandra
> Issue Type: Bug
> Components: Accord
>Reporter: Caleb Rackliffe
>Priority: Normal
> Labels: byteman, dtest, python
> Fix For: NA
>
>
> The Python {{read_repair_test}} relies on a Byteman rule on the {{doVerb()}}
> method in {{ReadCommandVerbHandler}}, but {{accord.messages.ReplyContext}}
> isn’t on the classpath. This is probably because we don't include it in the
> list of jars created in {{byteman_validate}}.
> {noformat}
> AssertionError: byteman script didn't compile
> Checking rule disable data reads against class
> org.apache.cassandra.db.ReadCommandVerbHandler
> Parsed rule "disable data reads" for class
> org.apache.cassandra.db.ReadCommandVerbHandler
> ERROR : Failed to check rule "disable data reads" loaded from
> /home/cassandra/cassandra-dtest/byteman/read_repair/stop_data_reads.btm line
> 8 against method doVerb(org.apache.cassandra.net.Message) void
> java.lang.NoClassDefFoundError: accord/messages/ReplyContext
> {noformat}
> ex.
> https://app.circleci.com/pipelines/github/maedhroz/cassandra/686/workflows/ffd1e528-b8ec-4534-a333-ab450e110e89/jobs/6481/tests#failed-test-0
> It might make sense to fix this after CASSANDRA-18204 wraps up, so we know
> exactly how the Accord library is pulled into C*. Then, once we do fix it, we
> should fix in a way that still works w/ 4.0 and 4.1, etc. (i.e. Don't assume
> the Accord library must be present.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18222) Fix flaky test org.apache.cassandra.distributed.test.accord.AccordIntegrationTest
[ https://issues.apache.org/jira/browse/CASSANDRA-18222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-18222: -- Bug Category: Parent values: Correctness(12982)Level 1 values: Test Failure(12990) Complexity: Normal Discovered By: Unit Test Fix Version/s: 4.x Severity: Normal Status: Open (was: Triage Needed) > Fix flaky test > org.apache.cassandra.distributed.test.accord.AccordIntegrationTest > - > > Key: CASSANDRA-18222 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18222 > Project: Cassandra > Issue Type: Bug > Components: Accord, CI >Reporter: David Capwell >Priority: Normal > Fix For: 4.x > > > The accord integration tests are currently flaky due to Preempt happening (we > convert that to a timeout). We need to address this to make sure the tests > are stable. > One solution may be make SimpleProgressLog’s frequency run different for > jvm-dtest (likely to depend on CASSANDRA-18221 ), this would be similar to > what we already do as timeouts maybe larger in jvm-dtest due to the extra > complexity running multiple processes in the same JVM. Another solution may > be to have the client coordinator “attach” to the new coordinator, this would > avoid the user being impacted by this preempt when the coordinator is > healthy. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18222) Fix flaky test org.apache.cassandra.distributed.test.accord.AccordIntegrationTest
David Capwell created CASSANDRA-18222: - Summary: Fix flaky test org.apache.cassandra.distributed.test.accord.AccordIntegrationTest Key: CASSANDRA-18222 URL: https://issues.apache.org/jira/browse/CASSANDRA-18222 Project: Cassandra Issue Type: Bug Components: Accord, CI Reporter: David Capwell The accord integration tests are currently flaky due to Preempt happening (we convert that to a timeout). We need to address this to make sure the tests are stable. One solution may be make SimpleProgressLog’s frequency run different for jvm-dtest (likely to depend on CASSANDRA-18221 ), this would be similar to what we already do as timeouts maybe larger in jvm-dtest due to the extra complexity running multiple processes in the same JVM. Another solution may be to have the client coordinator “attach” to the new coordinator, this would avoid the user being impacted by this preempt when the coordinator is healthy. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cep-15-accord updated: Ninja: CASSANDRA-18214 added rat checks to Accord, but our metadata files .BRANCH and .REPO do not have a license, causing the build to fail; exclude rat when
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch cep-15-accord
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cep-15-accord by this push:
new 1600f5fd3f Ninja: CASSANDRA-18214 added rat checks to Accord, but our
metadata files .BRANCH and .REPO do not have a license, causing the build to
fail; exclude rat when building accord
1600f5fd3f is described below
commit 1600f5fd3f036dc5624f659850c562fa241b43ce
Author: David Capwell
AuthorDate: Thu Feb 2 08:46:32 2023 -0800
Ninja: CASSANDRA-18214 added rat checks to Accord, but our metadata files
.BRANCH and .REPO do not have a license, causing the build to fail; exclude rat
when building accord
patch by David Capwell; reviewed by Ariel Weisberg for CASSANDRA-18214
---
.build/include-accord.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.build/include-accord.sh b/.build/include-accord.sh
index 2f55c1d18c..84bf993d32 100755
--- a/.build/include-accord.sh
+++ b/.build/include-accord.sh
@@ -50,7 +50,7 @@ _main() {
# are there changes?
git pull --rebase origin "$accord_branch"
if [[ $(git rev-parse HEAD) != $(cat .SHA || true) ]]; then
-./gradlew clean install -x test
+./gradlew clean install -x test -x rat
git rev-parse HEAD > .SHA
fi
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18204) CEP-15: (C*) Add git submodule for Accord
[ https://issues.apache.org/jira/browse/CASSANDRA-18204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18204: Reviewers: Caleb Rackliffe, Michael Semb Wever (was: Michael Semb Wever) Status: Review In Progress (was: Patch Available) > CEP-15: (C*) Add git submodule for Accord > - > > Key: CASSANDRA-18204 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18204 > Project: Cassandra > Issue Type: Task > Components: Accord >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.x > > Time Spent: 10m > Remaining Estimate: 0h > > As talked about in dev@ thread "Intra-project dependencies”, we talked about > adding git submodules but before doing this had to work out a few issues > first; this ticket is to track this work. > Goals > * when checking out an older commit, or pulling in newer commits, the > submodule should also be updated automatically > * release artifact must include the submodule and must be able to build > without issue > * build.xml must be updated to build the submodule > * build.xml must be updated to release the submodule jar -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18204) CEP-15: (C*) Add git submodule for Accord
[ https://issues.apache.org/jira/browse/CASSANDRA-18204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18204: Epic Link: CASSANDRA-17092 > CEP-15: (C*) Add git submodule for Accord > - > > Key: CASSANDRA-18204 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18204 > Project: Cassandra > Issue Type: Task > Components: Accord >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.x > > Time Spent: 10m > Remaining Estimate: 0h > > As talked about in dev@ thread "Intra-project dependencies”, we talked about > adding git submodules but before doing this had to work out a few issues > first; this ticket is to track this work. > Goals > * when checking out an older commit, or pulling in newer commits, the > submodule should also be updated automatically > * release artifact must include the submodule and must be able to build > without issue > * build.xml must be updated to build the submodule > * build.xml must be updated to release the submodule jar -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18221) CEP-15: (Accord) Define a configuration system for Accord to allow overriding of internal configs and integrate with Cassandra yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683491#comment-17683491 ] Kamalesh Palanisamy commented on CASSANDRA-18221: - [~dcapwell] I can look into this task if there are no takers yet. Can you assign it to me? > CEP-15: (Accord) Define a configuration system for Accord to allow overriding > of internal configs and integrate with Cassandra yaml > --- > > Key: CASSANDRA-18221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18221 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: David Capwell >Priority: Normal > Fix For: 4.x > > > Users are used to modifying cassandra via yaml and JMX (for dynamic configs) > but accord does not integrate with this right now; we should enhance Accord > to expose configs that are defined in cassandra yaml. > As an extension on this, we should figure out which configs are “dynamic” and > allow overriding via JMX or system vtable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18130) Log hardware and container params during test runs to help troubleshoot intermittent failures
[
https://issues.apache.org/jira/browse/CASSANDRA-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie reassigned CASSANDRA-18130:
-
Reviewers: Josh McKenzie (was: Michael Semb Wever)
Assignee: Michael Semb Wever (was: Josh McKenzie)
> Log hardware and container params during test runs to help troubleshoot
> intermittent failures
> -
>
> Key: CASSANDRA-18130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18130
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/java, Test/dtest/python, Test/unit
>Reporter: Josh McKenzie
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
>
> {color:#00}We’ve long had flakiness in our containerized ASF CI
> environment that we don’t see in circleci. The environment itself is both
> containerized and heterogenous, so there are differences in both the hardware
> environment and the software environment in which it executes. For reference,
> see:
> [https://github.com/apache/cassandra-builds/blob/trunk/ASF-jenkins-agents.md#current-agents]{color}
> {color:#00} {color}
> {color:#00}We should log a variety of hardware, container, and software
> environment details to help get to the bottom of where some test failures may
> be occurring. As we don’t have shell access to the machines it’ll be easier
> to have this information logged / retrieved during test runs than to try and
> profile each host independently.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18130) Log hardware and container params during test runs to help troubleshoot intermittent failures
[
https://issues.apache.org/jira/browse/CASSANDRA-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-18130:
--
Authors: Josh McKenzie, Michael Semb Wever (was: Michael Semb Wever)
> Log hardware and container params during test runs to help troubleshoot
> intermittent failures
> -
>
> Key: CASSANDRA-18130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18130
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/java, Test/dtest/python, Test/unit
>Reporter: Josh McKenzie
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
>
> {color:#00}We’ve long had flakiness in our containerized ASF CI
> environment that we don’t see in circleci. The environment itself is both
> containerized and heterogenous, so there are differences in both the hardware
> environment and the software environment in which it executes. For reference,
> see:
> [https://github.com/apache/cassandra-builds/blob/trunk/ASF-jenkins-agents.md#current-agents]{color}
> {color:#00} {color}
> {color:#00}We should log a variety of hardware, container, and software
> environment details to help get to the bottom of where some test failures may
> be occurring. As we don’t have shell access to the machines it’ll be easier
> to have this information logged / retrieved during test runs than to try and
> profile each host independently.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18219) Warning message on aggregation queries doesn't specify table name or aggregate type
[
https://issues.apache.org/jira/browse/CASSANDRA-18219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683467#comment-17683467
]
Stefan Miklosovic commented on CASSANDRA-18219:
---
[~blerer] would you mind to review my patch? I think the guardrail is not
necessary here.
I will try to gather more info whether we can put this to 4.0.x too.
> Warning message on aggregation queries doesn't specify table name or
> aggregate type
> ---
>
> Key: CASSANDRA-18219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18219
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The existing aggregation query warning messages in cassandra.log from
> SelectStatement.java:
> _'[WARN] Aggregation query used without partition key'_
> {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys
> (IN restriction)'_
> are missing the helpful details which would assist users in quickly
> identifying the offending query. The table name and type of aggregation would
> be useful additions in the WARN message.
> In addition, following the example of the slow query logger and printing out
> the query string at DEBUG level would be especially helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17992) Upgrade Netty on 4.x(current trunk)
[ https://issues.apache.org/jira/browse/CASSANDRA-17992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17992: Description: I haven't been able to identify from the Netty docs which was the lowest version where JDK17 was added but we are about 40 versions behind in netty 4 so I suspect we better update. -We need to consider there was an issue with class cast exceptions when building with JDK17 with newer versions of netty (the newest available in March 2022). For the record, we didn't see those when running CI on JDK8 and JDK11. We also need to carefully revise the changes between the netty versions. -->- CASSANDRA-18180 Upgrading will cover also a fix in netty that was discussed in [this|https://the-asf.slack.com/archives/CK23JSY2K/p1665567660202989] ASF Slack thread. CC [~benedict] , [~aleksey] was: I haven't been able to identify from the Netty docs which was the lowest version where JDK17 was added but we are about 40 versions behind in netty 4 so I suspect we better update. We need to consider there was an issue with class cast exceptions when building with JDK17 with newer versions of netty (the newest available in March 2022). For the record, we didn't see those when running CI on JDK8 and JDK11. We also need to carefully revise the changes between the netty versions. Upgrading will cover also a fix in netty that was discussed in [this|https://the-asf.slack.com/archives/CK23JSY2K/p1665567660202989] ASF Slack thread. CC [~benedict] , [~aleksey] > Upgrade Netty on 4.x(current trunk) > --- > > Key: CASSANDRA-17992 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17992 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 4.x > > > I haven't been able to identify from the Netty docs which was the lowest > version where JDK17 was added but we are about 40 versions behind in netty 4 > so I suspect we better update. > -We need to consider there was an issue with class cast exceptions when > building with JDK17 with newer versions of netty (the newest available in > March 2022). For the record, we didn't see those when running CI on JDK8 and > JDK11. We also need to carefully revise the changes between the netty > versions. -->- CASSANDRA-18180 > Upgrading will cover also a fix in netty that was discussed in > [this|https://the-asf.slack.com/archives/CK23JSY2K/p1665567660202989] ASF > Slack thread. > CC [~benedict] , [~aleksey] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18184) 'Maximum memory usage reached' chunk cache log message doesn't specify which cache is exhausted
[
https://issues.apache.org/jira/browse/CASSANDRA-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18184:
-
Status: Patch Available (was: In Progress)
> 'Maximum memory usage reached' chunk cache log message doesn't specify which
> cache is exhausted
> ---
>
> Key: CASSANDRA-18184
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18184
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Yong Jiang
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
> Attachments: 18184-trunk.txt
>
>
> With Cassandra 4.0.x, we are seeing this cassandra.log message very
> frequently on the majority of our Cassandra 4.0 clusters:
> _[INFO ] [epollEventLoopGroup-5-3] cluster_id=99 ip_address=127.0.0.50
> NoSpamLogger.java:92 - Maximum memory usage reached (128.000MiB), cannot
> allocate chunk of 8.000MiB_
> It took me several weeks to track down what it means, until I saw this
> start-up message
> _BufferPools.java:49 - Global buffer pool limit is 2.000GiB for
> chunk-cache and 128.000MiB for networking_
> This maximum memory usage warning would benefit from clarifying that its the
> {*}network cache{*}, not the *off-heap chunk* *cache* which is exhausted.
> With 'chunk cache' in both warning messages, they're too easily confused.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18184) 'Maximum memory usage reached' chunk cache log message doesn't specify which cache is exhausted
[
https://issues.apache.org/jira/browse/CASSANDRA-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18184:
-
Reviewers: Brandon Williams, Brandon Williams (was: Brandon Williams)
Brandon Williams, Brandon Williams (was: Brandon Williams)
Status: Review In Progress (was: Patch Available)
> 'Maximum memory usage reached' chunk cache log message doesn't specify which
> cache is exhausted
> ---
>
> Key: CASSANDRA-18184
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18184
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Yong Jiang
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
> Attachments: 18184-trunk.txt
>
>
> With Cassandra 4.0.x, we are seeing this cassandra.log message very
> frequently on the majority of our Cassandra 4.0 clusters:
> _[INFO ] [epollEventLoopGroup-5-3] cluster_id=99 ip_address=127.0.0.50
> NoSpamLogger.java:92 - Maximum memory usage reached (128.000MiB), cannot
> allocate chunk of 8.000MiB_
> It took me several weeks to track down what it means, until I saw this
> start-up message
> _BufferPools.java:49 - Global buffer pool limit is 2.000GiB for
> chunk-cache and 128.000MiB for networking_
> This maximum memory usage warning would benefit from clarifying that its the
> {*}network cache{*}, not the *off-heap chunk* *cache* which is exhausted.
> With 'chunk cache' in both warning messages, they're too easily confused.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18184) 'Maximum memory usage reached' chunk cache log message doesn't specify which cache is exhausted
[
https://issues.apache.org/jira/browse/CASSANDRA-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683464#comment-17683464
]
Brandon Williams commented on CASSANDRA-18184:
--
bq. I also ran CI tasks for my own education and practice.
You did a nice job! Unfortunately though we need to use higher (paid)
resources to cover the dtests that otherwise timeout. I've started that here:
||Branch||CI||
|[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18184-4.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/850/workflows/3a3799b4-0028-4841-9498-87f528df081a],
[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/850/workflows/0a6e87c6-a038-406a-add3-ddf817f39e0f]|
|[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-18184-4.1]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/852/workflows/bc232718-f790-4b1d-b153-580b7da134f9],
[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/852/workflows/d9595c72-0649-4d43-b8ee-f456e28d0caa]|
|[trunk|https://github.com/driftx/cassandra/tree/CASSANDRA-18184-trunk]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/851/workflows/ca6e2994-9ebe-40ca-b9f7-7d58da71c9cb],
[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/851/workflows/7a912e3f-e175-4fb4-98bb-d821b19b3efa]|
and will check back when that has completed.
> 'Maximum memory usage reached' chunk cache log message doesn't specify which
> cache is exhausted
> ---
>
> Key: CASSANDRA-18184
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18184
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Yong Jiang
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
> Attachments: 18184-trunk.txt
>
>
> With Cassandra 4.0.x, we are seeing this cassandra.log message very
> frequently on the majority of our Cassandra 4.0 clusters:
> _[INFO ] [epollEventLoopGroup-5-3] cluster_id=99 ip_address=127.0.0.50
> NoSpamLogger.java:92 - Maximum memory usage reached (128.000MiB), cannot
> allocate chunk of 8.000MiB_
> It took me several weeks to track down what it means, until I saw this
> start-up message
> _BufferPools.java:49 - Global buffer pool limit is 2.000GiB for
> chunk-cache and 128.000MiB for networking_
> This maximum memory usage warning would benefit from clarifying that its the
> {*}network cache{*}, not the *off-heap chunk* *cache* which is exhausted.
> With 'chunk cache' in both warning messages, they're too easily confused.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18219) Warning message on aggregation queries doesn't specify table name or aggregate type
[
https://issues.apache.org/jira/browse/CASSANDRA-18219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683460#comment-17683460
]
Brad Schoening commented on CASSANDRA-18219:
+1 on simple logging
Would be nice to have the keyspace+table+type in 4.0.x/4.1. Is it worth
considering a separate bug for just that simple two line fix (excluding the
debug logging)?
> Warning message on aggregation queries doesn't specify table name or
> aggregate type
> ---
>
> Key: CASSANDRA-18219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18219
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The existing aggregation query warning messages in cassandra.log from
> SelectStatement.java:
> _'[WARN] Aggregation query used without partition key'_
> {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys
> (IN restriction)'_
> are missing the helpful details which would assist users in quickly
> identifying the offending query. The table name and type of aggregation would
> be useful additions in the WARN message.
> In addition, following the example of the slow query logger and printing out
> the query string at DEBUG level would be especially helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18187) Add unit tests for per-row TTL and Timestamp usage in CQLSSTableWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18187: - Status: Ready to Commit (was: Review In Progress) > Add unit tests for per-row TTL and Timestamp usage in CQLSSTableWriter > -- > > Key: CASSANDRA-18187 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18187 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Doug Rohrer >Assignee: Doug Rohrer >Priority: Low > Fix For: 4.x > > Time Spent: 40m > Remaining Estimate: 0h > > CQLSSTableWriter supports per-row setting of both timestamp and TTL values, > but it’s not tested or documented today. Add tests to cover setting both TTL > and Timestamp values for rows using the CQLSSTableWriter. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18187) Add unit tests for per-row TTL and Timestamp usage in CQLSSTableWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18187: - Fix Version/s: 4.2 (was: 4.x) Source Control Link: https://github.com/apache/cassandra/commit/1e685219da4177fc5b5d6025618398532d2a0124 Resolution: Fixed Status: Resolved (was: Ready to Commit) Thanks all, committed. > Add unit tests for per-row TTL and Timestamp usage in CQLSSTableWriter > -- > > Key: CASSANDRA-18187 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18187 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Doug Rohrer >Assignee: Doug Rohrer >Priority: Low > Fix For: 4.2 > > Time Spent: 40m > Remaining Estimate: 0h > > CQLSSTableWriter supports per-row setting of both timestamp and TTL values, > but it’s not tested or documented today. Add tests to cover setting both TTL > and Timestamp values for rows using the CQLSSTableWriter. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Add unit tests for per-row TTL and Timestamp usage in CQLSSTableWriter
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 1e685219da Add unit tests for per-row TTL and Timestamp usage in
CQLSSTableWriter
1e685219da is described below
commit 1e685219da4177fc5b5d6025618398532d2a0124
Author: Doug Rohrer
AuthorDate: Mon Jan 23 13:53:09 2023 -0500
Add unit tests for per-row TTL and Timestamp usage in CQLSSTableWriter
Patch by Doug Rohrer; reviewed by brandonwilliams and dcapwell for
CASSANDRA-18187
CQLSSTableWriter supports per-row setting of both timestamp and TTL values,
but it’s not tested or documented today.
Add tests to cover setting both TTL and Timestamp values for rows using the
CQLSSTableWriter.
---
.../cassandra/io/sstable/CQLSSTableWriterTest.java | 118 +
1 file changed, 118 insertions(+)
diff --git
a/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java
b/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java
index 1851314b5d..19e578bd7e 100644
--- a/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java
@@ -22,6 +22,7 @@ import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.*;
import java.util.concurrent.ExecutionException;
+import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.BiPredicate;
import java.util.stream.Collectors;
@@ -55,6 +56,7 @@ import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.transport.ProtocolVersion;
import org.apache.cassandra.utils.*;
+import static org.apache.cassandra.utils.Clock.Global.currentTimeMillis;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
@@ -1116,6 +1118,122 @@ public class CQLSSTableWriterTest
assertEquals(0, filtered.size());
}
+@Test
+public void testWriteWithTimestamps() throws Exception
+{
+long now = currentTimeMillis();
+long then = now - 1000;
+final String schema = "CREATE TABLE " + qualifiedTable + " ("
+ + " k int,"
+ + " v1 int,"
+ + " v2 int,"
+ + " v3 text,"
+ + " PRIMARY KEY (k)"
+ + ")";
+
+CQLSSTableWriter writer = CQLSSTableWriter.builder()
+ .inDirectory(dataDir)
+ .forTable(schema)
+ .using("INSERT INTO " +
qualifiedTable +
+ " (k, v1, v2, v3)
VALUES (?,?,?,?) using timestamp ?" )
+ .build();
+
+// Note that, all other things being equal, Cassandra will sort these
rows lexicographically, so we use "higher" values in the
+// row we expect to "win" so that we're sure that it isn't just
accidentally picked due to the row sorting.
+writer.addRow( 1, 4, 5, "b", now); // This write should be the one
found at the end because it has a higher timestamp
+writer.addRow( 1, 2, 3, "a", then);
+writer.close();
+loadSSTables(dataDir, keyspace);
+
+UntypedResultSet resultSet = QueryProcessor.executeInternal("SELECT *
FROM " + qualifiedTable);
+assertEquals(1, resultSet.size());
+
+Iterator iter = resultSet.iterator();
+UntypedResultSet.Row r1 = iter.next();
+assertEquals(1, r1.getInt("k"));
+assertEquals(4, r1.getInt("v1"));
+assertEquals(5, r1.getInt("v2"));
+assertEquals("b", r1.getString("v3"));
+assertFalse(iter.hasNext());
+}
+@Test
+public void testWriteWithTtl() throws Exception
+{
+final String schema = "CREATE TABLE " + qualifiedTable + " ("
+ + " k int,"
+ + " v1 int,"
+ + " v2 int,"
+ + " v3 text,"
+ + " PRIMARY KEY (k)"
+ + ")";
+
+CQLSSTableWriter.Builder builder = CQLSSTableWriter.builder()
+ .inDirectory(dataDir)
+ .forTable(schema)
+ .using("INSERT INTO "
+ qualifiedTable +
+" (k, v1, v2,
v3) VALUES (?,?,?,?) using TTL ?");
+CQLSSTableWriter writer = builder.build();
+//
[jira] [Comment Edited] (CASSANDRA-18219) Warning message on aggregation queries doesn't specify table name or aggregate type
[
https://issues.apache.org/jira/browse/CASSANDRA-18219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683440#comment-17683440
]
Stefan Miklosovic edited comment on CASSANDRA-18219 at 2/2/23 2:49 PM:
---
Given the fact that it is quite difficult to differentiate between "select
count " and "select max(a), max(b) from ks.tb", or, in other words, between
aggregation queries which people do not want to forbid in general and those
which are causing problems on the application level, I would go with simple
logging only as in the patch instead of guardrail.
We can also print user as well, like this:
{code:java}
Aggregation query used without partition key on table ks.tb, aggregation type:
AGGREGATE_EVERYTHING, query: , user: cassandra
{code}
When user is anonymous, it would be
{code:java}
Aggregation query used without partition key on table ks.tb, aggregation type:
AGGREGATE_EVERYTHING, query: , user: anonymous
{code}
We can also log remote address for that matter if it helps you.
However, one little problem I see is that if we start to print user as well,
why do we do it {_}only here{_}? The output would start to become inconsistent
with other logging output for which we do not print user yet.
For this reason, unless we start to put users everywhere, I would omit user
from the query and we would just log it without it.
btw I think this is a new feature, not a bug. So backporting this to 4.0.x will
likely not happen. Only to trunk. We are fixing bugs only for anything lower
than trunk so you would need to upgrade to 4.2 to see this in action, I guess.
was (Author: smiklosovic):
Given the fact that it is quite difficult to differentiate between "select
count " and "select max(a), max(b) from ks.tb", or, in other words, between
aggregation queries which people do not want to forbid in general and those
which are causing problems on the application level, I would go with simple
logging only as in the patch instead of guardrail.
We can also print user as well, like this:
{code}
Aggregation query used without partition key on table ks.tb, aggregation type:
AGGREGATE_EVERYTHING, query: , user: cassandra
{code}
When user is anonymous, it would be
{code}
Aggregation query used without partition key on table ks.tb, aggregation type:
AGGREGATE_EVERYTHING, query: , user: anonymous
{code}
We can also log remote address for that matter if it helps you.
However, one little problem I see is that if we start to print user as well,
why do we do it _only here_? The output would start to become inconsistent with
other logging output for which we do not print user yet.
For this reason, unless we start to put users everywhere, I would omit user
from the query and we would just log it without it.
> Warning message on aggregation queries doesn't specify table name or
> aggregate type
> ---
>
> Key: CASSANDRA-18219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18219
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The existing aggregation query warning messages in cassandra.log from
> SelectStatement.java:
> _'[WARN] Aggregation query used without partition key'_
> {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys
> (IN restriction)'_
> are missing the helpful details which would assist users in quickly
> identifying the offending query. The table name and type of aggregation would
> be useful additions in the WARN message.
> In addition, following the example of the slow query logger and printing out
> the query string at DEBUG level would be especially helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18187) Add unit tests for per-row TTL and Timestamp usage in CQLSSTableWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683452#comment-17683452 ] Doug Rohrer commented on CASSANDRA-18187: - Thanks for the second review - made the recommended changes, so it should be ready to go. > Add unit tests for per-row TTL and Timestamp usage in CQLSSTableWriter > -- > > Key: CASSANDRA-18187 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18187 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Doug Rohrer >Assignee: Doug Rohrer >Priority: Low > Fix For: 4.x > > Time Spent: 40m > Remaining Estimate: 0h > > CQLSSTableWriter supports per-row setting of both timestamp and TTL values, > but it’s not tested or documented today. Add tests to cover setting both TTL > and Timestamp values for rows using the CQLSSTableWriter. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18219) Warning message on aggregation queries doesn't specify table name or aggregate type
[
https://issues.apache.org/jira/browse/CASSANDRA-18219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683440#comment-17683440
]
Stefan Miklosovic commented on CASSANDRA-18219:
---
Given the fact that it is quite difficult to differentiate between "select
count " and "select max(a), max(b) from ks.tb", or, in other words, between
aggregation queries which people do not want to forbid in general and those
which are causing problems on the application level, I would go with simple
logging only as in the patch instead of guardrail.
We can also print user as well, like this:
{code}
Aggregation query used without partition key on table ks.tb, aggregation type:
AGGREGATE_EVERYTHING, query: , user: cassandra
{code}
When user is anonymous, it would be
{code}
Aggregation query used without partition key on table ks.tb, aggregation type:
AGGREGATE_EVERYTHING, query: , user: anonymous
{code}
We can also log remote address for that matter if it helps you.
However, one little problem I see is that if we start to print user as well,
why do we do it _only here_? The output would start to become inconsistent with
other logging output for which we do not print user yet.
For this reason, unless we start to put users everywhere, I would omit user
from the query and we would just log it without it.
> Warning message on aggregation queries doesn't specify table name or
> aggregate type
> ---
>
> Key: CASSANDRA-18219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18219
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The existing aggregation query warning messages in cassandra.log from
> SelectStatement.java:
> _'[WARN] Aggregation query used without partition key'_
> {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys
> (IN restriction)'_
> are missing the helpful details which would assist users in quickly
> identifying the offending query. The table name and type of aggregation would
> be useful additions in the WARN message.
> In addition, following the example of the slow query logger and printing out
> the query string at DEBUG level would be especially helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18051) Optimize test_network_topology_strategy and test_network_topology_strategy_each_quorum for large containers in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683431#comment-17683431
]
Andres de la Peña commented on CASSANDRA-18051:
---
Yep, fine with me. As for simplifying the two super-heavy dtests, we could
later ask on Slack or the mail list to see if anyone sees a reason for them
needing three DCs. I think that if no one does we shouldn't keep us tied to
them.
> Optimize test_network_topology_strategy and
> test_network_topology_strategy_each_quorum for large containers in CircleCI
> ---
>
> Key: CASSANDRA-18051
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18051
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>
> As pointed in CASSANDRA-18001, there are two Python Large DTests that fail
> because of not enough resources with large containers. [~adelapena] looked
> into them and suggested that the tests themselves can be optimized for
> running on Large containers instead of using XLarge containers for only two
> tests from the whole suite.
> {quote}As for the two tests starting nine nodes each,
> {{test_network_topology_strategy}} and
> {{{}test_network_topology_strategy_each_quorum{}}}, I think they were added
> by CASSANDRA-10584 to test multi-DC consistency levels. I wonder if they
> actually need to start 3 data centers with 3 nodes each. Would two data
> centers with 3 nodes be enough to test multi-DC consistency levels with
> multiple data centers? 6 nodes should be doable on Circle with the current
> resources.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683430#comment-17683430 ] Stefan Miklosovic commented on CASSANDRA-18143: --- Great patch. [~jlewandowski] are you on top of this? Would be a bummer to let this one slip through. > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18219) Warning message on aggregation queries doesn't specify table name or aggregate type
[
https://issues.apache.org/jira/browse/CASSANDRA-18219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683426#comment-17683426
]
Brad Schoening commented on CASSANDRA-18219:
[~blerer] [~smiklosovic] both application code and ad hoc queries can generate
these WARN messages. Ad hoc may be something like select count(*). We
wouldn't want to block those and, in any event, if they're too slow they'll
time out.
The situations where the existing warning is a problem is with application
code. There may be multiple application teams using the same database, and
when the aggregation warn message is seen in the log, it's difficult to tie it
back to the code or even know which application ran the query. Multiple,
unrelated applications may share the same database using different keyspaces,
and the existing WARN message doesn't mention the keyspace.
We usually have to turn on settraceprobability to capture the queries. Having
just the keyspace, table and aggregation type in the cassandra.log would solve
90% of the issues.
The main issue is {+}traceability{+}, we're not trying to outright prevent
these queries. I'm not sure if guardrails in 4.1+ would be a better option
here, it may be worth further discussion. But it would be nice to fix the WARN
message with Stefan's PR in 4.0.x.
> Warning message on aggregation queries doesn't specify table name or
> aggregate type
> ---
>
> Key: CASSANDRA-18219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18219
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The existing aggregation query warning messages in cassandra.log from
> SelectStatement.java:
> _'[WARN] Aggregation query used without partition key'_
> {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys
> (IN restriction)'_
> are missing the helpful details which would assist users in quickly
> identifying the offending query. The table name and type of aggregation would
> be useful additions in the WARN message.
> In addition, following the example of the slow query logger and printing out
> the query string at DEBUG level would be especially helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18219) Warning message on aggregation queries doesn't specify table name or aggregate type
[
https://issues.apache.org/jira/browse/CASSANDRA-18219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683426#comment-17683426
]
Brad Schoening edited comment on CASSANDRA-18219 at 2/2/23 1:47 PM:
[~blerer] [~smiklosovic] both application code and ad hoc queries can generate
these WARN messages. Ad hoc may be something like 'select count *'. We
wouldn't want to block those and, in any event, if they're too slow they'll
time out.
The situations where the existing warning is a problem is with application
code. There may be multiple application teams using the same database, and
when the aggregation warn message is seen in the log, it's difficult to tie it
back to the code or even know which application ran the query. Multiple,
unrelated applications may share the same database using different keyspaces,
and the existing WARN message doesn't mention the keyspace.
We usually have to turn on settraceprobability to capture the queries. Having
just the keyspace, table and aggregation type in the cassandra.log would solve
90% of the issues.
The main issue is {+}traceability{+}, we're not trying to outright prevent
these queries. I'm not sure if guardrails in 4.1+ would be a better option
here, it may be worth further discussion. But it would be nice to fix the WARN
message with Stefan's PR in 4.0.x.
was (Author: bschoeni):
[~blerer] [~smiklosovic] both application code and ad hoc queries can generate
these WARN messages. Ad hoc may be something like select count(*). We
wouldn't want to block those and, in any event, if they're too slow they'll
time out.
The situations where the existing warning is a problem is with application
code. There may be multiple application teams using the same database, and
when the aggregation warn message is seen in the log, it's difficult to tie it
back to the code or even know which application ran the query. Multiple,
unrelated applications may share the same database using different keyspaces,
and the existing WARN message doesn't mention the keyspace.
We usually have to turn on settraceprobability to capture the queries. Having
just the keyspace, table and aggregation type in the cassandra.log would solve
90% of the issues.
The main issue is {+}traceability{+}, we're not trying to outright prevent
these queries. I'm not sure if guardrails in 4.1+ would be a better option
here, it may be worth further discussion. But it would be nice to fix the WARN
message with Stefan's PR in 4.0.x.
> Warning message on aggregation queries doesn't specify table name or
> aggregate type
> ---
>
> Key: CASSANDRA-18219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18219
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The existing aggregation query warning messages in cassandra.log from
> SelectStatement.java:
> _'[WARN] Aggregation query used without partition key'_
> {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys
> (IN restriction)'_
> are missing the helpful details which would assist users in quickly
> identifying the offending query. The table name and type of aggregation would
> be useful additions in the WARN message.
> In addition, following the example of the slow query logger and printing out
> the query string at DEBUG level would be especially helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18051) Optimize test_network_topology_strategy and test_network_topology_strategy_each_quorum for large containers in CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683413#comment-17683413
]
Ekaterina Dimitrova commented on CASSANDRA-18051:
-
If there are no objections should I move forward with the last suggestion?
> Optimize test_network_topology_strategy and
> test_network_topology_strategy_each_quorum for large containers in CircleCI
> ---
>
> Key: CASSANDRA-18051
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18051
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>
> As pointed in CASSANDRA-18001, there are two Python Large DTests that fail
> because of not enough resources with large containers. [~adelapena] looked
> into them and suggested that the tests themselves can be optimized for
> running on Large containers instead of using XLarge containers for only two
> tests from the whole suite.
> {quote}As for the two tests starting nine nodes each,
> {{test_network_topology_strategy}} and
> {{{}test_network_topology_strategy_each_quorum{}}}, I think they were added
> by CASSANDRA-10584 to test multi-DC consistency levels. I wonder if they
> actually need to start 3 data centers with 3 nodes each. Would two data
> centers with 3 nodes be enough to test multi-DC consistency levels with
> multiple data centers? 6 nodes should be doable on Circle with the current
> resources.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17773) Incorrect cassandra.logdir on Debian systems
[
https://issues.apache.org/jira/browse/CASSANDRA-17773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683410#comment-17683410
]
Claude Warren commented on CASSANDRA-17773:
---
Error at linke 25 of the new nodetool script.
it currently reads:
{{ENV_PRESERVE="$ENV_PRESERVE MAX_HEAP_SIZE JVM_OPTS"}}
but should read:
{{CASSANDRA_ENV_PRESERVE="$CASSENDRA_ENV_PRESERVE MAX_HEAP_SIZE JVM_OPTS"}}
> Incorrect cassandra.logdir on Debian systems
>
>
> Key: CASSANDRA-17773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17773
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Eric Evans
>Assignee: Claude Warren
>Priority: Normal
> Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 4.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> The Debian packaging patches bin/cassandra to use /var/log/cassandra for
> logs, it does so conditionally however, only if CASSANDRA_LOG_DIR is unset.
> This occurs _after_ cassandra-env.sh is sourced though, which also sets
> CASSANDRA_LOG_DIR if unset (to $CASSANDRA_HOME/logs). The result is that
> -Dcassandra.lodir is set to /usr/share/cassandra/logs on Debian systems.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17773) Incorrect cassandra.logdir on Debian systems
[
https://issues.apache.org/jira/browse/CASSANDRA-17773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683374#comment-17683374
]
Stefan Miklosovic edited comment on CASSANDRA-17773 at 2/2/23 1:21 PM:
---
_As you can see the new file is sourced as you suggest, though without the
$CASSANDRA_HOME_
All I am saying is to change
{code:java}
. ./cassandra-conf.sh
{code}
to
{code:java}
. $CASSANDRA_HOME/conf/cassandra-conf.sh
{code}
or wherever that cassandra-conf.sh ends up being located. Right now it is in
"bin". Would you mind to move it to "conf" directory instead?
Something like this would be preferable:
{code}
if [ -z $CASSANDRA_HOME ]; then
. $(dirname "$0")/../conf/cassandra-conf.sh
else
. $CASSANDRA_HOME/conf/cassandra-conf.sh
fi
{code}
was (Author: smiklosovic):
_As you can see the new file is sourced as you suggest, though without the
$CASSANDRA_HOME_
All I am saying is to change
{code:java}
. ./cassandra-conf.sh
{code}
to
{code:java}
. $CASSANDRA_HOME/conf/cassandra-conf.sh
{code}
or wherever that cassandra-conf.sh ends up being located. Right now it is in
"bin". Would you mind to move it to "conf" directory instead?
> Incorrect cassandra.logdir on Debian systems
>
>
> Key: CASSANDRA-17773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17773
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Eric Evans
>Assignee: Claude Warren
>Priority: Normal
> Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 4.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> The Debian packaging patches bin/cassandra to use /var/log/cassandra for
> logs, it does so conditionally however, only if CASSANDRA_LOG_DIR is unset.
> This occurs _after_ cassandra-env.sh is sourced though, which also sets
> CASSANDRA_LOG_DIR if unset (to $CASSANDRA_HOME/logs). The result is that
> -Dcassandra.lodir is set to /usr/share/cassandra/logs on Debian systems.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17773) Incorrect cassandra.logdir on Debian systems
[
https://issues.apache.org/jira/browse/CASSANDRA-17773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683387#comment-17683387
]
Stefan Miklosovic edited comment on CASSANDRA-17773 at 2/2/23 1:17 PM:
---
also, I am not completely sure why that is happening, but when I execute
"bin/nodetool status", it gives me this output:
{code}
$ ./bin/nodetool status
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset
(Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
. omitted for brevity
Datacenter: datacenter1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- AddressLoadTokens Owns (effective) Host ID
Rack
UN 127.0.0.1 238.76 KiB 16 100.0%
2f747977-340c-48f1-a641-4e0b933e9661 rack1
{code}
where is that log before the nodetool invocation coming from?
EDIT:
It is coming from here (1) as we are sourcing cassandra-env.sh in
cassandra-conf.sh. I guess we need to find some way how to not put it among
JVM_OPTS when it is run as a script to not pollute the output unnecessarily.
(1) https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L191
EDIT 2:
The reason it is not happening in trunk is that there is currently this in
nodetool script:
{code}
# Run cassandra-env.sh to pick up JMX_PORT
if [ -f "$CASSANDRA_CONF/cassandra-env.sh" ]; then
JVM_OPTS_SAVE=$JVM_OPTS
MAX_HEAP_SIZE_SAVE=$MAX_HEAP_SIZE
. "$CASSANDRA_CONF/cassandra-env.sh"
MAX_HEAP_SIZE=$MAX_HEAP_SIZE_SAVE
JVM_OPTS="$JVM_OPTS_SAVE"
fi
{code}
So we are saving options to JVM_OPTS_SAVE, then run cassandra-env.sh which sets
its own JVM_OPTS and then we return to JVM_OPTS_SAVE. I think this is done
somehow differently in the current version hence the log is written with these
compiler hints.
was (Author: smiklosovic):
also, I am not completely sure why that is happening, but when I execute
"bin/nodetool status", it gives me this output:
{code}
$ ./bin/nodetool status
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset
(Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
. omitted for brevity
Datacenter: datacenter1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- AddressLoadTokens Owns (effective) Host ID
Rack
UN 127.0.0.1 238.76 KiB 16 100.0%
2f747977-340c-48f1-a641-4e0b933e9661 rack1
{code}
where is that log before the nodetool invocation coming from?
EDIT:
It is coming from here (1) as we are sourcing cassandra-env.sh in
cassandra-conf.sh. I guess we need to find some way how to not put it among
JVM_OPTS when it is run as a script to not pollute the output unnecessarily.
(1) https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L191
> Incorrect cassandra.logdir on Debian systems
>
>
> Key: CASSANDRA-17773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17773
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Eric Evans
>Assignee: Claude Warren
>Priority: Normal
> Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 4.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> The Debian packaging patches bin/cassandra to use /var/log/cassandra for
> logs, it does so conditionally however, only if CASSANDRA_LOG_DIR is unset.
> This occurs _after_ cassandra-env.sh is sourced though, which also sets
> CASSANDRA_LOG_DIR if unset (to $CASSANDRA_HOME/logs). The result is that
> -Dcassandra.lodir is set to /usr/share/cassandra/logs on Debian systems.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail:
[jira] [Comment Edited] (CASSANDRA-17773) Incorrect cassandra.logdir on Debian systems
[
https://issues.apache.org/jira/browse/CASSANDRA-17773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683387#comment-17683387
]
Stefan Miklosovic edited comment on CASSANDRA-17773 at 2/2/23 1:00 PM:
---
also, I am not completely sure why that is happening, but when I execute
"bin/nodetool status", it gives me this output:
{code}
$ ./bin/nodetool status
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset
(Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
. omitted for brevity
Datacenter: datacenter1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- AddressLoadTokens Owns (effective) Host ID
Rack
UN 127.0.0.1 238.76 KiB 16 100.0%
2f747977-340c-48f1-a641-4e0b933e9661 rack1
{code}
where is that log before the nodetool invocation coming from?
EDIT:
It is coming from here (1) as we are sourcing cassandra-env.sh in
cassandra-conf.sh. I guess we need to find some way how to not put it among
JVM_OPTS when it is run as a script to not pollute the output unnecessarily.
(1) https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L191
was (Author: smiklosovic):
also, I am not completely sure why that is happening, but when I execute
"bin/nodetool status", it gives me this output:
{code}
$ ./bin/nodetool status
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset
(Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
. omitted for brevity
Datacenter: datacenter1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- AddressLoadTokens Owns (effective) Host ID
Rack
UN 127.0.0.1 238.76 KiB 16 100.0%
2f747977-340c-48f1-a641-4e0b933e9661 rack1
{code}
where is that log before the nodetool invocation coming from?
> Incorrect cassandra.logdir on Debian systems
>
>
> Key: CASSANDRA-17773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17773
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Eric Evans
>Assignee: Claude Warren
>Priority: Normal
> Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 4.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> The Debian packaging patches bin/cassandra to use /var/log/cassandra for
> logs, it does so conditionally however, only if CASSANDRA_LOG_DIR is unset.
> This occurs _after_ cassandra-env.sh is sourced though, which also sets
> CASSANDRA_LOG_DIR if unset (to $CASSANDRA_HOME/logs). The result is that
> -Dcassandra.lodir is set to /usr/share/cassandra/logs on Debian systems.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17698) sstabledump errors when dumping data from index
[
https://issues.apache.org/jira/browse/CASSANDRA-17698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683393#comment-17683393
]
maxwellguo commented on CASSANDRA-17698:
if so I can open new tickets
> sstabledump errors when dumping data from index
> ---
>
> Key: CASSANDRA-17698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17698
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Stefan Miklosovic
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 4.2
>
> Time Spent: 12h 40m
> Remaining Estimate: 0h
>
> {code:java}
> cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE ks1.tb1 ( id text, name text, primary key (id));
> cqlsh> CREATE INDEX IF NOT EXISTS ON ks1.tb1(name);
> cqlsh> INSERT INTO ks1.tb1 (id, name ) VALUES ( '1', 'Joe');
> cqlsh> exit
> ./bin/nodetool flush
> ./tools/bin/sstabledump
> data/data/ks1/tb1-1c3c5f10ee4711ecab82eda2f44200b3/.tb1_name_idx/nb-1-big-Data.db
>
> [
> {
> "partition" : {
> "key" : [ "Joe" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 17,
> "clustering" : [ ] } ] } ]Exception in thread "main"
> java.lang.UnsupportedOperationException
> at
> org.apache.cassandra.db.marshal.PartitionerDefinedOrder.toJSONString(PartitionerDefinedOrder.java:87)
> at
> org.apache.cassandra.db.marshal.AbstractType.toJSONString(AbstractType.java:187)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeClustering(JsonTransformer.java:372)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:269)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:235)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:113)
> at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:214) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17698) sstabledump errors when dumping data from index
[
https://issues.apache.org/jira/browse/CASSANDRA-17698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683392#comment-17683392
]
maxwellguo commented on CASSANDRA-17698:
I think make this fix as 5.0 and open a separate ticket for what you said to
fix bug for older branch is ok for me. and I think that old branches should be
3.0/3.11/4.0/4.1 and trunk.
> sstabledump errors when dumping data from index
> ---
>
> Key: CASSANDRA-17698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17698
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Stefan Miklosovic
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 4.2
>
> Time Spent: 12h 40m
> Remaining Estimate: 0h
>
> {code:java}
> cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE ks1.tb1 ( id text, name text, primary key (id));
> cqlsh> CREATE INDEX IF NOT EXISTS ON ks1.tb1(name);
> cqlsh> INSERT INTO ks1.tb1 (id, name ) VALUES ( '1', 'Joe');
> cqlsh> exit
> ./bin/nodetool flush
> ./tools/bin/sstabledump
> data/data/ks1/tb1-1c3c5f10ee4711ecab82eda2f44200b3/.tb1_name_idx/nb-1-big-Data.db
>
> [
> {
> "partition" : {
> "key" : [ "Joe" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 17,
> "clustering" : [ ] } ] } ]Exception in thread "main"
> java.lang.UnsupportedOperationException
> at
> org.apache.cassandra.db.marshal.PartitionerDefinedOrder.toJSONString(PartitionerDefinedOrder.java:87)
> at
> org.apache.cassandra.db.marshal.AbstractType.toJSONString(AbstractType.java:187)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeClustering(JsonTransformer.java:372)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:269)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:235)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:113)
> at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:214) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17773) Incorrect cassandra.logdir on Debian systems
[
https://issues.apache.org/jira/browse/CASSANDRA-17773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683387#comment-17683387
]
Stefan Miklosovic commented on CASSANDRA-17773:
---
also, I am not completely sure why that is happening, but when I execute
"bin/nodetool status", it gives me this output:
{code}
$ ./bin/nodetool status
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset
(Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: dontinline
org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize
(Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
. omitted for brevity
Datacenter: datacenter1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- AddressLoadTokens Owns (effective) Host ID
Rack
UN 127.0.0.1 238.76 KiB 16 100.0%
2f747977-340c-48f1-a641-4e0b933e9661 rack1
{code}
where is that log before the nodetool invocation coming from?
> Incorrect cassandra.logdir on Debian systems
>
>
> Key: CASSANDRA-17773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17773
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Eric Evans
>Assignee: Claude Warren
>Priority: Normal
> Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 4.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> The Debian packaging patches bin/cassandra to use /var/log/cassandra for
> logs, it does so conditionally however, only if CASSANDRA_LOG_DIR is unset.
> This occurs _after_ cassandra-env.sh is sourced though, which also sets
> CASSANDRA_LOG_DIR if unset (to $CASSANDRA_HOME/logs). The result is that
> -Dcassandra.lodir is set to /usr/share/cassandra/logs on Debian systems.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17773) Incorrect cassandra.logdir on Debian systems
[
https://issues.apache.org/jira/browse/CASSANDRA-17773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683374#comment-17683374
]
Stefan Miklosovic commented on CASSANDRA-17773:
---
_As you can see the new file is sourced as you suggest, though without the
$CASSANDRA_HOME_
All I am saying is to change
{code:java}
. ./cassandra-conf.sh
{code}
to
{code:java}
. $CASSANDRA_HOME/conf/cassandra-conf.sh
{code}
or wherever that cassandra-conf.sh ends up being located. Right now it is in
"bin". Would you mind to move it to "conf" directory instead?
> Incorrect cassandra.logdir on Debian systems
>
>
> Key: CASSANDRA-17773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17773
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Eric Evans
>Assignee: Claude Warren
>Priority: Normal
> Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 4.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> The Debian packaging patches bin/cassandra to use /var/log/cassandra for
> logs, it does so conditionally however, only if CASSANDRA_LOG_DIR is unset.
> This occurs _after_ cassandra-env.sh is sourced though, which also sets
> CASSANDRA_LOG_DIR if unset (to $CASSANDRA_HOME/logs). The result is that
> -Dcassandra.lodir is set to /usr/share/cassandra/logs on Debian systems.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18219) Warning message on aggregation queries doesn't specify table name or aggregate type
[
https://issues.apache.org/jira/browse/CASSANDRA-18219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683365#comment-17683365
]
Stefan Miklosovic edited comment on CASSANDRA-18219 at 2/2/23 12:01 PM:
[~blerer] we could do guardrails as well, sure. By default it would be on warn
and if turned on it would fail the query. We have done similar logic in
CASSANDRA-18042.
[~bschoeni] would be guardrail viable option for you? Do we want to emit
warnings or failures or we just enable / disable these queries?
The PR for simply logging it is here:
[https://github.com/apache/cassandra/pull/2128/files]
I will wait until there is a decision whether a guardrail is not better option
before implementing that instead.
was (Author: smiklosovic):
[~blerer] we could do guardrails as well, sure. By default it would be on warn
and if turned on it would fail the query. We have done similar logic in
CASSANDRA-18042.
[~bschoeni] would be guardrail viable option for you? Do we want to emit
warnings or failures or we just enable / disable these queries?
> Warning message on aggregation queries doesn't specify table name or
> aggregate type
> ---
>
> Key: CASSANDRA-18219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18219
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The existing aggregation query warning messages in cassandra.log from
> SelectStatement.java:
> _'[WARN] Aggregation query used without partition key'_
> {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys
> (IN restriction)'_
> are missing the helpful details which would assist users in quickly
> identifying the offending query. The table name and type of aggregation would
> be useful additions in the WARN message.
> In addition, following the example of the slow query logger and printing out
> the query string at DEBUG level would be especially helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18219) Warning message on aggregation queries doesn't specify table name or aggregate type
[
https://issues.apache.org/jira/browse/CASSANDRA-18219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683365#comment-17683365
]
Stefan Miklosovic edited comment on CASSANDRA-18219 at 2/2/23 11:52 AM:
[~blerer] we could do guardrails as well, sure. By default it would be on warn
and if turned on it would fail the query. We have done similar logic in
CASSANDRA-18042.
[~bschoeni] would be guardrail viable option for you? Do we want to emit
warnings or failures or we just enable / disable these queries?
was (Author: smiklosovic):
[~blerer] we could do guardrails as well, sure. By default it would be on warn
and if turned on it would fail the query. We have done similar logic in
CASSANDRA-18042.
> Warning message on aggregation queries doesn't specify table name or
> aggregate type
> ---
>
> Key: CASSANDRA-18219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18219
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The existing aggregation query warning messages in cassandra.log from
> SelectStatement.java:
> _'[WARN] Aggregation query used without partition key'_
> {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys
> (IN restriction)'_
> are missing the helpful details which would assist users in quickly
> identifying the offending query. The table name and type of aggregation would
> be useful additions in the WARN message.
> In addition, following the example of the slow query logger and printing out
> the query string at DEBUG level would be especially helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18219) Warning message on aggregation queries doesn't specify table name or aggregate type
[
https://issues.apache.org/jira/browse/CASSANDRA-18219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683365#comment-17683365
]
Stefan Miklosovic commented on CASSANDRA-18219:
---
[~blerer] we could do guardrails as well, sure. By default it would be on warn
and if turned on it would fail the query. We have done similar logic in
CASSANDRA-18042.
> Warning message on aggregation queries doesn't specify table name or
> aggregate type
> ---
>
> Key: CASSANDRA-18219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18219
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The existing aggregation query warning messages in cassandra.log from
> SelectStatement.java:
> _'[WARN] Aggregation query used without partition key'_
> {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys
> (IN restriction)'_
> are missing the helpful details which would assist users in quickly
> identifying the offending query. The table name and type of aggregation would
> be useful additions in the WARN message.
> In addition, following the example of the slow query logger and printing out
> the query string at DEBUG level would be especially helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
