[jira] [Commented] (CASSANDRA-17072) DebuggableThreadPoolExecutor does not propagate client warnings
[ https://issues.apache.org/jira/browse/CASSANDRA-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444325#comment-17444325 ] Berenguer Blasi commented on CASSANDRA-17072: - [~jlewandowski] did you trigger CI as well? > DebuggableThreadPoolExecutor does not propagate client warnings > --- > > Key: CASSANDRA-17072 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17072 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.0.x > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Extracted this as a separate ticket per discussion on CASSANDRA-17044 > The problem is in {{DebuggableThreadPoolExecutor}} - it does not propagate > executor locals to the thread when tracing is disabled. It is probably > expected that we propagate the state if at least one executor local is > defined, but we only check for tracing and completely ignore client warnings. > The attached PR fixes the problem, adds some tests and reverts a workaround > for client warnings in some schema alteration statements implemented in > CASSANDRA-16296 (described below). > > h4. Old description - still valid, but this is just a manifestation of the > problem rather than the problem itself > This seemed to be screwed a bit. In just two schema alteration statements we > collect client warnings which are captured during the transformation into a > local collection. > I guess it is done that way because the transformation is being executed in a > different stage (migration) and client warnings collected in that stage are > not present in the stage where the query is executed. > Then, the client warnings are retrieved using {{clientWarnings}} method and > added to the captured client warnings in the stage which is executing the > query. > This mechanism was implemented only in two schema alteration statements. It > is possible that for other ones the client warnings can simply get lost. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17072) DebuggableThreadPoolExecutor does not propagate client warnings
[ https://issues.apache.org/jira/browse/CASSANDRA-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444318#comment-17444318 ] Jacek Lewandowski commented on CASSANDRA-17072: --- I rebased both PRs and squashed - I'd really appreciate if somebody confirm this https://github.com/apache/cassandra/pull/1320/files#diff-884063e4549a5202588049240e62b219f7d14f75d4e983416a61815d8467R173 changes is ok in trunk - although it seems obvious for me, I might have missed something > DebuggableThreadPoolExecutor does not propagate client warnings > --- > > Key: CASSANDRA-17072 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17072 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.0.x > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Extracted this as a separate ticket per discussion on CASSANDRA-17044 > The problem is in {{DebuggableThreadPoolExecutor}} - it does not propagate > executor locals to the thread when tracing is disabled. It is probably > expected that we propagate the state if at least one executor local is > defined, but we only check for tracing and completely ignore client warnings. > The attached PR fixes the problem, adds some tests and reverts a workaround > for client warnings in some schema alteration statements implemented in > CASSANDRA-16296 (described below). > > h4. Old description - still valid, but this is just a manifestation of the > problem rather than the problem itself > This seemed to be screwed a bit. In just two schema alteration statements we > collect client warnings which are captured during the transformation into a > local collection. > I guess it is done that way because the transformation is being executed in a > different stage (migration) and client warnings collected in that stage are > not present in the stage where the query is executed. > Then, the client warnings are retrieved using {{clientWarnings}} method and > added to the captured client warnings in the stage which is executing the > query. > This mechanism was implemented only in two schema alteration statements. It > is possible that for other ones the client warnings can simply get lost. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16912) Rename cassandra-topology.properties to cassandra-topology.properties.example
[ https://issues.apache.org/jira/browse/CASSANDRA-16912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16912: Status: Ready to Commit (was: Review In Progress) > Rename cassandra-topology.properties to cassandra-topology.properties.example > - > > Key: CASSANDRA-16912 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16912 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Gossip >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1 > > > Far too often people leave this file in place when using GPFS, when it is > only intended to be using during migration away from PFS. If we simply > rename this file, then the GPFS camp won't trip over it (without going to > effort to do so at least) and the PFS camp can simple rename the file > before/after editing it. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16912) Rename cassandra-topology.properties to cassandra-topology.properties.example
[ https://issues.apache.org/jira/browse/CASSANDRA-16912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444311#comment-17444311 ] Berenguer Blasi commented on CASSANDRA-16912: - Well, it finished and if I have parsed correctly the failures vs nightlies that failed vs known offenders vs the ones that look unrelated I think we're good to go. Please check failures yourself as well to be on the safe side. > Rename cassandra-topology.properties to cassandra-topology.properties.example > - > > Key: CASSANDRA-16912 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16912 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Gossip >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1 > > > Far too often people leave this file in place when using GPFS, when it is > only intended to be using during migration away from PFS. If we simply > rename this file, then the GPFS camp won't trip over it (without going to > effort to do so at least) and the PFS camp can simple rename the file > before/after editing it. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16912) Rename cassandra-topology.properties to cassandra-topology.properties.example
[ https://issues.apache.org/jira/browse/CASSANDRA-16912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16912: Status: Review In Progress (was: Patch Available) > Rename cassandra-topology.properties to cassandra-topology.properties.example > - > > Key: CASSANDRA-16912 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16912 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Gossip >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1 > > > Far too often people leave this file in place when using GPFS, when it is > only intended to be using during migration away from PFS. If we simply > rename this file, then the GPFS camp won't trip over it (without going to > effort to do so at least) and the PFS camp can simple rename the file > before/after editing it. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17072) DebuggableThreadPoolExecutor does not propagate client warnings
[ https://issues.apache.org/jira/browse/CASSANDRA-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444310#comment-17444310 ] Jacek Lewandowski commented on CASSANDRA-17072: --- I'll rebase and squash > DebuggableThreadPoolExecutor does not propagate client warnings > --- > > Key: CASSANDRA-17072 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17072 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.0.x > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Extracted this as a separate ticket per discussion on CASSANDRA-17044 > The problem is in {{DebuggableThreadPoolExecutor}} - it does not propagate > executor locals to the thread when tracing is disabled. It is probably > expected that we propagate the state if at least one executor local is > defined, but we only check for tracing and completely ignore client warnings. > The attached PR fixes the problem, adds some tests and reverts a workaround > for client warnings in some schema alteration statements implemented in > CASSANDRA-16296 (described below). > > h4. Old description - still valid, but this is just a manifestation of the > problem rather than the problem itself > This seemed to be screwed a bit. In just two schema alteration statements we > collect client warnings which are captured during the transformation into a > local collection. > I guess it is done that way because the transformation is being executed in a > different stage (migration) and client warnings collected in that stage are > not present in the stage where the query is executed. > Then, the client warnings are retrieved using {{clientWarnings}} method and > added to the captured client warnings in the stage which is executing the > query. > This mechanism was implemented only in two schema alteration statements. It > is possible that for other ones the client warnings can simply get lost. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17132) Fix startup issue with internode_application_timeout_in_ms
[ https://issues.apache.org/jira/browse/CASSANDRA-17132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-17132: Status: Changes Suggested (was: Review In Progress) Ok moved to suggest changes just for my benefit on tracking stuff until the PR gets those renames removed. > Fix startup issue with internode_application_timeout_in_ms > -- > > Key: CASSANDRA-17132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17132 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.x > > > While testing my patch for CASSANDRA-17131 I found that there is a problem > with _internode_application_timeout_in_ms_ in 4.0 and trunk. > Seems to me that we can just safely remove it for now? -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17141) internode_send_buff_size_in_bytes and internode_recv_buff_size_in_bytes from cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17141: Reviewers: David Capwell, Ekaterina Dimitrova Status: Review In Progress (was: Patch Available) > internode_send_buff_size_in_bytes and internode_recv_buff_size_in_bytes from > cassandra.yaml > > > Key: CASSANDRA-17141 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17141 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.x > > > While working on CASSANDRA-17132 I found that in 4.0 > _internode_send_buff_size_in_bytes_ and _internode_recv_buff_size_in_bytes_ > were renamed to > _internode_socket_send_buffer_size_in_bytes_ and > _internode_socket_receive_buffer_size_in_bytes_ > but only internally, not in cassandra.yaml. In cassandra.yaml we still have > the old names. > Options I see: > - we decide that there is difference to be acknowledged so we just rename > and announce the new parameters in NEWS.txt > - it is confirmed that just a rename was needed. Then do we want to add > backward compatibility so that anyone who is still looking to start upgrade > from 3 or 3.11 can directly go smoothly to 4.0.2 without breaking change? > CC [~benedict] , [~ifesdjeen] and [~aleksey] for your opinion what will be > the right way to handle this issue. > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17141) internode_send_buff_size_in_bytes and internode_recv_buff_size_in_bytes from cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17141: Test and Documentation Plan: ||[Patch 4.0|https://github.com/ekaterinadimitrova2/cassandra/commit/304683bc86b90c20f414c80d999c958116556b1b]|[CI J8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1199/workflows/a3f3cb0f-d341-4f01-986e-ead713fd7587/jobs/7168]|[CI J11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1199/workflows/5a5f5ec1-31f9-4af4-9ad2-d80c1e54674c]| ||[Patch trunk|https://github.com/apache/cassandra/commit/3dc31ccca771fc1adfc01e3feda8fffb425162e7]|[CI J8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1200/workflows/cfd02666-56aa-4081-ad7a-09c463978d89]|[CI J11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1200/workflows/70f9725d-fc27-4e8c-b6ca-39319cb68e6a]| Status: Patch Available (was: In Progress) > internode_send_buff_size_in_bytes and internode_recv_buff_size_in_bytes from > cassandra.yaml > > > Key: CASSANDRA-17141 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17141 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.x > > > While working on CASSANDRA-17132 I found that in 4.0 > _internode_send_buff_size_in_bytes_ and _internode_recv_buff_size_in_bytes_ > were renamed to > _internode_socket_send_buffer_size_in_bytes_ and > _internode_socket_receive_buffer_size_in_bytes_ > but only internally, not in cassandra.yaml. In cassandra.yaml we still have > the old names. > Options I see: > - we decide that there is difference to be acknowledged so we just rename > and announce the new parameters in NEWS.txt > - it is confirmed that just a rename was needed. Then do we want to add > backward compatibility so that anyone who is still looking to start upgrade > from 3 or 3.11 can directly go smoothly to 4.0.2 without breaking change? > CC [~benedict] , [~ifesdjeen] and [~aleksey] for your opinion what will be > the right way to handle this issue. > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17141) internode_send_buff_size_in_bytes and internode_recv_buff_size_in_bytes from cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444269#comment-17444269 ] Ekaterina Dimitrova commented on CASSANDRA-17141: - [~dcapwell] : ||[Patch 4.0|https://github.com/ekaterinadimitrova2/cassandra/commit/304683bc86b90c20f414c80d999c958116556b1b]|[CI J8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1199/workflows/a3f3cb0f-d341-4f01-986e-ead713fd7587/jobs/7168]|[CI J11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1199/workflows/5a5f5ec1-31f9-4af4-9ad2-d80c1e54674c]| ||[Patch trunk|https://github.com/apache/cassandra/commit/3dc31ccca771fc1adfc01e3feda8fffb425162e7]|[CI J8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1200/workflows/cfd02666-56aa-4081-ad7a-09c463978d89]|[CI J11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1200/workflows/70f9725d-fc27-4e8c-b6ca-39319cb68e6a]| There are only a few failures - two failures with java 8 which are the well known bootstrap tests and the python upgrade tests for which we also have a ticket already. 4.0 failures: readWriteDuringBootstrapTest - CASSANDRA-17139 bootstrapTest - CASSANDRA17076 Python upgrade tests - CASSANDRA-17086, CASSANDRA-17140, CASSANDRA-17080 trunk failures: TestHintedHandoff#test_hintedhandoff_window - CASSANDRA-17144 testWithoutStreamingEntireSSTablesWithCompression - this one was just committed when I pushed the patch > internode_send_buff_size_in_bytes and internode_recv_buff_size_in_bytes from > cassandra.yaml > > > Key: CASSANDRA-17141 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17141 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.x > > > While working on CASSANDRA-17132 I found that in 4.0 > _internode_send_buff_size_in_bytes_ and _internode_recv_buff_size_in_bytes_ > were renamed to > _internode_socket_send_buffer_size_in_bytes_ and > _internode_socket_receive_buffer_size_in_bytes_ > but only internally, not in cassandra.yaml. In cassandra.yaml we still have > the old names. > Options I see: > - we decide that there is difference to be acknowledged so we just rename > and announce the new parameters in NEWS.txt > - it is confirmed that just a rename was needed. Then do we want to add > backward compatibility so that anyone who is still looking to start upgrade > from 3 or 3.11 can directly go smoothly to 4.0.2 without breaking change? > CC [~benedict] , [~ifesdjeen] and [~aleksey] for your opinion what will be > the right way to handle this issue. > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17144) Fix test TestHintedHandoff#test_hintedhandoff_window
[ https://issues.apache.org/jira/browse/CASSANDRA-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444243#comment-17444243 ] Ekaterina Dimitrova commented on CASSANDRA-17144: - One more example: https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1200/workflows/cfd02666-56aa-4081-ad7a-09c463978d89/jobs/7198/tests#failed-test-0 > Fix test TestHintedHandoff#test_hintedhandoff_window > > > Key: CASSANDRA-17144 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17144 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: David Capwell >Priority: Normal > Fix For: 4.x > > > This test fails from time to time (see butler > https://butler.cassandra.apache.org/#/ci/upstream/workflow/Cassandra-trunk/failure/hintedhandoff_test/TestHintedHandoff/test_hintedhandoff_window) > Example: > https://app.circleci.com/pipelines/github/dcapwell/cassandra/1101/workflows/7afb1c7e-8330-4ac6-963d-d7864282f2f3/jobs/7877 > {code} > # Ensure second and third datasets are not present > for x in range(100, 300): > > query_c1c2(session, x, ConsistencyLevel.ONE, > > tolerate_missing=True, must_be_missing=True) > hintedhandoff_test.py:264: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > tools/data.py:44: in query_c1c2 > assertions.assert_length_equal(rows, 0) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > object_with_length = [Row(c1='value1', c2='value2')], expected_length = 0 > def assert_length_equal(object_with_length, expected_length): > """ > Assert an object has a specific length. > @param object_with_length The object whose length will be checked > @param expected_length The expected length of the object > > Examples: > assert_length_equal(res, nb_counter) > """ > assert len(object_with_length) == expected_length, \ > "Expected {} to have length {}, but instead is of length {}"\ > > .format(object_with_length, expected_length, > > len(object_with_length)) > E AssertionError: Expected [Row(c1='value1', c2='value2')] to have > length 0, but instead is of length 1 > tools/assertions.py:269: AssertionError > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17040) Upgrade Snappy version to support Apple M1
[ https://issues.apache.org/jira/browse/CASSANDRA-17040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444224#comment-17444224 ] Yuqi Gu commented on CASSANDRA-17040: - [~djoshi] [~stefan.miklosovic] Could you please give some comments on this, thanks. > Upgrade Snappy version to support Apple M1 > -- > > Key: CASSANDRA-17040 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17040 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Compression >Reporter: Yuqi Gu >Assignee: Yuqi Gu >Priority: Normal > Fix For: 4.1 > > Attachments: UTs.txt, UTs_2.txt > > Time Spent: 10m > Remaining Estimate: 0h > > Some Unit test cases were failed in Apple M1: > > {code:java} > [junit-timeout] Testcase: > testTableOptions(org.apache.cassandra.cql3.validation.miscellaneous.OverflowTest): > Caused an ERROR > [junit-timeout] SnappyCompressor.create() threw an error: > java.lang.NoClassDefFoundError Could not initialize class > org.xerial.snappy.Snappy > [junit-timeout] org.apache.cassandra.exceptions.ConfigurationException: > SnappyCompressor.create() threw an error: java.lang.NoClassDefFoundError > Could not initialize class org.xerial.snappy.Snappy > [junit-timeout] at > org.apache.cassandra.schema.CompressionParams.createCompressor(CompressionParams.java:344) > [junit-timeout] at > org.apache.cassandra.schema.CompressionParams.(CompressionParams.java:211) > [junit-timeout] at > org.apache.cassandra.schema.CompressionParams.fromMap(CompressionParams.java:124) > [junit-timeout] at > org.apache.cassandra.cql3.statements.schema.TableAttributes.build(TableAttributes.java:110) > [junit-timeout] at > org.apache.cassandra.cql3.statements.schema.TableAttributes.validate(TableAttributes.java:58) > [junit-timeout] at > > ... > .. > > {code} > > Snappy-java added M1 support since > 1.1.8.2.([https://github.com/xerial/snappy-java/pull/268).] > So let's upgrade snappy-java dependency to the latest release 1.1.8.4. > > > > > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16310) Track top partitions (by size and tombstone count) and display in nodetool tablestats
[ https://issues.apache.org/jira/browse/CASSANDRA-16310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444220#comment-17444220 ] David Capwell commented on CASSANDRA-16310: --- [~yifanc] [~marcuse] can you review the PR? I messed up in the rebase and all feedback from [~yifanc] got merged into the original commit; so there is only 2 commits (patch, ci). I fixed all Yifan's concerns > Track top partitions (by size and tombstone count) and display in nodetool > tablestats > - > > Key: CASSANDRA-16310 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16310 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.x > > > We should track the top partitions by size and tombstone count and display in > {{nodetool tablestats}} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17072) DebuggableThreadPoolExecutor does not propagate client warnings
[ https://issues.apache.org/jira/browse/CASSANDRA-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444203#comment-17444203 ] David Capwell commented on CASSANDRA-17072: --- test results look clean (need to rebase to fix most tests); will review tomorrow. > DebuggableThreadPoolExecutor does not propagate client warnings > --- > > Key: CASSANDRA-17072 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17072 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.0.x > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Extracted this as a separate ticket per discussion on CASSANDRA-17044 > The problem is in {{DebuggableThreadPoolExecutor}} - it does not propagate > executor locals to the thread when tracing is disabled. It is probably > expected that we propagate the state if at least one executor local is > defined, but we only check for tracing and completely ignore client warnings. > The attached PR fixes the problem, adds some tests and reverts a workaround > for client warnings in some schema alteration statements implemented in > CASSANDRA-16296 (described below). > > h4. Old description - still valid, but this is just a manifestation of the > problem rather than the problem itself > This seemed to be screwed a bit. In just two schema alteration statements we > collect client warnings which are captured during the transformation into a > local collection. > I guess it is done that way because the transformation is being executed in a > different stage (migration) and client warnings collected in that stage are > not present in the stage where the query is executed. > Then, the client warnings are retrieved using {{clientWarnings}} method and > added to the captured client warnings in the stage which is executing the > query. > This mechanism was implemented only in two schema alteration statements. It > is possible that for other ones the client warnings can simply get lost. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16310) Track top partitions (by size and tombstone count) and display in nodetool tablestats
[ https://issues.apache.org/jira/browse/CASSANDRA-16310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444202#comment-17444202 ] David Capwell commented on CASSANDRA-16310: --- Took [~marcuse] patch and applied [~yifanc] feedback https://github.com/apache/cassandra/pull/1324 https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=marcuse%2F16310 > Track top partitions (by size and tombstone count) and display in nodetool > tablestats > - > > Key: CASSANDRA-16310 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16310 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.x > > > We should track the top partitions by size and tombstone count and display in > {{nodetool tablestats}} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16310) Track top partitions (by size and tombstone count) and display in nodetool tablestats
[ https://issues.apache.org/jira/browse/CASSANDRA-16310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444200#comment-17444200 ] David Capwell commented on CASSANDRA-16310: --- bq. They fail with the --validate option. No sstables are selected since the corresponding PreviewKind.REPAIRED filters out unrepaired ones. This is because there is no data, you need to run an IR before calling --validate; in my patch I do this and the test passes > Track top partitions (by size and tombstone count) and display in nodetool > tablestats > - > > Key: CASSANDRA-16310 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16310 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.x > > > We should track the top partitions by size and tombstone count and display in > {{nodetool tablestats}} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17155) Increase the buckets count for timer histogram
[ https://issues.apache.org/jira/browse/CASSANDRA-17155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444192#comment-17444192 ] Yifan Cai commented on CASSANDRA-17155: --- PR: https://github.com/apache/cassandra/pull/1323 CI: https://app.circleci.com/pipelines/github/yifan-c/cassandra?branch=CASSANDRA-17155%2Ftrunk The PR increases the bucket count. > Increase the buckets count for timer histogram > -- > > Key: CASSANDRA-17155 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17155 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > EstimatedHistograms used in timers are allocated with 90 buckets, as part of > CASSANDRA-16760. It can represent the highest value of 25 seconds. The > setting is not free from hitting overflow error when calculating the mean > value from the histogram. We can increase the buckets count to 128 to > increase the upper limit to 25,628 seconds, which just covers the range of > 165 buckets prior CASSANDRA-16760. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17155) Increase the buckets count for timer histogram
[ https://issues.apache.org/jira/browse/CASSANDRA-17155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-17155: -- Description: EstimatedHistograms used in timers are allocated with 90 buckets, as part of CASSANDRA-16760. It can represent the highest value of 25 seconds. The setting is not free from hitting overflow error when calculating the mean value from the histogram. We can increase the buckets count to 127 to increase the upper limit to 21,356 seconds, which just covers the range of 164 buckets prior CASSANDRA-16760. (was: EstimatedHistograms used in timers are allocated with 90 buckets, as part of CASSANDRA-16760. It can represent the highest value of 25 seconds. The setting is not free from hitting overflow error when calculating the mean value from the histogram. We can increase the buckets count to 128 to increase the upper limit to 25,628 seconds, which just covers the range of 165 buckets prior CASSANDRA-16760.) > Increase the buckets count for timer histogram > -- > > Key: CASSANDRA-17155 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17155 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > EstimatedHistograms used in timers are allocated with 90 buckets, as part of > CASSANDRA-16760. It can represent the highest value of 25 seconds. The > setting is not free from hitting overflow error when calculating the mean > value from the histogram. We can increase the buckets count to 127 to > increase the upper limit to 21,356 seconds, which just covers the range of > 164 buckets prior CASSANDRA-16760. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16310) Track top partitions (by size and tombstone count) and display in nodetool tablestats
[ https://issues.apache.org/jira/browse/CASSANDRA-16310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444187#comment-17444187 ] David Capwell commented on CASSANDRA-16310: --- bq. The tests cases pass with the --preview option. Looked into this, this looks like a repair bug. {code} CLUSTER.get(1).nodetoolResult("repair", "--preview", KEYSPACE).asserts().success(); {code} this is a IR preview, which looks at the un-repaired data... which should NOT populate... but did... {code} ActiveRepairService.instance.getParentRepairSession(validationRequest.desc.parentSessionId).isIncremental // returns true {code} {code} isIncremental(desc.parentSessionId) // this is what the validation uses // returns false ActiveRepairService.instance.consistent.local.getSession(desc.parentSessionId) // used by isIncremental // returns null {code} > Track top partitions (by size and tombstone count) and display in nodetool > tablestats > - > > Key: CASSANDRA-16310 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16310 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.x > > > We should track the top partitions by size and tombstone count and display in > {{nodetool tablestats}} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17155) Increase the buckets count for timer histogram
[ https://issues.apache.org/jira/browse/CASSANDRA-17155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-17155: -- Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear Impact(13164) Complexity: Low Hanging Fruit Discovered By: User Report Severity: Low Status: Open (was: Triage Needed) > Increase the buckets count for timer histogram > -- > > Key: CASSANDRA-17155 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17155 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > EstimatedHistograms used in timers are allocated with 90 buckets, as part of > CASSANDRA-16760. It can represent the highest value of 25 seconds. The > setting is not free from hitting overflow error when calculating the mean > value from the histogram. We can increase the buckets count to 128 to > increase the upper limit to 25,628 seconds, which just covers the range of > 165 buckets prior CASSANDRA-16760. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17155) Increase the buckets count for timer histogram
Yifan Cai created CASSANDRA-17155: - Summary: Increase the buckets count for timer histogram Key: CASSANDRA-17155 URL: https://issues.apache.org/jira/browse/CASSANDRA-17155 Project: Cassandra Issue Type: Bug Components: Observability/Metrics Reporter: Yifan Cai Assignee: Yifan Cai EstimatedHistograms used in timers are allocated with 90 buckets, as part of CASSANDRA-16760. It can represent the highest value of 25 seconds. The setting is not free from hitting overflow error when calculating the mean value from the histogram. We can increase the buckets count to 128 to increase the upper limit to 25,628 seconds, which just covers the range of 165 buckets prior CASSANDRA-16760. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17121) Allow column_index_size_in_kb to be configurable through nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-17121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRA-17121: --- Reviewers: Dinesh Joshi > Allow column_index_size_in_kb to be configurable through nodetool > - > > Key: CASSANDRA-17121 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17121 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > > Configuring the column_index_size_in_kb setting requires a cassandra.yaml > change and bouncing the instance. > Allowing column_index_size_in_kb to be configurable through nodetool can help > in the operational landscape. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16310) Track top partitions (by size and tombstone count) and display in nodetool tablestats
[ https://issues.apache.org/jira/browse/CASSANDRA-16310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444131#comment-17444131 ] David Capwell commented on CASSANDRA-16310: --- bq. Having setters for max_top_size_partition_count and max_top_tombstone_partition_count suggests that we want to tweak it at runtime. However, the property consumers, TopPartitionTracker and ColumnFamilyStore do not update the value once initialized. So it has no effects when operator update the values. org.apache.cassandra.metrics.TopPartitionTracker.Collector#Collector and org.apache.cassandra.metrics.TopPartitionTracker#merge. So lets say you start up with max_top_tombstone_partition_count=10 and you really want 20 so set it via JMX, the next time we do repair we will populate 20 and will update the reference; so will take affect with the new configs. bq. TopPartitionsTest only has the test cases for full repair. Can we add tests that run with preview repair? (I simply replaced the option in the nodetool commands in the tests.) Going to try to help out with this patch, so will work on this test; just need to do IR + --validate bq. The tests cases pass with the --preview option. Will look into this bq. They fail with the --validate option. No sstables are selected since the corresponding PreviewKind.REPAIRED filters out unrepaired ones. Will look into this bq. The todo comment at TableStatsPrinter line#131 can be removed, since it is implemented. will look into this bq. Nit: consider adding a isValidation() method to PreviewKind. So in ValidationManager#doValidation, we can have validator.getPreviewKind().isValidation(), instead of validator.getPreviewKind() == PreviewKind.REPAIRED, which is less readable. will look into this bq. Document the feature. I will work on a NEWS entry and conf/cassandra.yaml > Track top partitions (by size and tombstone count) and display in nodetool > tablestats > - > > Key: CASSANDRA-16310 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16310 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.x > > > We should track the top partitions by size and tombstone count and display in > {{nodetool tablestats}} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-9633) Add ability to encrypt sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444123#comment-17444123 ] Jeremiah Jordan commented on CASSANDRA-9633: Is there a link to the branch you have been working on? Thanks. > Add ability to encrypt sstables > --- > > Key: CASSANDRA-9633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9633 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Jason Brown >Assignee: shylaja kokoori >Priority: Normal > Labels: encryption, security, sstable > Fix For: 4.x > > > Add option to allow encrypting of sstables. > I have a version of this functionality built on cassandra 2.0 that > piggy-backs on the existing sstable compression functionality and ICompressor > interface (similar in nature to what DataStax Enterprise does). However, if > we're adding the feature to the main OSS product, I'm not sure if we want to > use the pluggable compression framework or if it's worth investigating a > different path. I think there's a lot of upside in reusing the sstable > compression scheme, but perhaps add a new component in cqlsh for table > encryption and a corresponding field in CFMD. > Encryption configuration in the yaml can use the same mechanism as > CASSANDRA-6018 (which is currently pending internal review). -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16630) Migrate to JUnit5
[ https://issues.apache.org/jira/browse/CASSANDRA-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444108#comment-17444108 ] Ekaterina Dimitrova commented on CASSANDRA-16630: - Thank you for your response [~azotcsit] , I am not saying this ticket is a blocker but I was more curious where do we stand on this one and how it will work with Java 17. It seems only JUnit 5 is already on track with Java 17 so it will be on us to figure out how to deal with JUnit 4 and what issues will pop up as it is obsolete. If you try to run ant and Java17 you will need to update it to a newer version, heads up. Also, we haven't migrated yet from nashorn to support the
[cassandra] 04/04: CASSANDRA-16630. Updated Cassandra test formatters.
This is an automated email from the ASF dual-hosted git repository. azotcsit pushed a commit to branch cassandra-16630_junit5 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 8b04e685195d0a9a5f8eb1afa676200af14da689 Author: Aleksei Zotov AuthorDate: Tue Nov 16 00:56:30 2021 +0400 CASSANDRA-16630. Updated Cassandra test formatters. --- build.xml | 155 --- .../CassandraBriefJUnitResultFormatter.java| 309 ++--- .../CassandraXMLJUnitResultFormatter.java | 477 ++--- .../org/apache/cassandra/Junit4SampleTest.java | 2 + .../org/apache/cassandra/Junit5SampleTest.java | 2 + 5 files changed, 257 insertions(+), 688 deletions(-) diff --git a/build.xml b/build.xml index 6b6d71b..46d20d9 100644 --- a/build.xml +++ b/build.xml @@ -1340,6 +1340,10 @@ else=""> + + + + @@ -1348,43 +1352,17 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + @@ -1399,11 +1377,48 @@ - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -2091,26 +2106,27 @@ - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + @@ -2130,13 +2146,14 @@ - - - - - + + + + + + + + diff --git a/test/unit/org/apache/cassandra/CassandraBriefJUnitResultFormatter.java b/test/unit/org/apache/cassandra/CassandraBriefJUnitResultFormatter.java index 88dbc52..8080d81 100644 --- a/test/unit/org/apache/cassandra/CassandraBriefJUnitResultFormatter.java +++ b/test/unit/org/apache/cassandra/CassandraBriefJUnitResultFormatter.java @@ -18,302 +18,45 @@ package org.apache.cassandra; -import java.io.BufferedWriter; -import java.io.IOException; -import java.io.OutputStream; -import java.io.StringWriter; -import java.text.NumberFormat; +import org.junit.platform.launcher.TestIdentifier; -import junit.framework.AssertionFailedError; -import junit.framework.Test; - -import org.apache.tools.ant.BuildException; -import org.apache.tools.ant.taskdefs.optional.junit.IgnoredTestListener; -import org.apache.tools.ant.taskdefs.optional.junit.JUnitResultFormatter; -import org.apache.tools.ant.taskdefs.optional.junit.JUnitTest; -import org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner; -import org.apache.tools.ant.taskdefs.optional.junit.JUnitVersionHelper; -import org.apache.tools.ant.util.FileUtils; -import org.apache.tools.ant.util.StringUtils; +import org.apache.tools.ant.taskdefs.optional.junitlauncher2.LegacyBriefResultFormatter; /** * Prints plain text output of the test to a specified Writer. - * Inspired by the PlainJUnitResultFormatter. - * - * @see FormatterElement - * @see PlainJUnitResultFormatter */ -public class CassandraBriefJUnitResultFormatter implements JUnitResultFormatter, IgnoredTestListener { - -private static final double ONE_SECOND = 1000.0; - +public class CassandraBriefJUnitResultFormatter extends LegacyBriefResultFormatter +{ private static final String tag = System.getProperty("cassandra.testtag", ""); - private static final Boolean keepBriefBrief = Boolean.getBoolean("cassandra.keepBriefBrief"); -/** - * Where to write the log to. - */ -private OutputStream out; - -/** - * Used for writing the results. - */ -private BufferedWriter output; - -/** - * Used as part of formatting the results. - */ -private StringWriter results; - -/** - * Used for writing formatted results to. - */ -private BufferedWriter resultWriter; - -/** - * Formatter for timings. - */ -private NumberFormat
[cassandra] 02/04: CASSANDRA-16630. Copied Ant JUnit classes.
This is an automated email from the ASF dual-hosted git repository. azotcsit pushed a commit to branch cassandra-16630_junit5 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 56a32b87b9d91b50d3a228dff6369ea751714435 Author: Aleksei Zotov AuthorDate: Sun Nov 14 20:07:36 2021 +0400 CASSANDRA-16630. Copied Ant JUnit classes. --- .../AbstractJUnitResultFormatter.java | 308 +++ .../junitlauncher/LegacyBriefResultFormatter.java | 34 ++ .../junitlauncher/LegacyPlainResultFormatter.java | 290 ++ .../junitlauncher/LegacyXmlResultFormatter.java| 424 + 4 files changed, 1056 insertions(+) diff --git a/test/unit/org/apache/tools/ant/taskdefs/optional/junitlauncher/AbstractJUnitResultFormatter.java b/test/unit/org/apache/tools/ant/taskdefs/optional/junitlauncher/AbstractJUnitResultFormatter.java new file mode 100644 index 000..221aadb --- /dev/null +++ b/test/unit/org/apache/tools/ant/taskdefs/optional/junitlauncher/AbstractJUnitResultFormatter.java @@ -0,0 +1,308 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package org.apache.tools.ant.taskdefs.optional.junitlauncher; + +import org.apache.tools.ant.Project; +import org.apache.tools.ant.util.FileUtils; +import org.junit.platform.engine.TestSource; +import org.junit.platform.engine.support.descriptor.ClassSource; +import org.junit.platform.launcher.TestIdentifier; +import org.junit.platform.launcher.TestPlan; + +import java.io.BufferedReader; +import java.io.ByteArrayInputStream; +import java.io.Closeable; +import java.io.FileReader; +import java.io.IOException; +import java.io.InputStreamReader; +import java.io.OutputStream; +import java.io.Reader; +import java.io.Writer; +import java.nio.BufferOverflowException; +import java.nio.ByteBuffer; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Objects; +import java.util.Optional; + +/** + * Contains some common behaviour that's used by our internal {@link TestResultFormatter}s + */ +abstract class AbstractJUnitResultFormatter implements TestResultFormatter { + +protected TestExecutionContext context; + +private SysOutErrContentStore sysOutStore; +private SysOutErrContentStore sysErrStore; + +@Override +public void sysOutAvailable(final byte[] data) { +if (this.sysOutStore == null) { +this.sysOutStore = new SysOutErrContentStore(context, true); +} +try { +this.sysOutStore.store(data); +} catch (IOException e) { +handleException(e); +} +} + +@Override +public void sysErrAvailable(final byte[] data) { +if (this.sysErrStore == null) { +this.sysErrStore = new SysOutErrContentStore(context, false); +} +try { +this.sysErrStore.store(data); +} catch (IOException e) { +handleException(e); +} +} + +@Override +public void setContext(final TestExecutionContext context) { +this.context = context; +} + +/** + * @return Returns true if there's any stdout data, that was generated during the + * tests, is available for use. Else returns false. + */ +boolean hasSysOut() { +return this.sysOutStore != null && this.sysOutStore.hasData(); +} + +/** + * @return Returns true if there's any stderr data, that was generated during the + * tests, is available for use. Else returns false. + */ +boolean hasSysErr() { +return this.sysErrStore != null && this.sysErrStore.hasData(); +} + +/** + * @return Returns a {@link Reader} for reading any stdout data that was generated + * during the test execution. It is expected that the {@link #hasSysOut()} be first + * called to see if any such data is available and only if there is, then this method + * be called + * @throws IOException If there's any I/O problem while creating the {@link Reader} + */ +Reader getSysOutReader() throws IOException { +return this.sysOutStore.getReader(); +} + +/** + * @return Returns a {@link Reader} for reading any stderr data that was
[cassandra] branch cassandra-16630_junit5 created (now 8b04e68)
This is an automated email from the ASF dual-hosted git repository. azotcsit pushed a change to branch cassandra-16630_junit5 in repository https://gitbox.apache.org/repos/asf/cassandra.git. at 8b04e68 CASSANDRA-16630. Updated Cassandra test formatters. This branch includes the following new commits: new 667a894 CASSANDRA-16630. Updated dependencies. new 56a32b8 CASSANDRA-16630. Copied Ant JUnit classes. new 31d8eea CASSANDRA-16630. Updated Ant JUnit classes. new 8b04e68 CASSANDRA-16630. Updated Cassandra test formatters. The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 03/04: CASSANDRA-16630. Updated Ant JUnit classes.
This is an automated email from the ASF dual-hosted git repository. azotcsit pushed a commit to branch cassandra-16630_junit5 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 31d8eea6bb9a5cb3829ccfc3de632c989e900409 Author: Aleksei Zotov AuthorDate: Tue Nov 16 00:55:45 2021 +0400 CASSANDRA-16630. Updated Ant JUnit classes. --- .../unit/org/apache/cassandra/JStackJUnitTask.java | 25 +--- .../junitlauncher/LegacyBriefResultFormatter.java | 34 -- .../AbstractJUnitResultFormatter.java | 90 +++--- .../junitlauncher2/LegacyBriefResultFormatter.java | 36 ++ .../LegacyPlainResultFormatter.java| 52 .../LegacyXmlResultFormatter.java | 135 - 6 files changed, 185 insertions(+), 187 deletions(-) diff --git a/test/unit/org/apache/cassandra/JStackJUnitTask.java b/test/unit/org/apache/cassandra/JStackJUnitTask.java index 69480a6..f8f806c 100644 --- a/test/unit/org/apache/cassandra/JStackJUnitTask.java +++ b/test/unit/org/apache/cassandra/JStackJUnitTask.java @@ -22,28 +22,14 @@ import java.io.IOException; import java.io.InputStreamReader; import java.lang.reflect.Field; -import org.apache.tools.ant.BuildException; import org.apache.tools.ant.taskdefs.ExecuteWatchdog; -import org.apache.tools.ant.taskdefs.optional.junit.JUnitTask; +import org.apache.tools.ant.taskdefs.optional.junitlauncher.confined.JUnitLauncherTask; import org.apache.tools.ant.util.Watchdog; -public class JStackJUnitTask extends JUnitTask +public class JStackJUnitTask extends JUnitLauncherTask { -private Integer timeout; - -public JStackJUnitTask() throws Exception -{ -} - -@Override -public void setTimeout(Integer timeout) -{ -this.timeout = timeout; -super.setTimeout(timeout); -} - @Override -public ExecuteWatchdog createWatchdog() throws BuildException +protected ExecuteWatchdog createExecuteWatchdog(long timeout) { return new JStackWatchDog(timeout); } @@ -57,11 +43,6 @@ public class JStackJUnitTask extends JUnitTask super(timeout); } -public JStackWatchDog(int timeout) -{ -super(timeout); -} - @Override public synchronized void start(Process process) { diff --git a/test/unit/org/apache/tools/ant/taskdefs/optional/junitlauncher/LegacyBriefResultFormatter.java b/test/unit/org/apache/tools/ant/taskdefs/optional/junitlauncher/LegacyBriefResultFormatter.java deleted file mode 100644 index 7debbf0..000 --- a/test/unit/org/apache/tools/ant/taskdefs/optional/junitlauncher/LegacyBriefResultFormatter.java +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - */ -package org.apache.tools.ant.taskdefs.optional.junitlauncher; - -import org.junit.platform.engine.TestExecutionResult; -import org.junit.platform.launcher.TestIdentifier; - -/** - * A {@link TestResultFormatter} which prints a brief statistic for tests that have - * failed, aborted or skipped - */ -class LegacyBriefResultFormatter extends LegacyPlainResultFormatter implements TestResultFormatter { - -@Override -protected boolean shouldReportExecutionFinished(final TestIdentifier testIdentifier, final TestExecutionResult testExecutionResult) { -final TestExecutionResult.Status resultStatus = testExecutionResult.getStatus(); -return resultStatus == TestExecutionResult.Status.ABORTED || resultStatus == TestExecutionResult.Status.FAILED; -} -} diff --git a/test/unit/org/apache/tools/ant/taskdefs/optional/junitlauncher/AbstractJUnitResultFormatter.java b/test/unit/org/apache/tools/ant/taskdefs/optional/junitlauncher2/AbstractJUnitResultFormatter.java similarity index 78% rename from test/unit/org/apache/tools/ant/taskdefs/optional/junitlauncher/AbstractJUnitResultFormatter.java rename to test/unit/org/apache/tools/ant/taskdefs/optional/junitlauncher2/AbstractJUnitResultFormatter.java index 221aadb..456a2cd 100644 --- a/test/unit/org/apache/tools/ant/taskdefs/optional/junitlauncher/AbstractJUnitResultFormatter.java +++
[cassandra] 01/04: CASSANDRA-16630. Updated dependencies.
This is an automated email from the ASF dual-hosted git repository. azotcsit pushed a commit to branch cassandra-16630_junit5 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 667a8946a01b128b61b06e96f5f09f5c11973c2a Author: Aleksei Zotov AuthorDate: Sat Nov 13 23:47:47 2021 +0400 CASSANDRA-16630. Updated dependencies. --- build.xml | 27 +++ .../org/apache/cassandra/Junit4SampleTest.java | 39 + .../org/apache/cassandra/Junit5SampleTest.java | 40 ++ 3 files changed, 91 insertions(+), 15 deletions(-) diff --git a/build.xml b/build.xml index 085211a..6b6d71b 100644 --- a/build.xml +++ b/build.xml @@ -120,6 +120,7 @@ + @@ -524,10 +525,6 @@ - - - - @@ -565,8 +562,6 @@ - - @@ -650,11 +645,15 @@ - - - + + + + + + + + @@ -716,7 +715,6 @@ artifactId="cassandra-parent" version="${version}" relativePath="${final.name}-parent.pom"/> - @@ -727,10 +725,9 @@ - - - + + + diff --git a/test/unit/org/apache/cassandra/Junit4SampleTest.java b/test/unit/org/apache/cassandra/Junit4SampleTest.java new file mode 100644 index 000..7eb27a5 --- /dev/null +++ b/test/unit/org/apache/cassandra/Junit4SampleTest.java @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.cassandra; + +import org.junit.Test; + +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +public class Junit4SampleTest +{ +@Test +public void testSuccess() +{ +assertTrue(true); +} + +@Test +public void testFailure() +{ +fail("Expected failure"); +} +} diff --git a/test/unit/org/apache/cassandra/Junit5SampleTest.java b/test/unit/org/apache/cassandra/Junit5SampleTest.java new file mode 100644 index 000..0388d3c --- /dev/null +++ b/test/unit/org/apache/cassandra/Junit5SampleTest.java @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.cassandra; + + +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +public class Junit5SampleTest +{ +@Test +public void testSuccess() +{ +assertTrue(true); +} + +@Test +public void testFailure() +{ +fail("Expected failure"); +} +} - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 02/03: scripts: add scripts/cassandra-harry
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git commit 02296963c97b43e7247ef9d1228ed244cfe3f778 Author: Amos Kong AuthorDate: Tue Jan 19 22:24:20 2021 +0800 scripts: add scripts/cassandra-harry We want to use cassandra-harry in automation test, and it's not convenient to set eveything in yaml config. This patch added a script to accept the cmdline options and editing the yaml config. Signed-off-by: Amos Kong --- scripts/cassandra-harry | 9 + scylla-usage.md | 15 +++ 2 files changed, 24 insertions(+) diff --git a/scripts/cassandra-harry b/scripts/cassandra-harry new file mode 100755 index 000..dd27f7b --- /dev/null +++ b/scripts/cassandra-harry @@ -0,0 +1,9 @@ +#!/bin/sh + +if [ "x$HARRY_HOME" == 'x' ]; then +HARRY_HOME=~/cassandra-harry +fi + +java -Dlogback.configurationFile=$HARRY_HOME/test/conf/logback-dtest.xml \ + -jar $HARRY_HOME/harry-integration-external/target/harry-integration-external-0.0.1-SNAPSHOT.jar \ + $HARRY_HOME/conf/external.yaml diff --git a/scylla-usage.md b/scylla-usage.md new file mode 100644 index 000..b70f261 --- /dev/null +++ b/scylla-usage.md @@ -0,0 +1,15 @@ + +### Install + +``` +git clone -b scylla-branch https://github.com/amoskong/cassandra-harry +cd cassandra-harry +make mvn +sudo ln -s `realpath scripts/cassandra-harry` /usr/bin/cassandra-harry +``` + +### Execute + +``` +HARRY_HOME=~/cassandra-harry cassandra-harry +``` - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 01/03: Makefile: support to build without creating docker image
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git commit b566e8fb8aaff5a1420f55dd44ac366b4465ace7 Author: Amos Kong AuthorDate: Tue Jan 19 22:03:46 2021 +0800 Makefile: support to build without creating docker image When external cluster is used, docker isn't needed. Signed-off-by: Amos Kong --- Makefile | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index c1e99dd..86d4b28 100644 --- a/Makefile +++ b/Makefile @@ -16,9 +16,12 @@ DOCKER_REPO = org.apache.cassandra/harry/harry-runner -img: +mvn: rm -fr shared/* - mvn clean && mvn package -DskipTests && docker build -t ${DOCKER_REPO}:latest-local ./ -f docker/Dockerfile.local + mvn clean && mvn package -DskipTests + +img: mvn + docker build -t ${DOCKER_REPO}:latest-local ./ -f docker/Dockerfile.local run: img docker run -v `pwd`/shared:/shared -it ${DOCKER_REPO}:latest-local - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] branch trunk updated (f6b4df6 -> 26bb669)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git. from f6b4df6 Add history builder and an ability to write unit-tests with Harry new b566e8f Makefile: support to build without creating docker image new 0229696 scripts: add scripts/cassandra-harry new 26bb669 script: add options to cassandra-harry script The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: Makefile| 7 -- scripts/cassandra-harry | 57 + scylla-usage.md | 15 + 3 files changed, 77 insertions(+), 2 deletions(-) create mode 100755 scripts/cassandra-harry create mode 100644 scylla-usage.md - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 03/03: script: add options to cassandra-harry script
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git commit 26bb6696ba1b18ff5c062d780a9360e750691052 Author: Amos Kong AuthorDate: Tue Jun 1 21:50:48 2021 +0800 script: add options to cassandra-harry script This patch added options to wrap script, it supported to assign test duraton from cmdline. Signed-off-by: Amos Kong --- scripts/cassandra-harry | 48 scylla-usage.md | 2 +- 2 files changed, 49 insertions(+), 1 deletion(-) diff --git a/scripts/cassandra-harry b/scripts/cassandra-harry index dd27f7b..94ee991 100755 --- a/scripts/cassandra-harry +++ b/scripts/cassandra-harry @@ -4,6 +4,54 @@ if [ "x$HARRY_HOME" == 'x' ]; then HARRY_HOME=~/cassandra-harry fi +print_usage() { +cat <
[jira] [Commented] (CASSANDRA-17072) DebuggableThreadPoolExecutor does not propagate client warnings
[ https://issues.apache.org/jira/browse/CASSANDRA-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444078#comment-17444078 ] David Capwell commented on CASSANDRA-17072: --- 4.0: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=pr%2F1314 trunk: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=pr%2F1320 > DebuggableThreadPoolExecutor does not propagate client warnings > --- > > Key: CASSANDRA-17072 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17072 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.0.x > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Extracted this as a separate ticket per discussion on CASSANDRA-17044 > The problem is in {{DebuggableThreadPoolExecutor}} - it does not propagate > executor locals to the thread when tracing is disabled. It is probably > expected that we propagate the state if at least one executor local is > defined, but we only check for tracing and completely ignore client warnings. > The attached PR fixes the problem, adds some tests and reverts a workaround > for client warnings in some schema alteration statements implemented in > CASSANDRA-16296 (described below). > > h4. Old description - still valid, but this is just a manifestation of the > problem rather than the problem itself > This seemed to be screwed a bit. In just two schema alteration statements we > collect client warnings which are captured during the transformation into a > local collection. > I guess it is done that way because the transformation is being executed in a > different stage (migration) and client warnings collected in that stage are > not present in the stage where the query is executed. > Then, the client warnings are retrieved using {{clientWarnings}} method and > added to the captured client warnings in the stage which is executing the > query. > This mechanism was implemented only in two schema alteration statements. It > is possible that for other ones the client warnings can simply get lost. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17145) utests_system_keyspace_directory not run in Jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-17145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444072#comment-17444072 ] Ekaterina Dimitrova commented on CASSANDRA-17145: - TIL, Thanks [~brandon.williams] :) > utests_system_keyspace_directory not run in Jenkins > --- > > Key: CASSANDRA-17145 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17145 > Project: Cassandra > Issue Type: Improvement > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > > It seems to me utests_system_keyspace_directory which we run with 4.0 and > trunk in CircleCI do not run in Jenkins as they all fail in trunk now in > CircleCI and nothing in Jenkins. The issue is not environmental and I don't > see that group of tests under Stage View in Jenkins so they are probably not > added there? We need to investigate that further. > CC [~mck] and [~blerer] if they know anything about those - like work in > progress to be added or a reason not to be added. > FYI [~bereng] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17145) utests_system_keyspace_directory not run in Jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-17145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444067#comment-17444067 ] Brandon Williams commented on CASSANDRA-17145: -- I've changed it. You have to 'move' the ticket. > utests_system_keyspace_directory not run in Jenkins > --- > > Key: CASSANDRA-17145 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17145 > Project: Cassandra > Issue Type: Improvement > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > > It seems to me utests_system_keyspace_directory which we run with 4.0 and > trunk in CircleCI do not run in Jenkins as they all fail in trunk now in > CircleCI and nothing in Jenkins. The issue is not environmental and I don't > see that group of tests under Stage View in Jenkins so they are probably not > added there? We need to investigate that further. > CC [~mck] and [~blerer] if they know anything about those - like work in > progress to be added or a reason not to be added. > FYI [~bereng] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17145) utests_system_keyspace_directory not run in Jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-17145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-17145: - Workflow: Copy of Cassandra Default Workflow (was: Copy of Cassandra Bug Workflow) Issue Type: Improvement (was: Bug) > utests_system_keyspace_directory not run in Jenkins > --- > > Key: CASSANDRA-17145 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17145 > Project: Cassandra > Issue Type: Improvement > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > > It seems to me utests_system_keyspace_directory which we run with 4.0 and > trunk in CircleCI do not run in Jenkins as they all fail in trunk now in > CircleCI and nothing in Jenkins. The issue is not environmental and I don't > see that group of tests under Stage View in Jenkins so they are probably not > added there? We need to investigate that further. > CC [~mck] and [~blerer] if they know anything about those - like work in > progress to be added or a reason not to be added. > FYI [~bereng] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17147) Guardrails prototype
[ https://issues.apache.org/jira/browse/CASSANDRA-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-17147: -- Status: Review In Progress (was: Patch Available) started first batch of review, left comments in PR but had a few high level ones * Why try to offer a pluggable implementation for the leaf types? What value do we get allowing a pluggable way to override how we check a volatile field? * what are the plans for listeners? They are mostly dead code * why 2 different configuration systems? I 100% get why you would want a new MBean, but don't get why we need 2 different config systems: GuardrailOptions vs GuardrailConfig * ClientState being part of the base interface could become a problem when you integrate with track_warnings, as they plugin to places shared by query and background services (compaction/repair); not a blocker for this patch as you have scoped moving track_warnings to guardrails, but just something to think about * some checks offer warn/fail, and others make this 2 different checks: disallowTableProperties, etc. Should these not be simplified to a single check? > Guardrails prototype > > > Key: CASSANDRA-17147 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17147 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 4.1 > > Time Spent: 10m > Remaining Estimate: 0h > > The purpose of this ticket is creating an initial implementation of the > guardrails framework, as well as adding a few simple guardrails using this > framework. > To keep things easy, this initial implementation would only support > guardrails that are triggered on the coordinator, and they would be > dynamically updatable only through JMX. > Once we have this initial framework ready in a feature branch we can have > multiple tickets addressing all the things that would have been left out of > the scope of this ticket, such as: > * Dynamic updates through virtual tables > * Being able to notify about guardrails triggered on replicas > * Using custom exceptions other than {{InvalidRequestException}}. > * Porting existing limits to use the new guardrails framework > * Adding new guardrails beyond the initial ones > The reason for having this simpler prototype is that it will give us a common > ground to parallelize work on the parts mentioned above. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17145) utests_system_keyspace_directory not run in Jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-17145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444058#comment-17444058 ] Ekaterina Dimitrova commented on CASSANDRA-17145: - Thanks for confirming. Unfortunately, I cannot change this to type improvement now, at least I didn't find a way. > utests_system_keyspace_directory not run in Jenkins > --- > > Key: CASSANDRA-17145 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17145 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > > It seems to me utests_system_keyspace_directory which we run with 4.0 and > trunk in CircleCI do not run in Jenkins as they all fail in trunk now in > CircleCI and nothing in Jenkins. The issue is not environmental and I don't > see that group of tests under Stage View in Jenkins so they are probably not > added there? We need to investigate that further. > CC [~mck] and [~blerer] if they know anything about those - like work in > progress to be added or a reason not to be added. > FYI [~bereng] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17145) utests_system_keyspace_directory not run in Jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-17145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444041#comment-17444041 ] Michael Semb Wever commented on CASSANDRA-17145: Never been implemented. This is a valid improvement (not a bug). > utests_system_keyspace_directory not run in Jenkins > --- > > Key: CASSANDRA-17145 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17145 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > > It seems to me utests_system_keyspace_directory which we run with 4.0 and > trunk in CircleCI do not run in Jenkins as they all fail in trunk now in > CircleCI and nothing in Jenkins. The issue is not environmental and I don't > see that group of tests under Stage View in Jenkins so they are probably not > added there? We need to investigate that further. > CC [~mck] and [~blerer] if they know anything about those - like work in > progress to be added or a reason not to be added. > FYI [~bereng] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17147) Guardrails prototype
[ https://issues.apache.org/jira/browse/CASSANDRA-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-17147: Reviewers: Caleb Rackliffe, David Capwell > Guardrails prototype > > > Key: CASSANDRA-17147 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17147 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 4.1 > > > The purpose of this ticket is creating an initial implementation of the > guardrails framework, as well as adding a few simple guardrails using this > framework. > To keep things easy, this initial implementation would only support > guardrails that are triggered on the coordinator, and they would be > dynamically updatable only through JMX. > Once we have this initial framework ready in a feature branch we can have > multiple tickets addressing all the things that would have been left out of > the scope of this ticket, such as: > * Dynamic updates through virtual tables > * Being able to notify about guardrails triggered on replicas > * Using custom exceptions other than {{InvalidRequestException}}. > * Porting existing limits to use the new guardrails framework > * Adding new guardrails beyond the initial ones > The reason for having this simpler prototype is that it will give us a common > ground to parallelize work on the parts mentioned above. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17151) Guardrail for column size
[ https://issues.apache.org/jira/browse/CASSANDRA-17151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-17151: -- Summary: Guardrail for column size (was: Guardrails for column size) > Guardrail for column size > - > > Key: CASSANDRA-17151 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17151 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Priority: Normal > > Add a guardrail for limiting the max size of column values, for example: > {code} > # Failure threshold to prevent writing large column values. > # Defaults to -1 to disable. > column_value_size_failure_threshold_in_kb: -1 > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17151) Guardrails for column size
[ https://issues.apache.org/jira/browse/CASSANDRA-17151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-17151: -- Description: Add a guardrail for limiting the max size of column values, for example: {code} # Failure threshold to prevent writing large column values. # Defaults to -1 to disable. column_value_size_failure_threshold_in_kb: -1 {code} was: Add a guardrail for limiting the max size of column values, for example: {code} # Failure threshold to prevent writing large column values. # Defaults to -1 to disable. # column_value_size_failure_threshold_in_kb: -1 {code} > Guardrails for column size > -- > > Key: CASSANDRA-17151 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17151 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Priority: Normal > > Add a guardrail for limiting the max size of column values, for example: > {code} > # Failure threshold to prevent writing large column values. > # Defaults to -1 to disable. > column_value_size_failure_threshold_in_kb: -1 > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17154) Guardrail to disable read-before-write list operations
Andres de la Peña created CASSANDRA-17154: - Summary: Guardrail to disable read-before-write list operations Key: CASSANDRA-17154 URL: https://issues.apache.org/jira/browse/CASSANDRA-17154 Project: Cassandra Issue Type: New Feature Components: Feature/Guardrails Reporter: Andres de la Peña Add guardrail to disable list operations implying read-before-write. For example: {code} # Whether read-before-write operation is allowed, eg. setting list element by index, removing list element. # Defaults to true to allow read before write operations read_before_write_list_operations_enabled: true {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17153) Guardrails for collection/UDT items and size
[ https://issues.apache.org/jira/browse/CASSANDRA-17153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-17153: -- Summary: Guardrails for collection/UDT items and size (was: Guardrail for collection/UDT items and size) > Guardrails for collection/UDT items and size > > > Key: CASSANDRA-17153 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17153 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Priority: Normal > > Add guardrails for the number of items in a collection/UDT and possibly also > the size of a collection. For example: > {code} > # Failure threshold to prevent creating more fields in user-defined-type than > threshold. > # Defaults -1 to disable. > # fields_per_udt_failure_threshold: -1 > # Warning threshold to warn when encountering larger size of collection data > than threshold. > # Defaults -1 to disable. > # collection_size_warn_threshold_in_kb: -1 > # Warning threshold to warn when encountering more elements in collection > than threshold. > # Defaults -1 to disable. > # items_per_collection_warn_threshold: -1 > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17153) Guardrail for collection/UDT items and size
Andres de la Peña created CASSANDRA-17153: - Summary: Guardrail for collection/UDT items and size Key: CASSANDRA-17153 URL: https://issues.apache.org/jira/browse/CASSANDRA-17153 Project: Cassandra Issue Type: New Feature Components: Feature/Guardrails Reporter: Andres de la Peña Add guardrails for the number of items in a collection/UDT and possibly also the size of a collection. For example: {code} # Failure threshold to prevent creating more fields in user-defined-type than threshold. # Defaults -1 to disable. # fields_per_udt_failure_threshold: -1 # Warning threshold to warn when encountering larger size of collection data than threshold. # Defaults -1 to disable. # collection_size_warn_threshold_in_kb: -1 # Warning threshold to warn when encountering more elements in collection than threshold. # Defaults -1 to disable. # items_per_collection_warn_threshold: -1 {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17152) Guardrail for partition size
Andres de la Peña created CASSANDRA-17152: - Summary: Guardrail for partition size Key: CASSANDRA-17152 URL: https://issues.apache.org/jira/browse/CASSANDRA-17152 Project: Cassandra Issue Type: New Feature Components: Feature/Guardrails Reporter: Andres de la Peña Add a guardrail for large partitions. Finding the size of a partition is tricky because partitions can be spread over multiple sstable and we want to avoid read before write. Thus, this might be just a warning thrown during compaction and not be linked to specific user queries, for example: {code} # Log a warning when compacting partitions larger than this value. # Defaults to -1 to disable # partition_size_warn_threshold_in_mb: -1 {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17151) Guardrails for column size
Andres de la Peña created CASSANDRA-17151: - Summary: Guardrails for column size Key: CASSANDRA-17151 URL: https://issues.apache.org/jira/browse/CASSANDRA-17151 Project: Cassandra Issue Type: New Feature Components: Feature/Guardrails Reporter: Andres de la Peña Add a guardrail for limiting the max size of column values, for example: {code} # Failure threshold to prevent writing large column values. # Defaults to -1 to disable. # column_value_size_failure_threshold_in_kb: -1 {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17150) Guardrails for disk usage
[ https://issues.apache.org/jira/browse/CASSANDRA-17150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-17150: -- Description: Add guardrails for disk usage establishing soft/hard limits on the percentage of used disk space. For example: {code} # Warning threshold to warn when local disk usage exceeds threshold. Valid values: (1, 100] # Defaults to -1 to disable. # disk_usage_percentage_warn_threshold: -1 # Failure threshold to reject write requests if replica disk usage exceeds threshold. Valid values: (1, 100] # Defaults to -1 to disable. # disk_usage_percentage_failure_threshold: -1 {code} was: Add guardrails for disk usage establishing soft/hard limits on the percentage of used disk space. For example: {code} # Warning threshold to warn when local disk usage exceeds threshold. Valid values: (1, 100] # Defaults to -1 to disable. # disk_usage_percentage_warn_threshold: -1 # Failure threshold to reject write requests if replica disk usage exceeds threshold. Valid values: (1, 100] # Defaults to -1 to disable. # disk_usage_percentage_failure_threshold: -1 {code} > Guardrails for disk usage > - > > Key: CASSANDRA-17150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17150 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Priority: Normal > > Add guardrails for disk usage establishing soft/hard limits on the percentage > of used disk space. For example: > {code} > # Warning threshold to warn when local disk usage exceeds threshold. Valid > values: (1, 100] > # Defaults to -1 to disable. > # disk_usage_percentage_warn_threshold: -1 > # Failure threshold to reject write requests if replica disk usage exceeds > threshold. Valid values: (1, 100] > # Defaults to -1 to disable. > # disk_usage_percentage_failure_threshold: -1 > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17150) Guardrails for disk usage
Andres de la Peña created CASSANDRA-17150: - Summary: Guardrails for disk usage Key: CASSANDRA-17150 URL: https://issues.apache.org/jira/browse/CASSANDRA-17150 Project: Cassandra Issue Type: New Feature Components: Feature/Guardrails Reporter: Andres de la Peña Add guardrails for disk usage establishing soft/hard limits on the percentage of used disk space. For example: {code} # Warning threshold to warn when local disk usage exceeds threshold. Valid values: (1, 100] # Defaults to -1 to disable. # disk_usage_percentage_warn_threshold: -1 # Failure threshold to reject write requests if replica disk usage exceeds threshold. Valid values: (1, 100] # Defaults to -1 to disable. # disk_usage_percentage_failure_threshold: -1 {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17149) Make Guardrails configurable through virtual tables
Andres de la Peña created CASSANDRA-17149: - Summary: Make Guardrails configurable through virtual tables Key: CASSANDRA-17149 URL: https://issues.apache.org/jira/browse/CASSANDRA-17149 Project: Cassandra Issue Type: New Feature Components: Feature/Guardrails Reporter: Andres de la Peña Guardrails should support configuration through virtual tables. This probably will require some previous work to make virtual tables writable. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17148) Adapt track_warnings to Guardrails
Andres de la Peña created CASSANDRA-17148: - Summary: Adapt track_warnings to Guardrails Key: CASSANDRA-17148 URL: https://issues.apache.org/jira/browse/CASSANDRA-17148 Project: Cassandra Issue Type: New Feature Components: Feature/Guardrails Reporter: Andres de la Peña The purpose of this ticket is adapting the warnings added by CASSANDRA-16850 to use the new Guardrails framework. This will involve extending the framework to be able to propagate warnings and failures triggered on replicas, taking advantage of the machinery added in that ticket. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17147) Guardrails prototype
[ https://issues.apache.org/jira/browse/CASSANDRA-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-17147: -- Test and Documentation Plan: The PR includes tests for each new guardrail and for the framework itself. Status: Patch Available (was: In Progress) > Guardrails prototype > > > Key: CASSANDRA-17147 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17147 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > > The purpose of this ticket is creating an initial implementation of the > guardrails framework, as well as adding a few simple guardrails using this > framework. > To keep things easy, this initial implementation would only support > guardrails that are triggered on the coordinator, and they would be > dynamically updatable only through JMX. > Once we have this initial framework ready in a feature branch we can have > multiple tickets addressing all the things that would have been left out of > the scope of this ticket, such as: > * Dynamic updates through virtual tables > * Being able to notify about guardrails triggered on replicas > * Using custom exceptions other than {{InvalidRequestException}}. > * Porting existing limits to use the new guardrails framework > * Adding new guardrails beyond the initial ones > The reason for having this simpler prototype is that it will give us a common > ground to parallelize work on the parts mentioned above. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17147) Guardrails prototype
[ https://issues.apache.org/jira/browse/CASSANDRA-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-17147: -- Fix Version/s: 4.1 > Guardrails prototype > > > Key: CASSANDRA-17147 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17147 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 4.1 > > > The purpose of this ticket is creating an initial implementation of the > guardrails framework, as well as adding a few simple guardrails using this > framework. > To keep things easy, this initial implementation would only support > guardrails that are triggered on the coordinator, and they would be > dynamically updatable only through JMX. > Once we have this initial framework ready in a feature branch we can have > multiple tickets addressing all the things that would have been left out of > the scope of this ticket, such as: > * Dynamic updates through virtual tables > * Being able to notify about guardrails triggered on replicas > * Using custom exceptions other than {{InvalidRequestException}}. > * Porting existing limits to use the new guardrails framework > * Adding new guardrails beyond the initial ones > The reason for having this simpler prototype is that it will give us a common > ground to parallelize work on the parts mentioned above. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17147) Guardrails prototype
[ https://issues.apache.org/jira/browse/CASSANDRA-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443995#comment-17443995 ] Andres de la Peña commented on CASSANDRA-17147: --- Here is the initial implementation of guardrails: ||PR||CI|| |[trunk|https://github.com/apache/cassandra/pull/1321]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1145/workflows/9163675f-0392-447b-baab-e2456a4fef40] [j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1145/workflows/b936cd54-1220-4b9b-8414-81f3be5ac59b]| The proposed PR is based on a series of patches by [~slebresne], [~jasonstack], [~Ge], [~jakubzytka] and myself. Some characteristics of this implementation are: * Guardrails configuration is under [a new section|https://github.com/apache/cassandra/blob/c900258b2de05315706e3733fe7cff254fcc33fb/conf/cassandra.yaml#L1614-L1643] in {{cassandra.yaml}}. * Each guardrail and the framework itself is disabled by default. * Guardrails can be dynamically modified through JMX using [this mbean|https://github.com/apache/cassandra/blob/c900258b2de05315706e3733fe7cff254fcc33fb/src/java/org/apache/cassandra/db/guardrails/GuardrailsConfigMBean.java]. Future work will make them updatable also through virtual tables, as soon we have writable VTs. * Guardrails are only applied to the queries of regular users, superusers and internal queries are excluded from guardrails. * Guardrails associated to client requests can only be triggered on the coordinator. We will support replica-side guardrails associated to client requests in a followup ticket. * There are initial threshold guardrails for the number of tables, indexes and materialized views. * There is an enable/disable guardrail for user-provided timestamps. * There are guardrails for disallowing and/or warning on a set of table properties. > Guardrails prototype > > > Key: CASSANDRA-17147 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17147 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > > The purpose of this ticket is creating an initial implementation of the > guardrails framework, as well as adding a few simple guardrails using this > framework. > To keep things easy, this initial implementation would only support > guardrails that are triggered on the coordinator, and they would be > dynamically updatable only through JMX. > Once we have this initial framework ready in a feature branch we can have > multiple tickets addressing all the things that would have been left out of > the scope of this ticket, such as: > * Dynamic updates through virtual tables > * Being able to notify about guardrails triggered on replicas > * Using custom exceptions other than {{InvalidRequestException}}. > * Porting existing limits to use the new guardrails framework > * Adding new guardrails beyond the initial ones > The reason for having this simpler prototype is that it will give us a common > ground to parallelize work on the parts mentioned above. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17143) Fix test NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling
[ https://issues.apache.org/jira/browse/CASSANDRA-17143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-17143: -- Since Version: NA Source Control Link: https://github.com/apache/cassandra/commit/6b7166a7a12fde3c3f786608ee9a731c41a87acb Resolution: Fixed Status: Resolved (was: Ready to Commit) > Fix test > NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling > --- > > Key: CASSANDRA-17143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17143 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: NA > > > https://app.circleci.com/pipelines/github/dcapwell/cassandra/1099/workflows/2f08ff99-9ea5-4023-b027-84150efbd2e9/jobs/7850 > Found in CASSANDRA-17069. > What I see is that there is a race condition, which is easier to hit when we > disable throttling. If zero-copy-streaming starts and completes within the > gap between nodetool calls, then the test fails with a timeout as the > expected condition is not seen (waits to see streaming). We can avoid this > by detecting that streaming completed by grepping the logs. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Fix test NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new ad4d2b3 Fix test NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling ad4d2b3 is described below commit ad4d2b3a266535734ef1bdebb4a3545fb3306769 Author: David Capwell AuthorDate: Mon Nov 15 08:51:43 2021 -0800 Fix test NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling patch by David Capwell; reviewed by David Capwell, Stefan Miklosovic for CASSANDRA-17143 --- .../distributed/test/AbstractNetstatsStreaming.java | 16 1 file changed, 16 insertions(+) diff --git a/test/distributed/org/apache/cassandra/distributed/test/AbstractNetstatsStreaming.java b/test/distributed/org/apache/cassandra/distributed/test/AbstractNetstatsStreaming.java index 2e828e0..93d26f1 100644 --- a/test/distributed/org/apache/cassandra/distributed/test/AbstractNetstatsStreaming.java +++ b/test/distributed/org/apache/cassandra/distributed/test/AbstractNetstatsStreaming.java @@ -40,6 +40,7 @@ import org.slf4j.LoggerFactory; import com.datastax.driver.core.Session; import org.apache.cassandra.distributed.Cluster; import org.apache.cassandra.distributed.api.IInvokableInstance; +import org.apache.cassandra.distributed.api.LogResult; import org.apache.cassandra.distributed.api.NodeToolResult; import org.apache.cassandra.utils.Pair; @@ -509,6 +510,7 @@ public abstract class AbstractNetstatsStreaming extends TestBaseImpl boolean sawAnyStreamingOutput = false; +long mark = 0; while (true) { try @@ -523,6 +525,20 @@ public abstract class AbstractNetstatsStreaming extends TestBaseImpl { sawAnyStreamingOutput = true; } +else +{ +// there is a race condition that streaming starts/stops between calls to netstats +// to detect this, check to see if the node has completed a stream +// expected log: [Stream (.*)?] All sessions completed +LogResult> logs = node.logs().grep(mark, "\\[Stream .*\\] All sessions completed"); +mark = logs.getMark(); +if (!logs.getResult().isEmpty()) +{ +// race condition detected... +logger.info("Test race condition detected where streaming started/stopped between calls to netstats"); +sawAnyStreamingOutput = true; +} +} } if (sawAnyStreamingOutput && (!result.getStdout().contains("Receiving") && !result.getStdout().contains("Sending"))) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17143) Fix test NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling
[ https://issues.apache.org/jira/browse/CASSANDRA-17143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443946#comment-17443946 ] David Capwell commented on CASSANDRA-17143: --- I didn't run tests in Jenkins as I think it is still in bad shape; since this is just a jvm-dtest failure based off a race condition, I felt Jenkins didn't matter. > Fix test > NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling > --- > > Key: CASSANDRA-17143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17143 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: NA > > > https://app.circleci.com/pipelines/github/dcapwell/cassandra/1099/workflows/2f08ff99-9ea5-4023-b027-84150efbd2e9/jobs/7850 > Found in CASSANDRA-17069. > What I see is that there is a race condition, which is easier to hit when we > disable throttling. If zero-copy-streaming starts and completes within the > gap between nodetool calls, then the test fails with a timeout as the > expected condition is not seen (waits to see streaming). We can avoid this > by detecting that streaming completed by grepping the logs. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17143) Fix test NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling
[ https://issues.apache.org/jira/browse/CASSANDRA-17143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443945#comment-17443945 ] David Capwell commented on CASSANDRA-17143: --- Starting commit CI Results (pending): ||Branch||Source||Circle CI||Jenkins|| |trunk|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-17143-trunk-D95E13F6-1BF2-489C-A543-64A960E26D9D]|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-17143-trunk-D95E13F6-1BF2-489C-A543-64A960E26D9D]|[build|unknown]| > Fix test > NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling > --- > > Key: CASSANDRA-17143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17143 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: NA > > > https://app.circleci.com/pipelines/github/dcapwell/cassandra/1099/workflows/2f08ff99-9ea5-4023-b027-84150efbd2e9/jobs/7850 > Found in CASSANDRA-17069. > What I see is that there is a race condition, which is easier to hit when we > disable throttling. If zero-copy-streaming starts and completes within the > gap between nodetool calls, then the test fails with a timeout as the > expected condition is not seen (waits to see streaming). We can avoid this > by detecting that streaming completed by grepping the logs. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17143) Fix test NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling
[ https://issues.apache.org/jira/browse/CASSANDRA-17143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-17143: -- Status: Ready to Commit (was: Review In Progress) > Fix test > NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling > --- > > Key: CASSANDRA-17143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17143 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: NA > > > https://app.circleci.com/pipelines/github/dcapwell/cassandra/1099/workflows/2f08ff99-9ea5-4023-b027-84150efbd2e9/jobs/7850 > Found in CASSANDRA-17069. > What I see is that there is a race condition, which is easier to hit when we > disable throttling. If zero-copy-streaming starts and completes within the > gap between nodetool calls, then the test fails with a timeout as the > expected condition is not seen (waits to see streaming). We can avoid this > by detecting that streaming completed by grepping the logs. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17143) Fix test NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling
[ https://issues.apache.org/jira/browse/CASSANDRA-17143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-17143: -- Reviewers: Stefan Miklosovic, David Capwell Stefan Miklosovic, David Capwell (was: David Capwell, Stefan Miklosovic) Status: Review In Progress (was: Patch Available) > Fix test > NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest#testWithStreamingEntireSSTablesWithoutCompressionWithoutThrottling > --- > > Key: CASSANDRA-17143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17143 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: NA > > > https://app.circleci.com/pipelines/github/dcapwell/cassandra/1099/workflows/2f08ff99-9ea5-4023-b027-84150efbd2e9/jobs/7850 > Found in CASSANDRA-17069. > What I see is that there is a race condition, which is easier to hit when we > disable throttling. If zero-copy-streaming starts and completes within the > gap between nodetool calls, then the test fails with a timeout as the > expected condition is not seen (waits to see streaming). We can avoid this > by detecting that streaming completed by grepping the logs. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16912) Rename cassandra-topology.properties to cassandra-topology.properties.example
[ https://issues.apache.org/jira/browse/CASSANDRA-16912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443932#comment-17443932 ] Berenguer Blasi commented on CASSANDRA-16912: - Yeah I noticed all cassandra nodes are offline. We'll have to wait... > Rename cassandra-topology.properties to cassandra-topology.properties.example > - > > Key: CASSANDRA-16912 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16912 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Gossip >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1 > > > Far too often people leave this file in place when using GPFS, when it is > only intended to be using during migration away from PFS. If we simply > rename this file, then the GPFS camp won't trip over it (without going to > effort to do so at least) and the PFS camp can simple rename the file > before/after editing it. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17146) CEP-3: Guardrails
[ https://issues.apache.org/jira/browse/CASSANDRA-17146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-17146: - Component/s: Feature/Guardrails > CEP-3: Guardrails > - > > Key: CASSANDRA-17146 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17146 > Project: Cassandra > Issue Type: Epic > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > > Implementation of guardrails as described in > [CEP-3|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-3%3A+Guardrails]. > The goal of this feature is to have an easy way for operators to enforce > system-wide soft and hard limits that ensure good practices, foster > availability, and guard the system from wrong usage patterns. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17147) Guardrails prototype
[ https://issues.apache.org/jira/browse/CASSANDRA-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-17147: -- Change Category: Operability Complexity: Challenging Component/s: Feature/Guardrails Status: Open (was: Triage Needed) > Guardrails prototype > > > Key: CASSANDRA-17147 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17147 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > > The purpose of this ticket is creating an initial implementation of the > guardrails framework, as well as adding a few simple guardrails using this > framework. > To keep things easy, this initial implementation would only support > guardrails that are triggered on the coordinator, and they would be > dynamically updatable only through JMX. > Once we have this initial framework ready in a feature branch we can have > multiple tickets addressing all the things that would have been left out of > the scope of this ticket, such as: > * Dynamic updates through virtual tables > * Being able to notify about guardrails triggered on replicas > * Using custom exceptions other than {{InvalidRequestException}}. > * Porting existing limits to use the new guardrails framework > * Adding new guardrails beyond the initial ones > The reason for having this simpler prototype is that it will give us a common > ground to parallelize work on the parts mentioned above. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17065) Introduce separate rate limiting settings for entire SSTable streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-17065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443925#comment-17443925 ] Francisco Guerrero commented on CASSANDRA-17065: [~jmckenzie] That test only failed in Jenkins. In Circle CI the test is passing. I don't think it deserved a ticket at the moment. > Introduce separate rate limiting settings for entire SSTable streaming > -- > > Key: CASSANDRA-17065 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17065 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode, Tool/nodetool >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Fix For: 4.1 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > With the introduction of entire SSTable streaming, it is desirable to > introduce new settings for the rate limit for entire SSTable streaming > operations. > Currently, regular streaming and SSTable streaming are rate limited by the > same setting. However, zero-copy streaming reduces load in the JVM and we can > have a setting specifically for this use case. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17147) Guardrails prototype
Andres de la Peña created CASSANDRA-17147: - Summary: Guardrails prototype Key: CASSANDRA-17147 URL: https://issues.apache.org/jira/browse/CASSANDRA-17147 Project: Cassandra Issue Type: New Feature Reporter: Andres de la Peña Assignee: Andres de la Peña The purpose of this ticket is creating an initial implementation of the guardrails framework, as well as adding a few simple guardrails using this framework. To keep things easy, this initial implementation would only support guardrails that are triggered on the coordinator, and they would be dynamically updatable only through JMX. Once we have this initial framework ready in a feature branch we can have multiple tickets addressing all the things that would have been left out of the scope of this ticket, such as: * Dynamic updates through virtual tables * Being able to notify about guardrails triggered on replicas * Using custom exceptions other than {{InvalidRequestException}}. * Porting existing limits to use the new guardrails framework * Adding new guardrails beyond the initial ones The reason for having this simpler prototype is that it will give us a common ground to parallelize work on the parts mentioned above. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16912) Rename cassandra-topology.properties to cassandra-topology.properties.example
[ https://issues.apache.org/jira/browse/CASSANDRA-16912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443909#comment-17443909 ] Brandon Williams commented on CASSANDRA-16912: -- bq. iiuc the problem would only repro on jenkins Yeah, I have a jenkins run linked up there, but then jenkins blew up and it looks like yours will be stuck a while too. :( > Rename cassandra-topology.properties to cassandra-topology.properties.example > - > > Key: CASSANDRA-16912 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16912 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Gossip >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1 > > > Far too often people leave this file in place when using GPFS, when it is > only intended to be using during migration away from PFS. If we simply > rename this file, then the GPFS camp won't trip over it (without going to > effort to do so at least) and the PFS camp can simple rename the file > before/after editing it. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17146) CEP-3: Guardrails
Andres de la Peña created CASSANDRA-17146: - Summary: CEP-3: Guardrails Key: CASSANDRA-17146 URL: https://issues.apache.org/jira/browse/CASSANDRA-17146 Project: Cassandra Issue Type: Epic Reporter: Andres de la Peña Assignee: Andres de la Peña Implementation of guardrails as described in [CEP-3|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-3%3A+Guardrails]. The goal of this feature is to have an easy way for operators to enforce system-wide soft and hard limits that ensure good practices, foster availability, and guard the system from wrong usage patterns. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17145) utests_system_keyspace_directory not run in Jenkins
Ekaterina Dimitrova created CASSANDRA-17145: --- Summary: utests_system_keyspace_directory not run in Jenkins Key: CASSANDRA-17145 URL: https://issues.apache.org/jira/browse/CASSANDRA-17145 Project: Cassandra Issue Type: Bug Reporter: Ekaterina Dimitrova It seems to me utests_system_keyspace_directory which we run with 4.0 and trunk in CircleCI do not run in Jenkins as they all fail in trunk now in CircleCI and nothing in Jenkins. The issue is not environmental and I don't see that group of tests under Stage View in Jenkins so they are probably not added there? We need to investigate that further. CC [~mck] and [~blerer] if they know anything about those - like work in progress to be added or a reason not to be added. FYI [~bereng] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17145) utests_system_keyspace_directory not run in Jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-17145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17145: Bug Category: Parent values: Correctness(12982) Complexity: Normal Component/s: CI Discovered By: User Report Severity: Normal Status: Open (was: Triage Needed) > utests_system_keyspace_directory not run in Jenkins > --- > > Key: CASSANDRA-17145 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17145 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > > It seems to me utests_system_keyspace_directory which we run with 4.0 and > trunk in CircleCI do not run in Jenkins as they all fail in trunk now in > CircleCI and nothing in Jenkins. The issue is not environmental and I don't > see that group of tests under Stage View in Jenkins so they are probably not > added there? We need to investigate that further. > CC [~mck] and [~blerer] if they know anything about those - like work in > progress to be added or a reason not to be added. > FYI [~bereng] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17140) Broken test_rolling_upgrade - upgrade_tests.upgrade_through_versions_test.TestUpgrade_indev_3_0_x_To_indev_4_0_x
[ https://issues.apache.org/jira/browse/CASSANDRA-17140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh McKenzie updated CASSANDRA-17140: -- Bug Category: Parent values: Correctness(12982)Level 1 values: Test Failure(12990) > Broken test_rolling_upgrade - > upgrade_tests.upgrade_through_versions_test.TestUpgrade_indev_3_0_x_To_indev_4_0_x > > > Key: CASSANDRA-17140 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17140 > Project: Cassandra > Issue Type: Bug >Reporter: Yifan Cai >Priority: Normal > > The tests "test_rolling_upgrade" fail with the below error. > > [https://app.circleci.com/pipelines/github/yifan-c/cassandra/279/workflows/6340cd42-0b27-42c2-8418-9f8b56c57bea/jobs/1990] > > I am able to alway produce it by running the test locally too. > {{$ pytest --execute-upgrade-tests-only --upgrade-target-version-only > --upgrade-version-selection all --cassandra-version=4.0 > upgrade_tests/upgrade_through_versions_test.py::TestUpgrade_indev_3_11_x_To_indev_4_0_x::test_rolling_upgrade}} > > {code:java} > self = > object at 0x7ffba4242fd0> > subprocs = [, > ] > def _check_on_subprocs(self, subprocs): > """ > Check on given subprocesses. > > If any are not alive, we'll go ahead and terminate any remaining > alive subprocesses since this test is going to fail. > """ > subproc_statuses = [s.is_alive() for s in subprocs] > if not all(subproc_statuses): > message = "A subprocess has terminated early. Subprocess > statuses: " > for s in subprocs: > message += "{name} (is_alive: {aliveness}), > ".format(name=s.name, aliveness=s.is_alive()) > message += "attempting to terminate remaining subprocesses now." > self._terminate_subprocs() > > raise RuntimeError(message) > E RuntimeError: A subprocess has terminated early. Subprocess > statuses: Process-1 (is_alive: True), Process-2 (is_alive: False), attempting > to terminate remaining subprocesses now.{code} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17065) Introduce separate rate limiting settings for entire SSTable streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-17065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443883#comment-17443883 ] Josh McKenzie commented on CASSANDRA-17065: --- {quote}`failThresholdSinglePartition` fails in the build, but it is passing locally {quote} Is failing only on your branch in CI env or already failing on trunk? Do we already have a [ticket|https://issues.apache.org/jira/issues/?filter=12350869] for it? > Introduce separate rate limiting settings for entire SSTable streaming > -- > > Key: CASSANDRA-17065 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17065 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode, Tool/nodetool >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Fix For: 4.1 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > With the introduction of entire SSTable streaming, it is desirable to > introduce new settings for the rate limit for entire SSTable streaming > operations. > Currently, regular streaming and SSTable streaming are rate limited by the > same setting. However, zero-copy streaming reduces load in the JVM and we can > have a setting specifically for this use case. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17132) Fix startup issue with internode_application_timeout_in_ms
[ https://issues.apache.org/jira/browse/CASSANDRA-17132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443873#comment-17443873 ] Ekaterina Dimitrova commented on CASSANDRA-17132: - CASSANDRA-17141 linked. True, I will remove those two renames from the current patch and handle the properties in the other ticket. > Fix startup issue with internode_application_timeout_in_ms > -- > > Key: CASSANDRA-17132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17132 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.x > > > While testing my patch for CASSANDRA-17131 I found that there is a problem > with _internode_application_timeout_in_ms_ in 4.0 and trunk. > Seems to me that we can just safely remove it for now? -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17132) Fix startup issue with internode_application_timeout_in_ms
[ https://issues.apache.org/jira/browse/CASSANDRA-17132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443858#comment-17443858 ] Berenguer Blasi commented on CASSANDRA-17132: - You can have a link with the right compare base as in [here|https://github.com/apache/cassandra/compare/cassandra-4.0...ekaterinadimitrova2:17132-4.0?expand=1] #justyi Ok so iiuc that name change is seen as a bug, you've opened a ticket to address that (can you link it please?) and you'll remove that from the commits in here. Is that correct? > Fix startup issue with internode_application_timeout_in_ms > -- > > Key: CASSANDRA-17132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17132 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.x > > > While testing my patch for CASSANDRA-17131 I found that there is a problem > with _internode_application_timeout_in_ms_ in 4.0 and trunk. > Seems to me that we can just safely remove it for now? -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17132) Fix startup issue with internode_application_timeout_in_ms
[ https://issues.apache.org/jira/browse/CASSANDRA-17132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443845#comment-17443845 ] Ekaterina Dimitrova commented on CASSANDRA-17132: - {quote}[~e.dimitrova] your '4.0 patch' link point to a compare between _trunk_ and your 4.0 PR instead of 4.0 and your 4.0 PR #justfyi {quote} I grabbed the link on commit but I didn't open a PR, it is just the comparison so it requires to get the 4.0 from the drop-down menu. Works perfect with trunk, but I had to open a PR for 4.0. {quote} - Which rename is a breaking change in 4.0? {{internode_socket_send_buffer_size_in_bytes}} was changed in the code long ago and here we're only fixing the yml. it's like we're fixing a bug rather than introducing a rename right?{quote} The thing is that we have on one side a bug(the name not changed in cassandra.yaml), on the other one a breaking change so it is good to have the backward compatibility with the old name for people who upgrade, similar to what we did with metrics. I already pulled a ticket and agreed on the details with Benedict, Alex and Aleksey. Thank you for the review [~bereng] > Fix startup issue with internode_application_timeout_in_ms > -- > > Key: CASSANDRA-17132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17132 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.x > > > While testing my patch for CASSANDRA-17131 I found that there is a problem > with _internode_application_timeout_in_ms_ in 4.0 and trunk. > Seems to me that we can just safely remove it for now? -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17072) DebuggableThreadPoolExecutor does not propagate client warnings
[ https://issues.apache.org/jira/browse/CASSANDRA-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-17072: -- Fix Version/s: (was: 4.x) > DebuggableThreadPoolExecutor does not propagate client warnings > --- > > Key: CASSANDRA-17072 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17072 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.0.x > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Extracted this as a separate ticket per discussion on CASSANDRA-17044 > The problem is in {{DebuggableThreadPoolExecutor}} - it does not propagate > executor locals to the thread when tracing is disabled. It is probably > expected that we propagate the state if at least one executor local is > defined, but we only check for tracing and completely ignore client warnings. > The attached PR fixes the problem, adds some tests and reverts a workaround > for client warnings in some schema alteration statements implemented in > CASSANDRA-16296 (described below). > > h4. Old description - still valid, but this is just a manifestation of the > problem rather than the problem itself > This seemed to be screwed a bit. In just two schema alteration statements we > collect client warnings which are captured during the transformation into a > local collection. > I guess it is done that way because the transformation is being executed in a > different stage (migration) and client warnings collected in that stage are > not present in the stage where the query is executed. > Then, the client warnings are retrieved using {{clientWarnings}} method and > added to the captured client warnings in the stage which is executing the > query. > This mechanism was implemented only in two schema alteration statements. It > is possible that for other ones the client warnings can simply get lost. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17072) DebuggableThreadPoolExecutor does not propagate client warnings
[ https://issues.apache.org/jira/browse/CASSANDRA-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443717#comment-17443717 ] Jacek Lewandowski commented on CASSANDRA-17072: --- The review remarks regarding dead code and testing were addressed. I refactored the tests and added more (also for tracing). Also, the same problem was in {{SEPExecutor}} (plus one more nit where the {{ExecutorLocals}} instance was passed as a result rather than as locals due to overloading confusion). For trunk there are only tests. > DebuggableThreadPoolExecutor does not propagate client warnings > --- > > Key: CASSANDRA-17072 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17072 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.0.x, 4.x > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Extracted this as a separate ticket per discussion on CASSANDRA-17044 > The problem is in {{DebuggableThreadPoolExecutor}} - it does not propagate > executor locals to the thread when tracing is disabled. It is probably > expected that we propagate the state if at least one executor local is > defined, but we only check for tracing and completely ignore client warnings. > The attached PR fixes the problem, adds some tests and reverts a workaround > for client warnings in some schema alteration statements implemented in > CASSANDRA-16296 (described below). > > h4. Old description - still valid, but this is just a manifestation of the > problem rather than the problem itself > This seemed to be screwed a bit. In just two schema alteration statements we > collect client warnings which are captured during the transformation into a > local collection. > I guess it is done that way because the transformation is being executed in a > different stage (migration) and client warnings collected in that stage are > not present in the stage where the query is executed. > Then, the client warnings are retrieved using {{clientWarnings}} method and > added to the captured client warnings in the stage which is executing the > query. > This mechanism was implemented only in two schema alteration statements. It > is possible that for other ones the client warnings can simply get lost. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17132) Fix startup issue with internode_application_timeout_in_ms
[ https://issues.apache.org/jira/browse/CASSANDRA-17132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443669#comment-17443669 ] Berenguer Blasi commented on CASSANDRA-17132: - [~e.dimitrova] your '4.0 patch' link point to a compare between _trunk_ and your 4.0 PR instead of 4.0 and your 4.0 PR #justfyi Just a couple comments: - I _think_ [~dcapwell] was on top of the bootstrap failures and there is some umbrella ticket somewhere already? - The dtest failures on an address already being used I concur they are probably infra problems. I have seen than before and it sounds like a bad ccm env cleanup imo. - Which rename is a breaking change in 4.0? {{internode_socket_send_buffer_size_in_bytes}} was changed in the code long ago and here we're only fixing the yml. it's like we're fixing a bug rather than introducing a rename right? Otherwise lgtm. > Fix startup issue with internode_application_timeout_in_ms > -- > > Key: CASSANDRA-17132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17132 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.x > > > While testing my patch for CASSANDRA-17131 I found that there is a problem > with _internode_application_timeout_in_ms_ in 4.0 and trunk. > Seems to me that we can just safely remove it for now? -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17132) Fix startup issue with internode_application_timeout_in_ms
[ https://issues.apache.org/jira/browse/CASSANDRA-17132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-17132: Reviewers: Berenguer Blasi, Berenguer Blasi Berenguer Blasi, Berenguer Blasi (was: Berenguer Blasi) Status: Review In Progress (was: Patch Available) > Fix startup issue with internode_application_timeout_in_ms > -- > > Key: CASSANDRA-17132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17132 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.x > > > While testing my patch for CASSANDRA-17131 I found that there is a problem > with _internode_application_timeout_in_ms_ in 4.0 and trunk. > Seems to me that we can just safely remove it for now? -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17119) Broken test_speculative_data_request
[ https://issues.apache.org/jira/browse/CASSANDRA-17119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-17119: Since Version: 4.1 Source Control Link: https://github.com/apache/cassandra-dtest/commit/01a647f4a1ec42d42198fe4c49fe654a7705d7d7 Resolution: Fixed Status: Resolved (was: Ready to Commit) > Broken test_speculative_data_request > > > Key: CASSANDRA-17119 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17119 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0.x, 4.x > > > This has been failing consistently lately > {noformat} > Error Message > KeyError: '127.0.0.3' > Stacktrace > self = > @since('4.0') > def test_speculative_data_request(self): > """ If one node doesn't respond to a full data request, it should > query the other """ > node1, node2, node3 = self.cluster.nodelist() > assert isinstance(node1, Node) > assert isinstance(node2, Node) > assert isinstance(node3, Node) > session = self.get_cql_connection(node1, timeout=2) > > session.execute(quorum("INSERT INTO ks.tbl (k, c, v) VALUES (1, 0, > 1)")) > > node2.byteman_submit(['./byteman/read_repair/stop_writes.btm']) > node3.byteman_submit(['./byteman/read_repair/stop_writes.btm']) > > session.execute("INSERT INTO ks.tbl (k, c, v) VALUES (1, 1, 2)") > > # re-enable writes > node2.byteman_submit(['-u', './byteman/read_repair/stop_writes.btm']) > > > node1.byteman_submit(['./byteman/read_repair/sorted_live_endpoints.btm']) > node1.byteman_submit(['./byteman/request_verb_timing.btm']) > > with StorageProxy(node1) as storage_proxy: > assert storage_proxy.blocking_read_repair == 0 > assert storage_proxy.speculated_rr_read == 0 > assert storage_proxy.speculated_rr_write == 0 > > session = self.get_cql_connection(node1) > > node2.byteman_submit(['./byteman/read_repair/stop_data_reads.btm']) > results = session.execute(quorum("SELECT * FROM ks.tbl WHERE > k=1")) > > timing = request_verb_timing(node1) > > repair_req_node3 = timing[node3.ip_addr].get('READ_REPAIR_REQ') > E KeyError: '127.0.0.3' > read_repair_test.py:621: KeyError > {noformat} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch trunk updated: Broken test_speculative_data_request
This is an automated email from the ASF dual-hosted git repository. bereng pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git The following commit(s) were added to refs/heads/trunk by this push: new 01a647f Broken test_speculative_data_request 01a647f is described below commit 01a647f4a1ec42d42198fe4c49fe654a7705d7d7 Author: Bereng AuthorDate: Mon Nov 8 07:45:47 2021 +0100 Broken test_speculative_data_request patch by Berenguer Blasi; reviewed by Brandon Williams for CASSANDRA-17119 --- byteman/{ => post4.0}/request_verb_timing.btm | 0 byteman/request_verb_timing.btm | 4 ++-- read_repair_test.py | 6 +- 3 files changed, 7 insertions(+), 3 deletions(-) diff --git a/byteman/request_verb_timing.btm b/byteman/post4.0/request_verb_timing.btm similarity index 100% copy from byteman/request_verb_timing.btm copy to byteman/post4.0/request_verb_timing.btm diff --git a/byteman/request_verb_timing.btm b/byteman/request_verb_timing.btm index c42968e..e0dc68e 100644 --- a/byteman/request_verb_timing.btm +++ b/byteman/request_verb_timing.btm @@ -3,10 +3,10 @@ CLASS org.apache.cassandra.net.MessagingService METHOD doSend AT ENTRY BIND prefix:String = "org.jboss.byteman."; # byteman in strict mode requires the o.j.b prefix -toHost:String = $to.getAddress().toString(); +toHost:String = $to.address.toString(); verb:String = $message.header.verb.toString(); prop:String = prefix + "|request_verb_timing|" + toHost + "|" + verb; IF true DO System.setProperty(prop, String.valueOf(System.currentTimeMillis())); -ENDRULE +ENDRULE \ No newline at end of file diff --git a/read_repair_test.py b/read_repair_test.py index f3cb6c2..ee189d5 100644 --- a/read_repair_test.py +++ b/read_repair_test.py @@ -606,7 +606,11 @@ class TestSpeculativeReadRepair(Tester): node2.byteman_submit(['-u', './byteman/read_repair/stop_writes.btm']) node1.byteman_submit(['./byteman/read_repair/sorted_live_endpoints.btm']) -node1.byteman_submit(['./byteman/request_verb_timing.btm']) +version = self.cluster.cassandra_version() +if version < '4.1': +node1.byteman_submit(['./byteman/request_verb_timing.btm']) +else: +node1.byteman_submit(['./byteman/post4.0/request_verb_timing.btm']) with StorageProxy(node1) as storage_proxy: assert storage_proxy.blocking_read_repair == 0 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-site updated (11a8f1c -> 56fcf3d)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/cassandra-website.git. from 11a8f1c generated for 6c65ab44b769d107fe13388763f37e8717c806c0 add 56fcf3d generate website for b954b931684e544ab50370269427f4c4ed9ae675 No new revisions were added by this update. Summary of changes: ...What-the-Future-Holds-for-Apache-Cassandra.html |2 +- content/_/development/ide.html | 154 +- content/_/development/index.html | 1846 +--- content/_/development/patches.html | 25 +- content/_/development/testing.html | 1466 +++- content/search-index.js|2 +- content/sitemap-Cassandra.xml | 840 - content/sitemap-_.xml | 122 +- 8 files changed, 3656 insertions(+), 801 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org