[jira] [Commented] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[ https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248276#comment-17248276 ] David Capwell commented on CASSANDRA-16328: --- I have classified and bucketed the failing tests and opened a ticket for each test as a child of CASSANDRA-16322. After upgrading the 3.0 version to .23 the amount of tests failing are ~30, but given the nature of the tests most are the same failure repeated over the different permutations. > python upgrade tests include tests which are not impacted by the version > under test > --- > > Key: CASSANDRA-16328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16328 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 0.5h > Remaining Estimate: 0h > > When we try to run the upgrade tests for trunk, we get tests such as > upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x > to run, which do not include the trunk source. Since these tests are costly > and can be flaky, running the unneeded tests for a release increases the cost > and can make it harder to see if any new failures. > See > https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16346) Fix upgrade python dtest test_crc_check_chance_upgrade - upgrade_crc_check_chance_test.TestCrcCheckChanceUpgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-16346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16346:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Low Hanging Fruit
Discovered By: Unit Test
Fix Version/s: 4.0-beta
Severity: Normal
Status: Open (was: Triage Needed)
> Fix upgrade python dtest test_crc_check_chance_upgrade -
> upgrade_crc_check_chance_test.TestCrcCheckChanceUpgrade
>
>
> Key: CASSANDRA-16346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16346
> Project: Cassandra
> Issue Type: Bug
> Components: CI, Test/dtest/python
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/846/workflows/bee81d86-f720-4fc2-b3e7-7ed19ec3c9f0/jobs/4988/tests
> test_crc_check_chance_upgrade -
> upgrade_crc_check_chance_test.TestCrcCheckChanceUpgrade
> {code}
> > self.verify_old_crc_check_chance(node1)
> upgrade_crc_check_chance_test.py:58:
> {code}
> {code}
> self = 0x7f2d1d3b4be0>
> node =
> def verify_old_crc_check_chance(self, node):
> session = self.patient_exclusive_cql_connection(node)
> session.cluster.refresh_schema_metadata(0)
> meta = session.cluster.metadata.keyspaces['ks'].tables['cf1']
> > logger.debug(meta.options['compression_parameters'])
> E NameError: name 'logger' is not defined
> upgrade_crc_check_chance_test.py:107: NameError
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16346) Fix upgrade python dtest test_crc_check_chance_upgrade - upgrade_crc_check_chance_test.TestCrcCheckChanceUpgrade
David Capwell created CASSANDRA-16346:
-
Summary: Fix upgrade python dtest test_crc_check_chance_upgrade -
upgrade_crc_check_chance_test.TestCrcCheckChanceUpgrade
Key: CASSANDRA-16346
URL: https://issues.apache.org/jira/browse/CASSANDRA-16346
Project: Cassandra
Issue Type: Bug
Components: CI, Test/dtest/python
Reporter: David Capwell
https://app.circleci.com/pipelines/github/dcapwell/cassandra/846/workflows/bee81d86-f720-4fc2-b3e7-7ed19ec3c9f0/jobs/4988/tests
test_crc_check_chance_upgrade -
upgrade_crc_check_chance_test.TestCrcCheckChanceUpgrade
{code}
> self.verify_old_crc_check_chance(node1)
upgrade_crc_check_chance_test.py:58:
{code}
{code}
self =
node =
def verify_old_crc_check_chance(self, node):
session = self.patient_exclusive_cql_connection(node)
session.cluster.refresh_schema_metadata(0)
meta = session.cluster.metadata.keyspaces['ks'].tables['cf1']
> logger.debug(meta.options['compression_parameters'])
E NameError: name 'logger' is not defined
upgrade_crc_check_chance_test.py:107: NameError
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16345) Test SSTables are in the correct location after range movement with 1/5/20 data directories
Yifan Cai created CASSANDRA-16345: - Summary: Test SSTables are in the correct location after range movement with 1/5/20 data directories Key: CASSANDRA-16345 URL: https://issues.apache.org/jira/browse/CASSANDRA-16345 Project: Cassandra Issue Type: Sub-task Components: Test/benchmark Reporter: Yifan Cai Assignee: Yifan Cai The testing cluster should be pre-populated with ~200GB data in each node. Change the token ranges and verify that each data directory only contains SSTables belongs to it. In other word, no SSTables are relocated by running nodetool "relocatesstables". Both LCS and STCS need to be covered. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16344) Print the result at the end of the nodetool "relocatesstables"
Yifan Cai created CASSANDRA-16344: - Summary: Print the result at the end of the nodetool "relocatesstables" Key: CASSANDRA-16344 URL: https://issues.apache.org/jira/browse/CASSANDRA-16344 Project: Cassandra Issue Type: Sub-task Components: Tool/nodetool Reporter: Yifan Cai Assignee: Yifan Cai Print the summary of sstables relocation at the end of the command. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16342) LCS steady state load of 4.0 vs. 3.x performance test
Yifan Cai created CASSANDRA-16342: - Summary: LCS steady state load of 4.0 vs. 3.x performance test Key: CASSANDRA-16342 URL: https://issues.apache.org/jira/browse/CASSANDRA-16342 Project: Cassandra Issue Type: Sub-task Components: Test/benchmark Reporter: Yifan Cai Assignee: Yifan Cai The testing cluster should be pre-populated with ~200GB data in each node. Run the steady state workload to compare the read, write and compaction performance between baseline (3.x cluster) and 4.0 cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16341) Add test to check nodetool "garbagecollect" should remove deleted data from SSTables.
Yifan Cai created CASSANDRA-16341: - Summary: Add test to check nodetool "garbagecollect" should remove deleted data from SSTables. Key: CASSANDRA-16341 URL: https://issues.apache.org/jira/browse/CASSANDRA-16341 Project: Cassandra Issue Type: Sub-task Components: Test/unit Reporter: Yifan Cai Assignee: Yifan Cai We currently have no test that test the garbagecollect nodetool command. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16343) STCS steady state load of 4.0 vs 3.x performance test.
Yifan Cai created CASSANDRA-16343: - Summary: STCS steady state load of 4.0 vs 3.x performance test. Key: CASSANDRA-16343 URL: https://issues.apache.org/jira/browse/CASSANDRA-16343 Project: Cassandra Issue Type: Sub-task Components: Test/benchmark Reporter: Yifan Cai Assignee: Yifan Cai The testing cluster should be pre-populated with ~200GB data in each node. Run the steady state workload to compare the read, write and compaction performance between baseline (3.x cluster) and 4.0 cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16339) LCS steady state load of table with vs. w/o GC performance test
Yifan Cai created CASSANDRA-16339:
-
Summary: LCS steady state load of table with vs. w/o GC
performance test
Key: CASSANDRA-16339
URL: https://issues.apache.org/jira/browse/CASSANDRA-16339
Project: Cassandra
Issue Type: Sub-task
Components: Test/benchmark
Reporter: Yifan Cai
Assignee: Yifan Cai
The testing cluster should be pre-populated with ~200GB data in each node. The
baseline cluster has the table created with {{provide_overlapping_tombstones}}
disabled. The other cluster has the table with {{provide_overlapping_tombstones
== row}}. Compare the read, write and compaction performance between those 2
clusters.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16340) STCS steady state load of table with vs. w/o GC performance test
Yifan Cai created CASSANDRA-16340:
-
Summary: STCS steady state load of table with vs. w/o GC
performance test
Key: CASSANDRA-16340
URL: https://issues.apache.org/jira/browse/CASSANDRA-16340
Project: Cassandra
Issue Type: Sub-task
Components: Test/benchmark
Reporter: Yifan Cai
Assignee: Yifan Cai
The testing cluster should be pre-populated with ~200GB data in each node. The
baseline cluster has the table created with {{provide_overlapping_tombstones}}
disabled. The other cluster has the table with {{provide_overlapping_tombstones
== row}}. Compare the read, write and compaction performance between those 2
clusters.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16337) LCS steady state load vs. steady state load with garbagecollect running performance test
[ https://issues.apache.org/jira/browse/CASSANDRA-16337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-16337: -- Parent: CASSANDRA-15581 Issue Type: Sub-task (was: Task) > LCS steady state load vs. steady state load with garbagecollect running > performance test > > > Key: CASSANDRA-16337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16337 > Project: Cassandra > Issue Type: Sub-task > Components: Test/benchmark >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > The testing cluster should be pre-populated with ~200GB data in each node. > Run the steady state workload to compare the read, write and compaction > performance between before and during garbagecollect is running. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16338) STCS steady state load vs. steady state load with garbagecollect running performance test
Yifan Cai created CASSANDRA-16338: - Summary: STCS steady state load vs. steady state load with garbagecollect running performance test Key: CASSANDRA-16338 URL: https://issues.apache.org/jira/browse/CASSANDRA-16338 Project: Cassandra Issue Type: Sub-task Components: Test/benchmark Reporter: Yifan Cai Assignee: Yifan Cai The testing cluster should be pre-populated with ~200GB data in each node. Run the steady state workload to compare the read, write and compaction performance between before and during garbagecollect is running. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16337) LCS steady state load vs. steady state load with garbagecollect running performance test
Yifan Cai created CASSANDRA-16337: - Summary: LCS steady state load vs. steady state load with garbagecollect running performance test Key: CASSANDRA-16337 URL: https://issues.apache.org/jira/browse/CASSANDRA-16337 Project: Cassandra Issue Type: Task Components: Test/benchmark Reporter: Yifan Cai Assignee: Yifan Cai The testing cluster should be pre-populated with ~200GB data in each node. Run the steady state workload to compare the read, write and compaction performance between before and during garbagecollect is running. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15581) 4.0 quality testing: Compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai reassigned CASSANDRA-15581: - Assignee: Yifan Cai > 4.0 quality testing: Compaction > --- > > Key: CASSANDRA-15581 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15581 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/python >Reporter: Josh McKenzie >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0-beta > > > Reference [doc from > NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] > for context. > *Shepherd: Marcus Eriksson* > Alongside the local and distributed read/write paths, we'll also want to > validate compaction. CASSANDRA-6696 introduced substantial > changes/improvements that require testing (esp. JBOD). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15581) 4.0 quality testing: Compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248268#comment-17248268 ] Yifan Cai commented on CASSANDRA-15581: --- I talked with [~marcuse] about the action items for this ticket. I am assigning it to myself and creating child tickets for each individual action item. As mentioned, the major compaction changes were CASSANDRA-6696 and CASSANDRA-7019. The testing work will cover performance and correctness relates to those 2 patches. > 4.0 quality testing: Compaction > --- > > Key: CASSANDRA-15581 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15581 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/python >Reporter: Josh McKenzie >Priority: Normal > Fix For: 4.0-beta > > > Reference [doc from > NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] > for context. > *Shepherd: Marcus Eriksson* > Alongside the local and distributed read/write paths, we'll also want to > validate compaction. CASSANDRA-6696 introduced substantial > changes/improvements that require testing (esp. JBOD). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[ https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248239#comment-17248239 ] Michael Semb Wever commented on CASSANDRA-16328: Solid write up [~dcapwell], immensely helpful to have all that written down. > python upgrade tests include tests which are not impacted by the version > under test > --- > > Key: CASSANDRA-16328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16328 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 0.5h > Remaining Estimate: 0h > > When we try to run the upgrade tests for trunk, we get tests such as > upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x > to run, which do not include the trunk source. Since these tests are costly > and can be flaky, running the unneeded tests for a release increases the cost > and can make it harder to see if any new failures. > See > https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248213#comment-17248213
]
Caleb Rackliffe edited comment on CASSANDRA-16226 at 12/11/20, 10:18 PM:
-
Thanks [~ifesdjeen].
bq. add tests for tombstones, since previously we weren't taking them into
consideration
Indeed, although there is some coverage of this logic across
{{SSTablesIteratedTest}}, {{DeleteTest}}, and {{UpgradeTest}}. It might be
useful to have a test or two around the number of SSTables hit with range
tombstones in {{SSTablesIteratedTest}}.
bq. should be "more recent than"
Fixed.
bq. should we consider renaming canRemoveRow?
I settled on {{isRowComplete()}}, which at least describes what the method
tells us, and _not_ what it tells use we can do as a result.
New commit with the changes above is
[here|https://github.com/apache/cassandra/pull/823/commits/b71d830ebbe2e5726de2c18b03179ca2b8a74023].
...and just to have everything in one place for [~mck]: [3.0
patch|https://github.com/apache/cassandra/pull/823],
[CircleCI|https://app.circleci.com/pipelines/github/maedhroz/cassandra/171/workflows/4781de11-c5e2-40b1-967c-0e96728f843b],
[CircleCI Unit Tests
Only|https://app.circleci.com/pipelines/github/maedhroz/cassandra/169/workflows/93ea354c-cd12-48a5-a0ec-d88ca1a0dc86].
(For some reason, the HIGHRES configuration isn't handling the unit tests
properly.)
I'll post the 3.11 and 4.0 branches, which I expect not to deviate much (if at
all) from the 3.0 diff, once we've got a second +1.
was (Author: maedhroz):
Thanks [~ifesdjeen].
bq. add tests for tombstones, since previously we weren't taking them into
consideration
Indeed, although there is some coverage of this logic across
{{SSTablesIteratedTest}}, {{DeleteTest}}, and {{UpgradeTest}}. It might be
useful to have a test or two around the number of SSTables hit with range
tombstones in {{SSTablesIteratedTest}}.
bq. should be "more recent than"
Fixed.
bq. should we consider renaming canRemoveRow?
I settled on {{isRowComplete()}}, which at least describes what the method
tells us, and _not_ what it tells use we can do as a result.
New commit with the changes above is
[here|https://github.com/apache/cassandra/pull/823/commits/b71d830ebbe2e5726de2c18b03179ca2b8a74023].
...and just to have everything in one place for [~mck]: [3.0
patch|https://github.com/apache/cassandra/pull/823],
[CircleCI|https://app.circleci.com/pipelines/github/maedhroz/cassandra/171/workflows/4781de11-c5e2-40b1-967c-0e96728f843b]
I'll post the 3.11 and 4.0 branches, which I expect not to deviate much (if at
all) from the 3.0 diff, once we've got a second +1.
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}}
[jira] [Commented] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248213#comment-17248213
]
Caleb Rackliffe commented on CASSANDRA-16226:
-
Thanks [~ifesdjeen].
bq. add tests for tombstones, since previously we weren't taking them into
consideration
Indeed, although there is some coverage of this logic across
{{SSTablesIteratedTest}}, {{DeleteTest}}, and {{UpgradeTest}}. It might be
useful to have a test or two around the number of SSTables hit with range
tombstones in {{SSTablesIteratedTest}}.
bq. should be "more recent than"
Fixed.
bq. should we consider renaming canRemoveRow?
I settled on {{isRowComplete()}}, which at least describes what the method
tells us, and _not_ what it tells use we can do as a result.
New commit with the changes above is
[here|https://github.com/apache/cassandra/pull/823/commits/b71d830ebbe2e5726de2c18b03179ca2b8a74023].
...and just to have everything in one place for [~mck]: [3.0
patch|https://github.com/apache/cassandra/pull/823],
[CircleCI|https://app.circleci.com/pipelines/github/maedhroz/cassandra/171/workflows/4781de11-c5e2-40b1-967c-0e96728f843b]
I'll post the 3.11 and 4.0 branches, which I expect not to deviate much (if at
all) from the 3.0 diff, once we've got a second +1.
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key
> liveness information for non-compact/CQL tables. Tombstones seem to be
> handled at a level above that anyway. (One potential problem with that is
> whether or not the distinction will continue to exist in 4.0, and dropping
> compact storage from a table doesn't magically make pk liveness information
> appear.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16329) python dtest upgrade tests do not generate 3.0 to trunk tests
[ https://issues.apache.org/jira/browse/CASSANDRA-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16329: -- Test and Documentation Plan: tested in CASSANDRA-16328 Status: Patch Available (was: In Progress) > python dtest upgrade tests do not generate 3.0 to trunk tests > - > > Key: CASSANDRA-16329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16329 > Project: Cassandra > Issue Type: Improvement > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > It looks like 3.11 is the only pair tested against trunk, we should update > this to include 3.0 -> trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16329) python dtest upgrade tests do not generate 3.0 to trunk tests
[ https://issues.apache.org/jira/browse/CASSANDRA-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248190#comment-17248190 ] David Capwell commented on CASSANDRA-16329: --- since the patch is small, moving it to CASSANDRA-16328 so I can also fix CI > python dtest upgrade tests do not generate 3.0 to trunk tests > - > > Key: CASSANDRA-16329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16329 > Project: Cassandra > Issue Type: Improvement > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > It looks like 3.11 is the only pair tested against trunk, we should update > this to include 3.0 -> trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[
https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248189#comment-17248189
]
David Capwell edited comment on CASSANDRA-16328 at 12/11/20, 9:25 PM:
--
Ok, I found out the root cause why 3.0 was missing
1) Marcus was right
2) python is dynamic and a small typo will break you... silently... =D
3) no option allowed 3.0 release -> trunk AND 3.0 branch -> trunk
With a change to fix both issues, and updating CI with a new flag, we now get
{code}
$ wc -l tests-only-no-static.txt
1850 tests-only-no-static.txt
$ grep current_3_0_x_To_indev_trunk tests-only-no-static.txt | head -n 2
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_trunk::test_static_cf
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_trunk::test_large_collection_errors
$ grep indev_3_0_x_To_indev_trunk tests-only-no-static.txt | head -n 2
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_indev_3_0_x_To_indev_trunk::test_static_cf
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_indev_3_0_x_To_indev_trunk::test_large_collection_errors
{code}
was (Author: dcapwell):
Ok, I found out the root cause why 3.0 was missing
1) Marcus was right
2) python is dynamic and a small typo will break you... silently... =D
With a change to fix both issues, and updating CI with a new flag, we now get
{code}
$ wc -l tests-only-no-static.txt
1850 tests-only-no-static.txt
$ grep current_3_0_x_To_indev_trunk tests-only-no-static.txt | head -n 2
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_trunk::test_static_cf
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_trunk::test_large_collection_errors
$ grep indev_3_0_x_To_indev_trunk tests-only-no-static.txt | head -n 2
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_indev_3_0_x_To_indev_trunk::test_static_cf
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_indev_3_0_x_To_indev_trunk::test_large_collection_errors
{code}
> python upgrade tests include tests which are not impacted by the version
> under test
> ---
>
> Key: CASSANDRA-16328
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16328
> Project: Cassandra
> Issue Type: Bug
> Components: CI, Test/dtest/python
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When we try to run the upgrade tests for trunk, we get tests such as
> upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x
> to run, which do not include the trunk source. Since these tests are costly
> and can be flaky, running the unneeded tests for a release increases the cost
> and can make it harder to see if any new failures.
> See
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[
https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248189#comment-17248189
]
David Capwell commented on CASSANDRA-16328:
---
Ok, I found out the root cause why 3.0 was missing
1) Marcus was right
2) python is dynamic and a small typo will break you... silently... =D
With a change to fix both issues, and updating CI with a new flag, we now get
{code}
$ wc -l tests-only-no-static.txt
1850 tests-only-no-static.txt
$ grep current_3_0_x_To_indev_trunk tests-only-no-static.txt | head -n 2
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_trunk::test_static_cf
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_trunk::test_large_collection_errors
$ grep indev_3_0_x_To_indev_trunk tests-only-no-static.txt | head -n 2
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_indev_3_0_x_To_indev_trunk::test_static_cf
upgrade_tests/cql_tests.py::TestCQLNodes3RF3_Upgrade_indev_3_0_x_To_indev_trunk::test_large_collection_errors
{code}
> python upgrade tests include tests which are not impacted by the version
> under test
> ---
>
> Key: CASSANDRA-16328
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16328
> Project: Cassandra
> Issue Type: Bug
> Components: CI, Test/dtest/python
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When we try to run the upgrade tests for trunk, we get tests such as
> upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x
> to run, which do not include the trunk source. Since these tests are costly
> and can be flaky, running the unneeded tests for a release increases the cost
> and can make it harder to see if any new failures.
> See
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16329) python dtest upgrade tests do not generate 3.0 to trunk tests
[
https://issues.apache.org/jira/browse/CASSANDRA-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248174#comment-17248174
]
David Capwell commented on CASSANDRA-16329:
---
testing --upgrade-version-selection both and --upgrade-version-selection BOTH
both produce
{code}
upgrade_tests/upgrade_udtfix_test.py:149: in
for path in build_upgrade_pairs():
upgrade_tests/upgrade_manifest.py:196: in build_upgrade_pairs
version_select_strategy =
VersionSelectionStrategies[configured_strategy].value[0]
/opt/rh/rh-python36/root/usr/lib64/python3.6/enum.py:329: in __getitem__
return cls._member_map_[name]
E KeyError: 'BOTH'
{code}
> python dtest upgrade tests do not generate 3.0 to trunk tests
> -
>
> Key: CASSANDRA-16329
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16329
> Project: Cassandra
> Issue Type: Improvement
> Components: CI, Test/dtest/python
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> It looks like 3.11 is the only pair tested against trunk, we should update
> this to include 3.0 -> trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16329) python dtest upgrade tests do not generate 3.0 to trunk tests
[
https://issues.apache.org/jira/browse/CASSANDRA-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248170#comment-17248170
]
David Capwell commented on CASSANDRA-16329:
---
So, when running against a trunk branch, the reason 3.0 release isn't included
is that we filter it out here:
{code}
if not version_select_strategy(origin_meta, destination_meta):
continue
{code}
Turns out --upgrade-version-selection default is INDEV, so release builds are
exulted from testing
> python dtest upgrade tests do not generate 3.0 to trunk tests
> -
>
> Key: CASSANDRA-16329
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16329
> Project: Cassandra
> Issue Type: Improvement
> Components: CI, Test/dtest/python
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> It looks like 3.11 is the only pair tested against trunk, we should update
> this to include 3.0 -> trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16329) python dtest upgrade tests do not generate 3.0 to trunk tests
[ https://issues.apache.org/jira/browse/CASSANDRA-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell reassigned CASSANDRA-16329: - Assignee: David Capwell > python dtest upgrade tests do not generate 3.0 to trunk tests > - > > Key: CASSANDRA-16329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16329 > Project: Cassandra > Issue Type: Improvement > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > It looks like 3.11 is the only pair tested against trunk, we should update > this to include 3.0 -> trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[
https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248140#comment-17248140
]
David Capwell commented on CASSANDRA-16328:
---
According to git, the env variable was added in
{code}
Author: Russ Hatch
Date: Fri Jul 22 11:06:41 2016 -0600
default to env-relevant upgrade tests (#1095)
{code}
Comment would imply https://github.com/apache/cassandra-dtest/pull/1095 but
that does not exist, so not sure if I can track down the context of this commit.
[~rhatch], do you remember and able to comment?
> python upgrade tests include tests which are not impacted by the version
> under test
> ---
>
> Key: CASSANDRA-16328
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16328
> Project: Cassandra
> Issue Type: Bug
> Components: CI, Test/dtest/python
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When we try to run the upgrade tests for trunk, we get tests such as
> upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x
> to run, which do not include the trunk source. Since these tests are costly
> and can be flaky, running the unneeded tests for a release increases the cost
> and can make it harder to see if any new failures.
> See
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[ https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16328: -- Authors: David Capwell, Michael Semb Wever (was: David Capwell) > python upgrade tests include tests which are not impacted by the version > under test > --- > > Key: CASSANDRA-16328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16328 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 0.5h > Remaining Estimate: 0h > > When we try to run the upgrade tests for trunk, we get tests such as > upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x > to run, which do not include the trunk source. Since these tests are costly > and can be flaky, running the unneeded tests for a release increases the cost > and can make it harder to see if any new failures. > See > https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[ https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16328: -- Reviewers: Marcus Eriksson, Michael Semb Wever (was: Marcus Eriksson) > python upgrade tests include tests which are not impacted by the version > under test > --- > > Key: CASSANDRA-16328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16328 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 0.5h > Remaining Estimate: 0h > > When we try to run the upgrade tests for trunk, we get tests such as > upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x > to run, which do not include the trunk source. Since these tests are costly > and can be flaky, running the unneeded tests for a release increases the cost > and can make it harder to see if any new failures. > See > https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[
https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248139#comment-17248139
]
David Capwell commented on CASSANDRA-16328:
---
{noformat}
… RUN_STATIC_UPGRADE_MATRIX …
What is the purpose of this variable?
When would we want/need to use it?
{noformat}
Here is all the references I see
{code}
$ grep -r --exclude '*.pyc' RUN_STATIC_UPGRADE_MATRIX *
dtest.py:RUN_STATIC_UPGRADE_MATRIX =
os.environ.get('RUN_STATIC_UPGRADE_MATRIX', '').lower() in ('yes', 'true')
upgrade_tests/paging_test.py:from dtest import RUN_STATIC_UPGRADE_MATRIX,
run_scenarios, MAJOR_VERSION_4
upgrade_tests/paging_test.py:upgrade_applies_to_env =
RUN_STATIC_UPGRADE_MATRIX or
spec['UPGRADE_PATH'].upgrade_meta.matches_current_env_version_family
upgrade_tests/thrift_upgrade_test.py:from dtest import
RUN_STATIC_UPGRADE_MATRIX, Tester
upgrade_tests/thrift_upgrade_test.py:upgrade_applies_to_env =
RUN_STATIC_UPGRADE_MATRIX or
spec['UPGRADE_PATH'].upgrade_meta.matches_current_env_version_family
upgrade_tests/upgrade_through_versions_test.py:from dtest import
RUN_STATIC_UPGRADE_MATRIX, Tester
upgrade_tests/upgrade_through_versions_test.py:upgrade_applies_to_env =
RUN_STATIC_UPGRADE_MATRIX or
version_metas[-1].matches_current_env_version_family
upgrade_tests/upgrade_through_versions_test.py:if not
RUN_STATIC_UPGRADE_MATRIX:
upgrade_tests/cql_tests.py:from dtest import RUN_STATIC_UPGRADE_MATRIX,
MAJOR_VERSION_4
upgrade_tests/cql_tests.py:upgrade_applies_to_env =
RUN_STATIC_UPGRADE_MATRIX or
spec['UPGRADE_PATH'].upgrade_meta.matches_current_env_version_family
upgrade_tests/regression_test.py:from dtest import RUN_STATIC_UPGRADE_MATRIX
upgrade_tests/regression_test.py:upgrade_applies_to_env =
RUN_STATIC_UPGRADE_MATRIX or
path.upgrade_meta.matches_current_env_version_family
upgrade_tests/upgrade_manifest.py:from dtest import RUN_STATIC_UPGRADE_MATRIX
upgrade_tests/upgrade_manifest.py:if not (RUN_STATIC_UPGRADE_MATRIX
or OVERRIDE_MANIFEST):
upgrade_tests/upgrade_udtfix_test.py:from dtest import
RUN_STATIC_UPGRADE_MATRIX, Tester
upgrade_tests/upgrade_udtfix_test.py:upgrade_applies_to_env =
RUN_STATIC_UPGRADE_MATRIX or start_family_applies
{code}
For upgrade_tests/upgrade_manifest.py, its used as follows
{code}
if not (RUN_STATIC_UPGRADE_MATRIX or OVERRIDE_MANIFEST):
if destination_meta.matches_current_env_version_family:
# looks like this test should actually run in the current
env, so let's set the final version to match the env exactly
oldmeta = destination_meta
newmeta = destination_meta.clone_with_local_env_version()
logger.debug("{} appears applicable to current env.
Overriding final test version from {} to {}".format(path_name, oldmeta.version,
newmeta.version))
destination_meta = newmeta
{code}
This is what I was talked about when saying only committed code gets tested, if
RUN_STATIC_UPGRADE_MATRIX isn't defined (or is false) then we replace the
destination (why not origin?) with the version provided to cassandra_dir if
their "family" matches. The other usages look to be used for filtering, so if
RUN_STATIC_UPGRADE_MATRIX is defined, or the test version matches the
cassandra_dir version then run.
I am not sure why it is this way, or why origin was excluded (changes to 3.11
will not be tested against trunk upgrades until committed).
bq. When I look through the tests run I only see indev_3_11_x_To_indev_trunk
tests. I would have thought we should be including all
indev_3_0_x_To_indev_trunk as well?
bq. looks like we don't specify indev_3_0_x to trunk:
I filed CASSANDRA-16329 to look closer into this. Marcus you are correct that
3.0 branch excludes trunk, but 3.0 release includes it
(https://github.com/apache/cassandra-dtest/blob/trunk/upgrade_tests/upgrade_manifest.py#L155)...
yet 3.0 release isn't in the test output!
I was hoping to punt that issue into the other JIRA as I see that logging never
triggers, and stdout debugging also doesn't show to screen, so root causing
might take a bit more time.
bq. Here's the patch for the same in ci-cassandra.a.o in-sync
Thanks, was planning to file after review was happy with the change, ill update
the links to include yours!
> python upgrade tests include tests which are not impacted by the version
> under test
> ---
>
> Key: CASSANDRA-16328
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16328
> Project: Cassandra
> Issue Type: Bug
> Components: CI, Test/dtest/python
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
> Time
[jira] [Updated] (CASSANDRA-16311) Extend the exclusion of replica filtering protection to other indices instead of just SASI
[ https://issues.apache.org/jira/browse/CASSANDRA-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-16311: -- Since Version: 3.0.21 Source Control Link: https://github.com/apache/cassandra/commit/fa77676daa8e9726fd6ca96bb081cd288a21c200 Resolution: Fixed Status: Resolved (was: Ready to Commit) > Extend the exclusion of replica filtering protection to other indices instead > of just SASI > -- > > Key: CASSANDRA-16311 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16311 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SASI >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > There was a check introduced in CASSANDRA-8272 telling if an index is a SASI > index and if it is, replica filtering protection was not triggered. > There might be other custom index implementations which also do not support > filtering protection and they do not have to be SASI indices neccessarily, > however it is not possible to exclude them. > > PR trunk > [https://github.com/apache/cassandra/pull/844] > PR 3.11 > [https://github.com/apache/cassandra/pull/847] > PR 3.0 > https://github.com/apache/cassandra/pull/848 > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16336) cqlsh help command displays a broken url
[
https://issues.apache.org/jira/browse/CASSANDRA-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dylan Stephano-Shachter updated CASSANDRA-16336:
Description:
When running cqlsh the help command outputs a web page which returns 404
{code}
cqlsh> help create_table
*** No browser to display CQL help. URL for help topic create_table :
https://cassandra.apache.org/doc/cql3/CQL-3.2.html#createTableStmt
{code}
was:
When running cqlsh the help command outputs a web page which returns 404
{code}
cqlsh> help create_table
*** No browser to display CQL help. URL for help topic create_table :
https://cassandra.apache.org/doc/cql3/CQL-3.2.html#createTableStmt
{code}
> cqlsh help command displays a broken url
>
>
> Key: CASSANDRA-16336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16336
> Project: Cassandra
> Issue Type: Bug
>Reporter: Dylan Stephano-Shachter
>Priority: Normal
> Attachments: not_found.png
>
>
> When running cqlsh the help command outputs a web page which returns 404
> {code}
> cqlsh> help create_table
> *** No browser to display CQL help. URL for help topic create_table :
> https://cassandra.apache.org/doc/cql3/CQL-3.2.html#createTableStmt
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16311) Extend the exclusion of replica filtering protection to other indices instead of just SASI
[ https://issues.apache.org/jira/browse/CASSANDRA-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-16311: -- Status: Ready to Commit (was: Review In Progress) > Extend the exclusion of replica filtering protection to other indices instead > of just SASI > -- > > Key: CASSANDRA-16311 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16311 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SASI >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > There was a check introduced in CASSANDRA-8272 telling if an index is a SASI > index and if it is, replica filtering protection was not triggered. > There might be other custom index implementations which also do not support > filtering protection and they do not have to be SASI indices neccessarily, > however it is not possible to exclude them. > > PR trunk > [https://github.com/apache/cassandra/pull/844] > PR 3.11 > [https://github.com/apache/cassandra/pull/847] > PR 3.0 > https://github.com/apache/cassandra/pull/848 > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16336) cqlsh help command displays a broken url
[
https://issues.apache.org/jira/browse/CASSANDRA-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dylan Stephano-Shachter updated CASSANDRA-16336:
Description:
When running cqlsh the help command outputs a web page which returns 404
{code}
cqlsh> help create_table
*** No browser to display CQL help. URL for help topic create_table :
https://cassandra.apache.org/doc/cql3/CQL-3.2.html#createTableStmt
{code}
was:
When running cqlsh the help command outputs a web page which returns 404
```cqlsh> help create_table
*** No browser to display CQL help. URL for help topic create_table :
https://cassandra.apache.org/doc/cql3/CQL-3.2.html#createTableStmt```
> cqlsh help command displays a broken url
>
>
> Key: CASSANDRA-16336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16336
> Project: Cassandra
> Issue Type: Bug
>Reporter: Dylan Stephano-Shachter
>Priority: Normal
> Attachments: not_found.png
>
>
> When running cqlsh the help command outputs a web page which returns 404
>
> {code}
> cqlsh> help create_table
> *** No browser to display CQL help. URL for help topic create_table :
> https://cassandra.apache.org/doc/cql3/CQL-3.2.html#createTableStmt
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16311) Extend the exclusion of replica filtering protection to other indices instead of just SASI
[
https://issues.apache.org/jira/browse/CASSANDRA-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248109#comment-17248109
]
Andres de la Peña commented on CASSANDRA-16311:
---
The tests failures above don't seem related to the changes.
Committed to {{cassandra-3.0}} as
[fa77676daa8e9726fd6ca96bb081cd288a21c200|https://github.com/apache/cassandra/commit/fa77676daa8e9726fd6ca96bb081cd288a21c200]
and merged up to
[{{cassandra-3.11}}|https://github.com/apache/cassandra/commit/ae8981236ae06a5053775b7f55c8aeb77f8b9318]
and
[{{trunk}}|https://github.com/apache/cassandra/commit/8e61e216a67304482f4373fb8b53012a25404026].
Thanks for the patch and the review.
> Extend the exclusion of replica filtering protection to other indices instead
> of just SASI
> --
>
> Key: CASSANDRA-16311
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16311
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index, Feature/SASI
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> There was a check introduced in CASSANDRA-8272 telling if an index is a SASI
> index and if it is, replica filtering protection was not triggered.
> There might be other custom index implementations which also do not support
> filtering protection and they do not have to be SASI indices neccessarily,
> however it is not possible to exclude them.
>
> PR trunk
> [https://github.com/apache/cassandra/pull/844]
> PR 3.11
> [https://github.com/apache/cassandra/pull/847]
> PR 3.0
> https://github.com/apache/cassandra/pull/848
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16336) cqlsh help command displays a broken url
[ https://issues.apache.org/jira/browse/CASSANDRA-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Stephano-Shachter updated CASSANDRA-16336: Description: When running cqlsh the help command outputs a web page which returns 404 ```cqlsh> help create_table *** No browser to display CQL help. URL for help topic create_table : https://cassandra.apache.org/doc/cql3/CQL-3.2.html#createTableStmt``` was: When running cqlsh the help command outputs a web page which returns 404 cqlsh> help create_table *** No browser to display CQL help. URL for help topic create_table : https://cassandra.apache.org/doc/cql3/CQL-3.2.html#createTableStmt > cqlsh help command displays a broken url > > > Key: CASSANDRA-16336 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16336 > Project: Cassandra > Issue Type: Bug >Reporter: Dylan Stephano-Shachter >Priority: Normal > Attachments: not_found.png > > > When running cqlsh the help command outputs a web page which returns 404 > > ```cqlsh> help create_table > *** No browser to display CQL help. URL for help topic create_table : > https://cassandra.apache.org/doc/cql3/CQL-3.2.html#createTableStmt``` -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16336) cqlsh help command displays a broken url
Dylan Stephano-Shachter created CASSANDRA-16336: --- Summary: cqlsh help command displays a broken url Key: CASSANDRA-16336 URL: https://issues.apache.org/jira/browse/CASSANDRA-16336 Project: Cassandra Issue Type: Bug Reporter: Dylan Stephano-Shachter Attachments: not_found.png When running cqlsh the help command outputs a web page which returns 404 cqlsh> help create_table *** No browser to display CQL help. URL for help topic create_table : https://cassandra.apache.org/doc/cql3/CQL-3.2.html#createTableStmt -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Extend the exclusion of replica filtering protection to other indices instead of just SASI
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new fa77676 Extend the exclusion of replica filtering protection to other
indices instead of just SASI
fa77676 is described below
commit fa77676daa8e9726fd6ca96bb081cd288a21c200
Author: Stefan Miklosovic
AuthorDate: Fri Dec 11 18:05:52 2020 +
Extend the exclusion of replica filtering protection to other indices
instead of just SASI
patch by Stefan Miklosovic; reviewed by Andrés de la Peña and Zhao Yang for
CASSANDRA-16311#
---
CHANGES.txt| 1 +
src/java/org/apache/cassandra/index/Index.java | 17 +
.../org/apache/cassandra/service/DataResolver.java | 22 +-
3 files changed, 39 insertions(+), 1 deletion(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 1955168..d560d91 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.24:
+ * Extend the exclusion of replica filtering protection to other indices
instead of just SASI (CASSANDRA-16311)
* Synchronize transaction logs for JBOD (CASSANDRA-16225)
* Fix the counting of cells per partition (CASSANDRA-16259)
* Fix serial read/non-applying CAS linearizability (CASSANDRA-12126)
diff --git a/src/java/org/apache/cassandra/index/Index.java
b/src/java/org/apache/cassandra/index/Index.java
index 469ef07..843009b 100644
--- a/src/java/org/apache/cassandra/index/Index.java
+++ b/src/java/org/apache/cassandra/index/Index.java
@@ -433,6 +433,23 @@ public interface Index
}
/**
+ * Tells whether this index supports replica fitering protection or not.
+ *
+ * Replica filtering protection might need to run the query row filter in
the coordinator to detect stale results.
+ * An index implementation will be compatible with this protection
mechanism if it returns the same results for the
+ * row filter as CQL will return with {@code ALLOW FILTERING} and without
using the index. This means that index
+ * implementations using custom query syntax or applying transformations
to the indexed data won't support it.
+ * See CASSANDRA-8272 for further details.
+ *
+ * @param rowFilter rowFilter of query to decide if it supports replica
filtering protection or not
+ * @return true if this index supports replica filtering protection, false
otherwise
+ */
+default boolean supportsReplicaFilteringProtection(RowFilter rowFilter)
+{
+return true;
+}
+
+/**
* Return a function which performs post processing on the results of a
partition range read command.
* In future, this may be used as a generalized mechanism for transforming
results on the coordinator prior
* to returning them to the caller.
diff --git a/src/java/org/apache/cassandra/service/DataResolver.java
b/src/java/org/apache/cassandra/service/DataResolver.java
index 1d0bb47..cc17267 100644
--- a/src/java/org/apache/cassandra/service/DataResolver.java
+++ b/src/java/org/apache/cassandra/service/DataResolver.java
@@ -39,7 +39,9 @@ import org.apache.cassandra.dht.AbstractBounds;
import org.apache.cassandra.dht.ExcludingBounds;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.exceptions.ReadTimeoutException;
+import org.apache.cassandra.index.Index;
import org.apache.cassandra.net.*;
+import org.apache.cassandra.schema.IndexMetadata;
import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.utils.FBUtilities;
@@ -94,7 +96,25 @@ public class DataResolver extends ResponseResolver
private boolean needsReplicaFilteringProtection()
{
-return !command.rowFilter().isEmpty();
+if (command.rowFilter().isEmpty())
+return false;
+
+IndexMetadata indexMetadata = command.indexMetadata();
+
+if (indexMetadata == null || !indexMetadata.isCustom())
+{
+return true;
+}
+
+ColumnFamilyStore cfs =
ColumnFamilyStore.getIfExists(command.metadata().cfId);
+
+assert cfs != null;
+
+Index index = command.getIndex(cfs);
+
+assert index != null;
+
+return index.supportsReplicaFilteringProtection(command.rowFilter());
}
private class ResolveContext
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit ae8981236ae06a5053775b7f55c8aeb77f8b9318
Merge: d8a317f fa77676
Author: Andrés de la Peña
AuthorDate: Fri Dec 11 18:17:37 2020 +
Merge branch 'cassandra-3.0' into cassandra-3.11
# Conflicts:
# CHANGES.txt
# src/java/org/apache/cassandra/service/DataResolver.java
CHANGES.txt | 1 +
src/java/org/apache/cassandra/index/Index.java | 17 +
.../org/apache/cassandra/index/sasi/SASIIndex.java | 6 ++
.../org/apache/cassandra/service/DataResolver.java | 21 ++---
4 files changed, 38 insertions(+), 7 deletions(-)
diff --cc CHANGES.txt
index fb9921b,d560d91..a9a0cee
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,7 -1,5 +1,8 @@@
-3.0.24:
+3.11.10
+ * Rate limit validation compactions using compaction_throughput_mb_per_sec
(CASSANDRA-16161)
+ * SASI's `max_compaction_flush_memory_in_mb` settings over 100GB revert to
default of 1GB (CASSANDRA-16071)
+Merged from 3.0:
+ * Extend the exclusion of replica filtering protection to other indices
instead of just SASI (CASSANDRA-16311)
* Synchronize transaction logs for JBOD (CASSANDRA-16225)
* Fix the counting of cells per partition (CASSANDRA-16259)
* Fix serial read/non-applying CAS linearizability (CASSANDRA-12126)
diff --cc src/java/org/apache/cassandra/index/sasi/SASIIndex.java
index 4bf94ef,000..5ea7cec
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
+++ b/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
@@@ -1,352 -1,0 +1,358 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.index.sasi;
+
+import java.util.*;
+import java.util.concurrent.Callable;
+import java.util.function.BiFunction;
+
+import com.googlecode.concurrenttrees.common.Iterables;
+
+import org.apache.cassandra.config.*;
+import org.apache.cassandra.cql3.Operator;
+import org.apache.cassandra.cql3.statements.IndexTarget;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.db.compaction.OperationType;
+import org.apache.cassandra.db.filter.RowFilter;
+import org.apache.cassandra.db.lifecycle.Tracker;
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.partitions.PartitionIterator;
+import org.apache.cassandra.db.partitions.PartitionUpdate;
+import org.apache.cassandra.db.rows.Row;
+import org.apache.cassandra.dht.Murmur3Partitioner;
+import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.index.Index;
+import org.apache.cassandra.index.IndexRegistry;
+import org.apache.cassandra.index.SecondaryIndexBuilder;
+import org.apache.cassandra.index.TargetParser;
+import org.apache.cassandra.index.sasi.conf.ColumnIndex;
+import org.apache.cassandra.index.sasi.conf.IndexMode;
+import org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.Mode;
+import org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter;
+import org.apache.cassandra.index.sasi.plan.QueryPlan;
+import org.apache.cassandra.index.transactions.IndexTransaction;
+import org.apache.cassandra.io.sstable.Descriptor;
+import org.apache.cassandra.io.sstable.format.SSTableFlushObserver;
+import org.apache.cassandra.io.sstable.format.SSTableReader;
+import org.apache.cassandra.notifications.*;
+import org.apache.cassandra.schema.IndexMetadata;
+import org.apache.cassandra.utils.FBUtilities;
+import org.apache.cassandra.utils.Pair;
+import org.apache.cassandra.utils.concurrent.OpOrder;
+
+public class SASIIndex implements Index, INotificationConsumer
+{
+public final static String USAGE_WARNING = "SASI indexes are experimental
and are not recommended for production use.";
+
+private static class SASIIndexBuildingSupport implements
IndexBuildingSupport
+{
+public SecondaryIndexBuilder
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 8e61e216a67304482f4373fb8b53012a25404026
Merge: 1193180 ae89812
Author: Andrés de la Peña
AuthorDate: Fri Dec 11 18:34:25 2020 +
Merge branch 'cassandra-3.11' into trunk
# Conflicts:
# src/java/org/apache/cassandra/index/sasi/SASIIndex.java
# src/java/org/apache/cassandra/service/DataResolver.java
CHANGES.txt| 1 +
src/java/org/apache/cassandra/index/Index.java | 17 +
.../org/apache/cassandra/index/sasi/SASIIndex.java | 6 ++
.../cassandra/service/reads/DataResolver.java | 22 +++---
4 files changed, 39 insertions(+), 7 deletions(-)
diff --cc CHANGES.txt
index 520af90,a9a0cee..3b7c96d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,28 -1,8 +1,29 @@@
-3.11.10
- * Rate limit validation compactions using compaction_throughput_mb_per_sec
(CASSANDRA-16161)
+4.0-beta4
+ * Add dedicated tcp user timeout for streaming connection (CASSANDRA-16143)
+ * Add generatetokens script for offline token allocation strategy generation
(CASSANDRA-16205)
+ * Remove Windows scripts (CASSANDRA-16171)
+ * Improve checksumming and compression in protocol V5 (CASSANDRA-15299)
+ * Optimised repair streaming improvements (CASSANDRA-16274)
+ * Update jctools dependency to 3.1.0 (CASSANDRA-16255)
+ * 'SSLEngine closed already' exception on failed outbound connection
(CASSANDRA-16277)
+ * Drain and/or shutdown might throw because of slow messaging service
shutdown (CASSANDRA-16276)
+ * Upgrade JNA to 5.6.0, dropping support for <=glibc-2.6 systems
(CASSANDRA-16212)
+ * Add saved Host IDs to TokenMetadata at startup (CASSANDRA-16246)
+ * Ensure that CacheMetrics.requests is picked up by the metric reporter
(CASSANDRA-16228)
+ * Add a ratelimiter to snapshot creation and deletion (CASSANDRA-13019)
+ * Produce consistent tombstone for reads to avoid digest mistmatch
(CASSANDRA-15369)
+ * Fix SSTableloader issue when restoring a table named backups
(CASSANDRA-16235)
+ * Invalid serialized size for responses caused by increasing message time by
1ms which caused extra bytes in size calculation (CASSANDRA-16103)
+ * Throw BufferOverflowException from DataOutputBuffer for better visibility
(CASSANDRA-16214)
+ * TLS connections to the storage port on a node without server encryption
configured causes java.io.IOException accessing missing keystore
(CASSANDRA-16144)
+ * Internode messaging catches OOMs and does not rethrow (CASSANDRA-15214)
+ * When a table attempts to clean up metrics, it was cleaning up all global
table metrics (CASSANDRA-16095)
+ * Bring back the accepted encryption protocols list as configurable option
(CASSANDRA-13325)
+ * DigestResolver.getData throws AssertionError since dataResponse is null
(CASSANDRA-16097)
+Merged from 3.11:
* SASI's `max_compaction_flush_memory_in_mb` settings over 100GB revert to
default of 1GB (CASSANDRA-16071)
Merged from 3.0:
+ * Extend the exclusion of replica filtering protection to other indices
instead of just SASI (CASSANDRA-16311)
* Synchronize transaction logs for JBOD (CASSANDRA-16225)
* Fix the counting of cells per partition (CASSANDRA-16259)
* Fix serial read/non-applying CAS linearizability (CASSANDRA-12126)
diff --cc src/java/org/apache/cassandra/index/sasi/SASIIndex.java
index 592499e,5ea7cec..b1998bc
--- a/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
+++ b/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
@@@ -249,7 -244,13 +249,13 @@@ public class SASIIndex implements Index
public void validate(PartitionUpdate update) throws
InvalidRequestException
{}
+ @Override
+ public boolean supportsReplicaFilteringProtection(RowFilter rowFilter)
+ {
+ return false;
+ }
+
-public Indexer indexerFor(DecoratedKey key, PartitionColumns columns, int
nowInSec, OpOrder.Group opGroup, IndexTransaction.Type transactionType)
+public Indexer indexerFor(DecoratedKey key, RegularAndStaticColumns
columns, int nowInSec, WriteContext context, IndexTransaction.Type
transactionType)
{
return new Indexer()
{
diff --cc src/java/org/apache/cassandra/service/reads/DataResolver.java
index eeabb4b,000..7c76336
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/service/reads/DataResolver.java
+++ b/src/java/org/apache/cassandra/service/reads/DataResolver.java
@@@ -1,404 -1,0 +1,412 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
[cassandra] branch cassandra-3.11 updated (d8a317f -> ae89812)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from d8a317f Merge branch 'cassandra-3.0' into cassandra-3.11 new fa77676 Extend the exclusion of replica filtering protection to other indices instead of just SASI new ae89812 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/index/Index.java | 17 + .../org/apache/cassandra/index/sasi/SASIIndex.java | 6 ++ .../org/apache/cassandra/service/DataResolver.java | 21 ++--- 4 files changed, 38 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (1193180 -> 8e61e21)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 1193180 Merge branch 'cassandra-3.11' into trunk new fa77676 Extend the exclusion of replica filtering protection to other indices instead of just SASI new ae89812 Merge branch 'cassandra-3.0' into cassandra-3.11 new 8e61e21 Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + src/java/org/apache/cassandra/index/Index.java | 17 + .../org/apache/cassandra/index/sasi/SASIIndex.java | 6 ++ .../cassandra/service/reads/DataResolver.java | 22 +++--- 4 files changed, 39 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 1193180e54668fab6729bc2a483aa0e472b77b66 Merge: 19c153a d8a317f Author: Sam Tunnicliffe AuthorDate: Fri Dec 11 17:13:52 2020 + Merge branch 'cassandra-3.11' into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (19c153a -> 1193180)
This is an automated email from the ASF dual-hosted git repository. samt pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 19c153a Merge branch 'cassandra-3.11' into trunk new dc09f14 Ninja: fix docker image in config.yml new d8a317f Merge branch 'cassandra-3.0' into cassandra-3.11 new 1193180 Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (d297c78 -> d8a317f)
This is an automated email from the ASF dual-hosted git repository. samt pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from d297c78 Merge branch 'cassandra-3.0' into cassandra-3.11 new dc09f14 Ninja: fix docker image in config.yml new d8a317f Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .circleci/config.yml | 20 ++-- 1 file changed, 10 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit d8a317f463e0432966fd9670e008b3c8a05cff56
Merge: d297c78 dc09f14
Author: Sam Tunnicliffe
AuthorDate: Fri Dec 11 17:12:16 2020 +
Merge branch 'cassandra-3.0' into cassandra-3.11
.circleci/config.yml | 20 ++--
1 file changed, 10 insertions(+), 10 deletions(-)
diff --cc .circleci/config.yml
index c45f384,d49ace5..027ccb8
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@@ -330,55 -330,9 +330,55 @@@ jobs
- CCM_HEAP_NEWSIZE: 256M
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ utests_stress:
+docker:
- - image:
beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130
++- image:
beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130
+resource_class: medium
+working_directory: ~/
+shell: /bin/bash -eo pipefail -l
+parallelism: 1
+steps:
+- attach_workspace:
+at: /home/cassandra
+- run:
+name: Run Unit Tests (stress-test)
+command: |
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+ ant stress-test
-Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
-Dtest.classlistprefix=unit
+no_output_timeout: 15m
+- store_test_results:
+path: /tmp/cassandra/build/test/output/
+- store_artifacts:
+path: /tmp/cassandra/build/test/output
+destination: junitxml
+- store_artifacts:
+path: /tmp/cassandra/build/test/logs
+destination: logs
+environment:
+- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+- ANT_HOME: /usr/share/ant
+- LANG: en_US.UTF-8
+- KEEP_TEST_DIR: true
+- DEFAULT_DIR: /home/cassandra/cassandra-dtest
+- PYTHONIOENCODING: utf-8
+- PYTHONUNBUFFERED: true
+- CASS_DRIVER_NO_EXTENSIONS: true
+- CASS_DRIVER_NO_CYTHON: true
+- CASSANDRA_SKIP_SYNC: true
+- DTEST_REPO: git://github.com/apache/cassandra-dtest.git
+- DTEST_BRANCH: trunk
+- CCM_MAX_HEAP_SIZE: 1024M
+- CCM_HEAP_NEWSIZE: 256M
+- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
j8_unit_tests:
docker:
- - image:
beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130
+ - image:
beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130
resource_class: medium
working_directory: ~/
shell: /bin/bash -eo pipefail -l
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Ninja: fix docker image in config.yml
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-3.0 by this push: new dc09f14 Ninja: fix docker image in config.yml dc09f14 is described below commit dc09f1480df582051f42611cf49b0d6eb12cd70c Author: Sam Tunnicliffe AuthorDate: Fri Dec 11 17:11:43 2020 + Ninja: fix docker image in config.yml --- .circleci/config.yml | 20 ++-- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index dda7046..d49ace5 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -2,7 +2,7 @@ version: 2 jobs: j8_jvm_upgrade_dtests: docker: -- image: beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130 +- image: beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130 resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l @@ -94,7 +94,7 @@ jobs: - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64 build: docker: -- image: beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130 +- image: beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130 resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l @@ -175,7 +175,7 @@ jobs: - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64 j8_dtests-no-vnodes: docker: -- image: beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130 +- image: beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130 resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l @@ -233,7 +233,7 @@ jobs: - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64 j8_upgradetests-no-vnodes: docker: -- image: beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130 +- image: beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130 resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l @@ -332,7 +332,7 @@ jobs: - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64 j8_unit_tests: docker: -- image: beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130 +- image: beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130 resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l @@ -424,7 +424,7 @@ jobs: - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64 j8_dtests-with-vnodes: docker: -- image: beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130 +- image: beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130 resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l @@ -482,7 +482,7 @@ jobs: - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64 j8_jvm_dtests: docker: -- image: beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130 +- image: beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130 resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l @@ -574,7 +574,7 @@ jobs: - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64 utests_long: docker: -- image: beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130 +- image: beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130 resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l @@ -620,7 +620,7 @@ jobs: - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64 utests_compression: docker: -- image: beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130 +- image: beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130 resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l @@ -712,7 +712,7 @@ jobs: - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64 j8_dtest_jars_build: docker: -- image: beobal/cassandra-testing-ubuntu1810-java11-w-dependencies:20201130 +- image: beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130 resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16312) Can't build Cassandra 3.11 from source
[
https://issues.apache.org/jira/browse/CASSANDRA-16312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248018#comment-17248018
]
ab commented on CASSANDRA-16312:
I hit a similar problem. I was building with my system's default java 11. i
was able to build after switching my $JAVA_HOME to point at a java 8
installation.
> Can't build Cassandra 3.11 from source
> --
>
> Key: CASSANDRA-16312
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16312
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Vadim Kornev
>Priority: Normal
>
> Version of Cassandra is 3.11.9/3.11.10
> {code:bash}
> java -version
> openjdk version "1.8.0_275"
> OpenJDK Runtime Environment (build 1.8.0_275-b01)
> OpenJDK 64-Bit Server VM (build 25.275-b01, mixed mode)
> {code}
> {code:bash}
> Buildfile: .../cassandra/build.xml
>[script] Warning: Nashorn engine is planned to be removed from a future
> JDK release
> init:
> maven-ant-tasks-localrepo:
> maven-ant-tasks-download:
> maven-ant-tasks-init:
> maven-declare-dependencies:
> maven-ant-tasks-retrieve-build:
> init-dependencies:
> [echo] Loading dependency paths from file:
> .../cassandra/build/build-dependencies.xml
> init-dependencies:
> [echo] Loading dependency paths from file:
> .../cassandra/build/build-dependencies-sources.xml
> [unzip] Expanding:
> .../cassandra/build/lib/jars/org.jacoco.agent-0.7.5.201505241946.jar into
> .../cassandra/build/lib/jars
> check-gen-cql3-grammar:
> gen-cql3-grammar:
> generate-cql-html:
> generate-jflex-java:
> rat-init:
> rat-report:
> build-project:
> [echo] apache-cassandra: .../cassandra/build.xml
> [javac] Compiling 2 source files to .../build/classes/main
> [javac]
> .../cassandra/src/java/org/apache/cassandra/utils/Throwables.java:80: error:
> unreported exception Exception; must be caught or declared to be thrown
> [javac] perform(Stream.of(actions));
> [javac]^
> [javac]
> .../cassandra/src/java/org/apache/cassandra/utils/concurrent/Locks.java:46:
> error: cannot find symbol
> [javac] unsafe.monitorEnter(object);
> [javac] ^
> [javac] symbol: method monitorEnter(Object)
> [javac] location: variable unsafe of type Unsafe
> [javac]
> .../cassandra/src/java/org/apache/cassandra/utils/concurrent/Locks.java:52:
> error: cannot find symbol
> [javac] unsafe.monitorExit(object);
> [javac] ^
> [javac] symbol: method monitorExit(Object)
> [javac] location: variable unsafe of type Unsafe
> [javac] Note:
> .../cassandra/src/java/org/apache/cassandra/utils/Throwables.java uses
> unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 3 errors
> BUILD FAILED
> .../cassandra/build.xml:863: Compile failed; see the compiler error output
> for details.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248009#comment-17248009 ] Ekaterina Dimitrova edited comment on CASSANDRA-15897 at 12/11/20, 4:26 PM: First pass of review done. I don't see any issues with the patch itself, fixed formatting at a few places. Latest state of the branch [here|https://github.com/ekaterinadimitrova2/cassandra/pull/79] The failing tests are test issues: _TestGossipingPropertyFileSnitch_ - it required [update|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] of the VersionedValue in the assertions as the patch introduces a [new application state update|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-9bf2c26bc294ef9085e16bf287490223665eaa2eb8ec24bcf5bd8653c713644bR820]. _testDropSSTables_ - it turned out the previous commit I added is actually the [fix|https://github.com/ekaterinadimitrova2/cassandra/pull/79/commits/dba4a86306bc13f1b421c500570deb3a3f06414f]. As part of this patch a notification for added/loaded SSTables on start was added. This was done in order to accommodate this change and ensure no repetitive actions [here|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-57723368d79933b3d7fcc00ce9d3b98335391901c88cf09176d063fd6eb10ba3R249]. _CompactStorage2to3UpgradeTest_ - I am a bit puzzled about this one. It fails by complaining that the nodes do not have this patch. At the same time, I used ccm locally to run through the test scenario and everything works smoothly so I tend to believe I miss something from the nature of the jvm-upgrade tests While I am fixing the last mentioned test, I think this patch is ready for second review. [~ifesdjeen], [~aleksey], [~marcuse] is anyone of you available? Or if [~slebresne] can confirm my corrections/observations? [CI run|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/535/workflows/62835302-6aeb-4a47-b009-c706ced77869] [3.0 patch|https://github.com/ekaterinadimitrova2/cassandra/pull/79] [DTests patch|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] *NOTE:* test_prefer_local_reconnect_on_listen_address failed in circleci as I forgot to change the dtest repo but the test succeeds locally with the proposed dtest patch was (Author: e.dimitrova): First pass of review done. I don't see any issues with the patch itself, fixed formatting at a few places. Latest state of the branch [here|https://github.com/ekaterinadimitrova2/cassandra/pull/79] The failing tests are test issues: _TestGossipingPropertyFileSnitch_ - it required [update|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] of the VersionedValue in the assertions as the patch introduces a [new application state update|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-9bf2c26bc294ef9085e16bf287490223665eaa2eb8ec24bcf5bd8653c713644bR820]. _testDropSSTables_ - it turned out the previous commit I added is actually the [fix|https://github.com/ekaterinadimitrova2/cassandra/pull/79/commits/dba4a86306bc13f1b421c500570deb3a3f06414f]. As part of this patch a notification for added/loaded SSTables on start was added. This was done in order to accommodate this change and ensure no repetitive actions [here|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-57723368d79933b3d7fcc00ce9d3b98335391901c88cf09176d063fd6eb10ba3R249]. _CompactStorage2to3UpgradeTest_ - I am a bit puzzled about this one. It fails by complaining that the nodes do not have this patch. At the same time, I used ccm locally to run through the test scenario and everything works smoothly so I tend to believe I miss something from the nature of the jvm-upgrade tests While I am fixing the last mentioned test, I think this patch is ready for second review. [~ifesdjeen], [~aleksey], [~marcuse] is anyone of you available? Or if [~slebresne] can confirm my corrections/observations? [CI run|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/535/workflows/62835302-6aeb-4a47-b009-c706ced77869] [3.0 patch|https://github.com/ekaterinadimitrova2/cassandra/pull/79] [DTests patch|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] *NOTE:* test_prefer_local_reconnect_on_listen_address failed in circleci as I forgot to change the dtest repo but the test succeeds locally > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL:
[jira] [Comment Edited] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248009#comment-17248009 ] Ekaterina Dimitrova edited comment on CASSANDRA-15897 at 12/11/20, 4:25 PM: First pass of review done. I don't see any issues with the patch itself, fixed formatting at a few places. Latest state of the branch [here|https://github.com/ekaterinadimitrova2/cassandra/pull/79] The failing tests are test issues: _TestGossipingPropertyFileSnitch_ - it required [update|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] of the VersionedValue in the assertions as the patch introduces a [new application state update|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-9bf2c26bc294ef9085e16bf287490223665eaa2eb8ec24bcf5bd8653c713644bR820]. _testDropSSTables_ - it turned out the previous commit I added is actually the [fix|https://github.com/ekaterinadimitrova2/cassandra/pull/79/commits/dba4a86306bc13f1b421c500570deb3a3f06414f]. As part of this patch a notification for added/loaded SSTables on start was added. This was done in order to accommodate this change and ensure no repetitive actions [here|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-57723368d79933b3d7fcc00ce9d3b98335391901c88cf09176d063fd6eb10ba3R249]. _CompactStorage2to3UpgradeTest_ - I am a bit puzzled about this one. It fails by complaining that the nodes do not have this patch. At the same time, I used ccm locally to run through the test scenario and everything works smoothly so I tend to believe I miss something from the nature of the jvm-upgrade tests While I am fixing the last mentioned test, I think this patch is ready for second review. [~ifesdjeen], [~aleksey], [~marcuse] is anyone of you available? Or if [~slebresne] can confirm my corrections/observations? [CI run|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/535/workflows/62835302-6aeb-4a47-b009-c706ced77869] [3.0 patch|https://github.com/ekaterinadimitrova2/cassandra/pull/79] [DTests patch|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] *NOTE:* test_prefer_local_reconnect_on_listen_address failed in circleci as I forgot to change the dtest repo but the test succeeds locally was (Author: e.dimitrova): First pass of review done. I don't see any issues with the patch itself, fixed formatting at a few places. Latest state of the branch [here|https://github.com/ekaterinadimitrova2/cassandra/pull/79] The failing tests are test issues: _TestGossipingPropertyFileSnitch_ - it required [update|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] of the VersionedValue in the assertions as the patch introduces a [new application state update|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-9bf2c26bc294ef9085e16bf287490223665eaa2eb8ec24bcf5bd8653c713644bR820]. _testDropSSTables_ - it turned out the previous commit I added is actually the [fix|https://github.com/ekaterinadimitrova2/cassandra/pull/79/commits/dba4a86306bc13f1b421c500570deb3a3f06414f]. As part of this patch a notification for added/loaded SSTables on start was added. This was done in order to accommodate this change and ensure no repetitive actions [here|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-57723368d79933b3d7fcc00ce9d3b98335391901c88cf09176d063fd6eb10ba3R249]. _CompactStorage2to3UpgradeTest_ - I am a bit puzzled about this one. It fails by complaining that the nodes do not have this patch. At the same time, I used ccm locally to run through the test scenario and everything works smoothly so I tend to believe I miss something from the nature of the jvm-upgrade tests While I am fixing the last mentioned test, I think this patch is ready for second review. [~ifesdjeen], [~aleksey], [~marcuse] is anyone of you available? Or if [~slebresne] can confirm my corrections/observations? [CI run|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/535/workflows/62835302-6aeb-4a47-b009-c706ced77869] [3.0 patch|https://github.com/ekaterinadimitrova2/cassandra/pull/79] [DTests patch|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus
[jira] [Comment Edited] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248009#comment-17248009 ] Ekaterina Dimitrova edited comment on CASSANDRA-15897 at 12/11/20, 4:24 PM: First pass of review done. I don't see any issues with the patch itself, fixed formatting at a few places. Latest state of the branch [here|https://github.com/ekaterinadimitrova2/cassandra/pull/79] The failing tests are test issues: _TestGossipingPropertyFileSnitch_ - it required [update|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] of the VersionedValue in the assertions as the patch introduces a [new application state update|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-9bf2c26bc294ef9085e16bf287490223665eaa2eb8ec24bcf5bd8653c713644bR820]. _testDropSSTables_ - it turned out the previous commit I added is actually the [fix|https://github.com/ekaterinadimitrova2/cassandra/pull/79/commits/dba4a86306bc13f1b421c500570deb3a3f06414f]. As part of this patch a notification for added/loaded SSTables on start was added. This was done in order to accommodate this change and ensure no repetitive actions [here|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-57723368d79933b3d7fcc00ce9d3b98335391901c88cf09176d063fd6eb10ba3R249]. _CompactStorage2to3UpgradeTest_ - I am a bit puzzled about this one. It fails by complaining that the nodes do not have this patch. At the same time, I used ccm locally to run through the test scenario and everything works smoothly so I tend to believe I miss something from the nature of the jvm-upgrade tests While I am fixing the last mentioned test, I think this patch is ready for second review. [~ifesdjeen], [~aleksey], [~marcuse] is anyone of you available? Or if [~slebresne] can confirm my corrections/observations? [CI run|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/535/workflows/62835302-6aeb-4a47-b009-c706ced77869] [3.0 patch|https://github.com/ekaterinadimitrova2/cassandra/pull/79] [DTests patch|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] was (Author: e.dimitrova): First pass of review done. I don't see any issues with the patch itself, fixed formatting at a few places. Latest state of the branch [here|https://github.com/ekaterinadimitrova2/cassandra/pull/79] The failing tests are test issues: _TestGossipingPropertyFileSnitch_ - it required [update|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] of the VersionedValue in the assertions as the patch introduces a [new application state update|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-9bf2c26bc294ef9085e16bf287490223665eaa2eb8ec24bcf5bd8653c713644bR820]. _testDropSSTables_ - it turned out the previous commit I added is actually the [fix|https://github.com/ekaterinadimitrova2/cassandra/pull/79/commits/dba4a86306bc13f1b421c500570deb3a3f06414f]. As part of this patch a notification for added/loaded SSTables on start was added. This was done in order to accommodate this change and ensure no repetitive actions [here|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-57723368d79933b3d7fcc00ce9d3b98335391901c88cf09176d063fd6eb10ba3R249]. _CompactStorage2to3UpgradeTest_ - I am a bit puzzled about this one. It fails by complaining that the nodes do not have this patch. At the same time, I used ccm locally to play the test scenario and everything works smoothly so I tend to believe I miss something from the nature of the jvm-upgrade tests While I am fixing the last mentioned test, I think this patch is ready for second review. [~ifesdjeen], [~aleksey], [~marcuse] is anyone of you available? Or if [~slebresne] can confirm my corrections/observations? [CI run|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/535/workflows/62835302-6aeb-4a47-b009-c706ced77869] [3.0 patch|https://github.com/ekaterinadimitrova2/cassandra/pull/79] [DTests patch|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 4.0-beta > > > Test reproducing: >
[jira] [Updated] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15897: Reviewers: Ekaterina Dimitrova, Ekaterina Dimitrova (was: Ekaterina Dimitrova) Ekaterina Dimitrova, Ekaterina Dimitrova Status: Review In Progress (was: Patch Available) > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 4.0-beta > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248009#comment-17248009 ] Ekaterina Dimitrova edited comment on CASSANDRA-15897 at 12/11/20, 4:23 PM: First pass of review done. I don't see any issues with the patch itself, fixed formatting at a few places. Latest state of the branch [here|https://github.com/ekaterinadimitrova2/cassandra/pull/79] The failing tests are test issues: _TestGossipingPropertyFileSnitch_ - it required [update|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] of the VersionedValue in the assertions as the patch introduces a [new application state update|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-9bf2c26bc294ef9085e16bf287490223665eaa2eb8ec24bcf5bd8653c713644bR820]. _testDropSSTables_ - it turned out the previous commit I added is actually the [fix|https://github.com/ekaterinadimitrova2/cassandra/pull/79/commits/dba4a86306bc13f1b421c500570deb3a3f06414f]. As part of this patch a notification for added/loaded SSTables on start was added. This was done in order to accommodate this change and ensure no repetitive actions [here|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-57723368d79933b3d7fcc00ce9d3b98335391901c88cf09176d063fd6eb10ba3R249]. _CompactStorage2to3UpgradeTest_ - I am a bit puzzled about this one. It fails by complaining that the nodes do not have this patch. At the same time, I used ccm locally to play the test scenario and everything works smoothly so I tend to believe I miss something from the nature of the jvm-upgrade tests While I am fixing the last mentioned test, I think this patch is ready for second review. [~ifesdjeen], [~aleksey], [~marcuse] is anyone of you available? Or if [~slebresne] can confirm my corrections/observations? [CI run|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/535/workflows/62835302-6aeb-4a47-b009-c706ced77869] [3.0 patch|https://github.com/ekaterinadimitrova2/cassandra/pull/79] [DTests patch|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] was (Author: e.dimitrova): First pass of review done. I don't see any issues with the patch itself, fixed formatting at a few places. Latest state of the branch [here|https://github.com/ekaterinadimitrova2/cassandra/pull/79] The failing tests are test issues: _TestGossipingPropertyFileSnitch_ - it required [update|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] of the VersionedValue in the assertions as the patch introduces a [new application state update|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-9bf2c26bc294ef9085e16bf287490223665eaa2eb8ec24bcf5bd8653c713644bR820]. _testDropSSTables_ - it turned out the previous commit I added is actually the [fix|https://github.com/ekaterinadimitrova2/cassandra/pull/79/commits/dba4a86306bc13f1b421c500570deb3a3f06414f]. As part of this patch a notification for added/loaded SSTables on start was added. This was done in order to accommodate this change and ensure no repetitive actions [here|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-57723368d79933b3d7fcc00ce9d3b98335391901c88cf09176d063fd6eb10ba3R249]. _CompactStorage2to3UpgradeTest_ - I am a bit puzzled about this one. It fails by complaining that the nodes do not have this patch. At the same time, I used ccm locally to play the test scenario and everything works smoothly so I tend to believe I miss something from the nature of the jvm-upgrade tests While I am fixing the last mentioned test, I think this patch is ready for second review. [~ifesdjeen], [~aleksey], is anyone of you available? Or if [~slebresne] can confirm my corrections/observations? [CI run|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/535/workflows/62835302-6aeb-4a47-b009-c706ced77869] [3.0 patch|https://github.com/ekaterinadimitrova2/cassandra/pull/79] [DTests patch|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 4.0-beta > > > Test reproducing: >
[jira] [Commented] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248009#comment-17248009 ] Ekaterina Dimitrova commented on CASSANDRA-15897: - First pass of review done. I don't see any issues with the patch itself, fixed formatting at a few places. Latest state of the branch [here|https://github.com/ekaterinadimitrova2/cassandra/pull/79] The failing tests are test issues: _TestGossipingPropertyFileSnitch_ - it required [update|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] of the VersionedValue in the assertions as the patch introduces a [new application state update|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-9bf2c26bc294ef9085e16bf287490223665eaa2eb8ec24bcf5bd8653c713644bR820]. _testDropSSTables_ - it turned out the previous commit I added is actually the [fix|https://github.com/ekaterinadimitrova2/cassandra/pull/79/commits/dba4a86306bc13f1b421c500570deb3a3f06414f]. As part of this patch a notification for added/loaded SSTables on start was added. This was done in order to accommodate this change and ensure no repetitive actions [here|https://github.com/pcmanus/cassandra/commit/2dd0d1cdbd2c0869bfb8e22be34e0d67a79a8742#diff-57723368d79933b3d7fcc00ce9d3b98335391901c88cf09176d063fd6eb10ba3R249]. _CompactStorage2to3UpgradeTest_ - I am a bit puzzled about this one. It fails by complaining that the nodes do not have this patch. At the same time, I used ccm locally to play the test scenario and everything works smoothly so I tend to believe I miss something from the nature of the jvm-upgrade tests While I am fixing the last mentioned test, I think this patch is ready for second review. [~ifesdjeen], [~aleksey], is anyone of you available? Or if [~slebresne] can confirm my corrections/observations? [CI run|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/535/workflows/62835302-6aeb-4a47-b009-c706ced77869] [3.0 patch|https://github.com/ekaterinadimitrova2/cassandra/pull/79] [DTests patch|https://github.com/ekaterinadimitrova2/cassandra-dtest/commit/927a07d0cc9bfcc183d3a0e7c8959cd0c6c88c6a] > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 4.0-beta > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15897: Test and Documentation Plan: https://issues.apache.org/jira/browse/CASSANDRA-15897?focusedCommentId=17248009=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17248009 Status: Patch Available (was: In Progress) > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 4.0-beta > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16300) Cassandra 2.2 doesn't satisfy Java 1.7 language level
[ https://issues.apache.org/jira/browse/CASSANDRA-16300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-16300: -- Reviewers: Adam Holmberg, Brandon Williams, Ekaterina Dimitrova (was: Ekaterina Dimitrova) > Cassandra 2.2 doesn't satisfy Java 1.7 language level > - > > Key: CASSANDRA-16300 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16300 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 2.2.x > > Attachments: junitreport.zip > > Time Spent: 10m > Remaining Estimate: 0h > > I think that 2.2 needs to satisfy Java 1.7 language level. However, there are > a few places where newer syntax is used: > * > [MessagingService|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/net/MessagingService.java#L779] > * > [ExecutorUtils|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/utils/ExecutorUtils.java#L36-L47] > * > [Throwables|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/utils/Throwables.java#L36] > * > [YamlConfigurationLoader|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L142] > * > [YamlConfigurationLoaderTest|https://github.com/apache/cassandra/blob/cassandra-2.2/test/unit/org/apache/cassandra/config/YamlConfigurationLoaderTest.java#L38] > If I'm right and we really need 1.7 language level we should fix those, which > seems quite straightforward. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16335) Expose data dirs in ColumnFamilyStoreMBean
[
https://issues.apache.org/jira/browse/CASSANDRA-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-16335:
--
Description:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
backup and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are put somewhere under path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /backups/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of them
is not used anymore and I do not want to do something very smelly like listing
directories on disk and checking which one does not contain "dropped" directory
... Yes, one might use importing of SSTables - that is introduced in Cassandra
4, but for Cassandra 3, one can either copy it over or do hardlinks and refresh.
The second scenario is like this:
There is just one "active" table, no structure with "dropped" dir exists, but
its id (that part after table name) differs. If I want to copy files over and
refresh, I need to resolve this discrepancy and copy SSTables into a directory
ending on id which differs from id from backup.
I was trying to get this information from CFSMB but that information is not
exposed.
Is there any way how to retrieve via JMX where a table actually stores its data?
I have put this together: https://github.com/apache/cassandra/pull/850/files
was:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
backup and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /backups/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of them
is not used anymore and I do not want to do something very smelly like listing
directories on disk and checking which one does not contain "dropped" directory
... Yes, one might use importing of SSTables - that is introduced in Cassandra
4, but for Cassandra 3, one can either copy it over or do hardlinks and refresh.
The second
[jira] [Commented] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247972#comment-17247972
]
Alex Petrov commented on CASSANDRA-16226:
-
+1 on the patch. The only change I'd like to suggest is to add tests for
tombstones, since previously we weren't taking them into consideration.
Two nits (pre-existing):
*
[here|https://github.com/apache/cassandra/pull/823/files#diff-b4515f390c3b40e1f64feda8fd8746647b43cda26f8019ec96fc2dd4320fa96cR1040]
"more recent that" should be "more recent than".
* should we consider renaming {{canRemoveRow}}? The name is slightly
confusing, since the idea here is basically to reduce number of clustering keys
to perform search upon. I'm sure this made complete sense to whoever was
writing it at first, but out of context this name was a bit hard to parse.
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key
> liveness information for non-compact/CQL tables. Tombstones seem to be
> handled at a level above that anyway. (One potential problem with that is
> whether or not the distinction will continue to exist in 4.0, and dropping
> compact storage from a table doesn't magically make pk liveness information
> appear.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16335) Expose data dirs in ColumnFamilyStoreMBean
[
https://issues.apache.org/jira/browse/CASSANDRA-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-16335:
--
Priority: Low (was: Normal)
> Expose data dirs in ColumnFamilyStoreMBean
> ---
>
> Key: CASSANDRA-16335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16335
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Low
>
> As of now, I am not currently aware of any way how to get the information
> where a CF stores its data. While this might look like a detail, it is
> important for backup and restore purposes. Lets consider this workflow:
> 1) There is a keyspace "abc" with table "def", on disk, it will look like
> /my/data/abc/def-123445/...
> 2) I take a backup, all SSTables are restored somewhere under same path
> /backups/abc/def-12345/
> 3) I delete this table by CQL, data ends up in "dropped"
> 4) I create this table again, but now it will generate other ID - like
> /my/data/abc/def-6789/...
> 5) I want to restore /backups/abc/def-123445/... but right now there are two
> structures -
> {code:java}
> ├── data
> │ ├── abc
> │ │ ├── def-12345...
> │ │ │ ├── backups
> │ │ │ └── snapshots
> │ │ │ └── dropped-1607699318139-ghi
> │ │ │ ├── manifest.json
> │ │ │ ├── na-1-big-CompressionInfo.db
> │ │ │ ├── na-1-big-Data.db
> │ │ │ ├── na-1-big-Digest.crc32
> │ │ │ ├── na-1-big-Filter.db
> │ │ │ ├── na-1-big-Index.db
> │ │ │ ├── na-1-big-Statistics.db
> │ │ │ ├── na-1-big-Summary.db
> │ │ │ ├── na-1-big-TOC.txt
> │ │ │ └── schema.cql
> │ │ └── def-6789...
> │ │ ├── backups
> │ │ ├── na-1-big-CompressionInfo.db
> │ │ ├── na-1-big-Data.db
> │ │ ├── na-1-big-Digest.crc32
> │ │ ├── na-1-big-Filter.db
> │ │ ├── na-1-big-Index.db
> │ │ ├── na-1-big-Statistics.db
> │ │ ├── na-1-big-Summary.db
> │ │ └── na-1-big-TOC.txt
> {code}
> The question now is, what directory I should restore this to? Sure, into the
> "active" one, but I can not possibly know which one it is, because one of
> them is not used anymore and I do not want to do something very smelly like
> listing directories on disk and checking which one does not contain "dropped"
> directory ... Yes, one might use importing of SSTables - that is introduced
> in Cassandra 4, but for Cassandra 3, one can either copy it over or do
> hardlinks and refresh.
> The second scenario is like this:
> There is just one "active" table, no structure with "dropped" dir exists, but
> its id (that part after table name) differs. If I want to copy files over and
> refresh, I need to resolve this discrepancy and copy SSTables into a
> directory ending on id which differs from id from backup.
> I was trying to get this information from CFSMB but that information is not
> exposed.
> Is there any way how to retrieve via JMX where a table actually stores its
> data?
> I have put this together: https://github.com/apache/cassandra/pull/850/files
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16300) Cassandra 2.2 doesn't satisfy Java 1.7 language level
[ https://issues.apache.org/jira/browse/CASSANDRA-16300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-16300: -- Since Version: 2.2.19 Source Control Link: https://github.com/apache/cassandra/commit/20d1f9a69c18bfd11607063ec16fd24d1c835b55 Resolution: Fixed Status: Resolved (was: Ready to Commit) > Cassandra 2.2 doesn't satisfy Java 1.7 language level > - > > Key: CASSANDRA-16300 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16300 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 2.2.x > > Attachments: junitreport.zip > > Time Spent: 10m > Remaining Estimate: 0h > > I think that 2.2 needs to satisfy Java 1.7 language level. However, there are > a few places where newer syntax is used: > * > [MessagingService|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/net/MessagingService.java#L779] > * > [ExecutorUtils|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/utils/ExecutorUtils.java#L36-L47] > * > [Throwables|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/utils/Throwables.java#L36] > * > [YamlConfigurationLoader|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L142] > * > [YamlConfigurationLoaderTest|https://github.com/apache/cassandra/blob/cassandra-2.2/test/unit/org/apache/cassandra/config/YamlConfigurationLoaderTest.java#L38] > If I'm right and we really need 1.7 language level we should fix those, which > seems quite straightforward. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16300) Cassandra 2.2 doesn't satisfy Java 1.7 language level
[ https://issues.apache.org/jira/browse/CASSANDRA-16300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247969#comment-17247969 ] Andres de la Peña commented on CASSANDRA-16300: --- Thanks for the reviews. Committed to cassandra-2.2 as [20d1f9a69c18bfd11607063ec16fd24d1c835b55|https://github.com/apache/cassandra/commit/20d1f9a69c18bfd11607063ec16fd24d1c835b55] and merged up discarding the changes in more recent branches. > Cassandra 2.2 doesn't satisfy Java 1.7 language level > - > > Key: CASSANDRA-16300 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16300 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 2.2.x > > Attachments: junitreport.zip > > Time Spent: 10m > Remaining Estimate: 0h > > I think that 2.2 needs to satisfy Java 1.7 language level. However, there are > a few places where newer syntax is used: > * > [MessagingService|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/net/MessagingService.java#L779] > * > [ExecutorUtils|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/utils/ExecutorUtils.java#L36-L47] > * > [Throwables|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/utils/Throwables.java#L36] > * > [YamlConfigurationLoader|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L142] > * > [YamlConfigurationLoaderTest|https://github.com/apache/cassandra/blob/cassandra-2.2/test/unit/org/apache/cassandra/config/YamlConfigurationLoaderTest.java#L38] > If I'm right and we really need 1.7 language level we should fix those, which > seems quite straightforward. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit d297c78b958da694c6c82e953fafd121ada37765 Merge: 80a8deb ac7df51 Author: Andrés de la Peña AuthorDate: Fri Dec 11 15:25:28 2020 + Merge branch 'cassandra-3.0' into cassandra-3.11 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (80a8deb -> d297c78)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 80a8deb Merge branch 'cassandra-3.0' into cassandra-3.11 new 20d1f9a Satisfy Java 1.7 in sources new ac7df51 Merge branch 'cassandra-2.2' into cassandra-3.0 new d297c78 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 19c153a4e17d57f6da97ffa1cf2021d7b45ec03a Merge: ab9fcd7 d297c78 Author: Andrés de la Peña AuthorDate: Fri Dec 11 15:27:38 2020 + Merge branch 'cassandra-3.11' into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (ab9fcd7 -> 19c153a)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from ab9fcd7 Circle CI python upgrade test build includes non-upgrade tests new 20d1f9a Satisfy Java 1.7 in sources new ac7df51 Merge branch 'cassandra-2.2' into cassandra-3.0 new d297c78 Merge branch 'cassandra-3.0' into cassandra-3.11 new 19c153a Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (2bcbd92 -> ac7df51)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 2bcbd92 Ninja fix CASSANDRA-16225 and update changes. new 20d1f9a Satisfy Java 1.7 in sources new ac7df51 Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit ac7df513ac63b8699382f59fa86f5fc72f6779db Merge: 2bcbd92 20d1f9a Author: Andrés de la Peña AuthorDate: Fri Dec 11 15:25:06 2020 + Merge branch 'cassandra-2.2' into cassandra-3.0 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Satisfy Java 1.7 in sources
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new 20d1f9a Satisfy Java 1.7 in sources
20d1f9a is described below
commit 20d1f9a69c18bfd11607063ec16fd24d1c835b55
Author: Andrés de la Peña
AuthorDate: Fri Dec 11 15:15:49 2020 +
Satisfy Java 1.7 in sources
patch by Andrés de la Peña; reviewed by Ekaterina Dimitrova and Brandon
Williams for CASSANDRA-16300
---
.../cassandra/config/YamlConfigurationLoader.java | 2 +-
.../org/apache/cassandra/net/MessagingService.java | 3 ++-
.../org/apache/cassandra/utils/ExecutorUtils.java | 24 +-
.../org/apache/cassandra/utils/Throwables.java | 13 +---
.../config/YamlConfigurationLoaderTest.java| 3 ++-
5 files changed, 29 insertions(+), 16 deletions(-)
diff --git a/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
b/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
index 49418ac..9c066c4 100644
--- a/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
+++ b/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
@@ -139,7 +139,7 @@ public class YamlConfigurationLoader implements
ConfigurationLoader
YamlConfigurationLoader.MissingPropertiesChecker propertiesChecker =
new YamlConfigurationLoader.MissingPropertiesChecker();
constructor.setPropertyUtils(propertiesChecker);
Yaml yaml = new Yaml(constructor);
-Node node = yaml.represent(map);
+final Node node = yaml.represent(map);
constructor.setComposer(new Composer(null, null)
{
@Override
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java
b/src/java/org/apache/cassandra/net/MessagingService.java
index f125b09..4139a89 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -776,7 +776,8 @@ public final class MessagingService implements
MessagingServiceMBean
handleIOException(e);
}
-
connectionManagers.values().forEach(OutboundTcpConnectionPool::close);
+for (OutboundTcpConnectionPool pool : connectionManagers.values())
+pool.close();
}
catch (IOException e)
{
diff --git a/src/java/org/apache/cassandra/utils/ExecutorUtils.java
b/src/java/org/apache/cassandra/utils/ExecutorUtils.java
index 21933a3..78f4e89 100644
--- a/src/java/org/apache/cassandra/utils/ExecutorUtils.java
+++ b/src/java/org/apache/cassandra/utils/ExecutorUtils.java
@@ -31,18 +31,22 @@ import static java.util.concurrent.TimeUnit.NANOSECONDS;
public class ExecutorUtils
{
-public static Runnable runWithThreadName(Runnable runnable, String
threadName)
+public static Runnable runWithThreadName(final Runnable runnable, final
String threadName)
{
-return () -> {
-String oldThreadName = Thread.currentThread().getName();
-try
-{
-Thread.currentThread().setName(threadName);
-runnable.run();
-}
-finally
+return new Runnable()
+{
+public void run()
{
-Thread.currentThread().setName(oldThreadName);
+String oldThreadName = Thread.currentThread().getName();
+try
+{
+Thread.currentThread().setName(threadName);
+runnable.run();
+}
+finally
+{
+Thread.currentThread().setName(oldThreadName);
+}
}
};
}
diff --git a/src/java/org/apache/cassandra/utils/Throwables.java
b/src/java/org/apache/cassandra/utils/Throwables.java
index 82703c8..816255a 100644
--- a/src/java/org/apache/cassandra/utils/Throwables.java
+++ b/src/java/org/apache/cassandra/utils/Throwables.java
@@ -35,7 +35,7 @@ public class Throwables
public static void maybeFail(Throwable fail)
{
-if (failIfCanCast(fail, null))
+if (failIfCanCast(fail))
throw new RuntimeException(fail);
}
@@ -45,7 +45,7 @@ public class Throwables
throw new RuntimeException(fail);
}
-public static boolean failIfCanCast(Throwable fail,
Class checked) throws T
+public static boolean failIfCanCast(Throwable fail)
{
if (fail == null)
return false;
@@ -56,10 +56,17 @@ public class Throwables
if (fail instanceof RuntimeException)
throw (RuntimeException) fail;
+return true;
+}
+
+public static boolean failIfCanCast(Throwable fail,
Class checked) throws T
+{
+boolean result = failIfCanCast(fail);
+
[jira] [Updated] (CASSANDRA-16335) Expose data dirs in ColumnFamilyStoreMBean
[
https://issues.apache.org/jira/browse/CASSANDRA-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-16335:
--
Description:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
backup and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /backups/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of them
is not used anymore and I do not want to do something very smelly like listing
directories on disk and checking which one does not contain "dropped" directory
... Yes, one might use importing of SSTables - that is introduced in Cassandra
4, but for Cassandra 3, one can either copy it over or do hardlinks and refresh.
The second scenario is like this:
There is just one "active" table, no structure with "dropped" dir exists, but
its id (that part after table name) differs. If I want to copy files over and
refresh, I need to resolve this discrepancy and copy SSTables into a directory
ending on id which differs from id from backup.
I was trying to get this information from CFSMB but that information is not
exposed.
Is there any way how to retrieve via JMX where a table actually stores its data?
I have put this together: https://github.com/apache/cassandra/pull/850/files
was:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
backup and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /backups/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of the
is not used anymore.
The second scenario is like this:
There is just one "active" table, no structure with "dropped" dir exists, but
its id (that part after table name) differs. If I want to copy files over and
refresh, I need to resolve this discrepancy and copy SSTables into a directory
ending on id which differs from id
[jira] [Updated] (CASSANDRA-16335) Expose data dirs in ColumnFamilyStoreMBean
[
https://issues.apache.org/jira/browse/CASSANDRA-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-16335:
--
Description:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
backup and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /backups/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of the
is not used anymore.
The second scenario is like this:
There is just one "active" table, no structure with "dropped" dir exists, but
its id (that part after table name) differs. If I want to copy files over and
refresh, I need to resolve this discrepancy and copy SSTables into a directory
ending on id which differs from id from backup.
I was trying to get this information from CFSMB but that information is not
exposed.
Is there any way how to retrieve via JMX where a table actually stores its data?
I have put this together: https://github.com/apache/cassandra/pull/850/files
was:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
backup and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /backups/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of the
is not used anymore.
I was trying to get this information from CFSMB but that information is not
exposed.
Is there any way how to retrieve via JMX where a table actually stores its data?
I have put this together: https://github.com/apache/cassandra/pull/850/files
> Expose data dirs in ColumnFamilyStoreMBean
> ---
>
> Key: CASSANDRA-16335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16335
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>
[jira] [Updated] (CASSANDRA-16335) Expose data dirs in ColumnFamilyStoreMBean
[
https://issues.apache.org/jira/browse/CASSANDRA-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-16335:
--
Description:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
backup and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /backups/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of the
is not used anymore.
I was trying to get this information from CFSMB but that information is not
exposed.
Is there any way how to retrieve via JMX where a table actually stores its data?
I have put this together: https://github.com/apache/cassandra/pull/850/files
was:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
backup and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /my/data/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of the
is not used anymore.
I was trying to get this information from CFSMB but that information is not
exposed.
Is there any way how to retrieve via JMX where a table actually stores its data?
I have put this together: https://github.com/apache/cassandra/pull/850/files
> Expose data dirs in ColumnFamilyStoreMBean
> ---
>
> Key: CASSANDRA-16335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16335
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>
> As of now, I am not currently aware of any way how to get the information
> where a CF stores its data. While this might look like a detail, it is
> important for backup and restore purposes. Lets consider this workflow:
> 1) There is a keyspace "abc" with table "def", on disk, it will
[jira] [Updated] (CASSANDRA-16335) Expose data dirs in ColumnFamilyStoreMBean
[
https://issues.apache.org/jira/browse/CASSANDRA-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-16335:
--
Description:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
back and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /my/data/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of the
is not used anymore.
I was trying to get this information from CFSMB but that information is not
exposed.
Is there any way how to retrieve via JMX where a table actually stores its data?
I have put this together: https://github.com/apache/cassandra/pull/850/files
was:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
back and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /my/data/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of the
is not used anymore.
I was trying to get this information from CFSMB but that information is not
exposed.
Is there any way how to retrieve via JMX where a table actually stores its data?
> Expose data dirs in ColumnFamilyStoreMBean
> ---
>
> Key: CASSANDRA-16335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16335
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>
> As of now, I am not currently aware of any way how to get the information
> where a CF stores its data. While this might look like a detail, it is
> important for back and restore purposes. Lets consider this workflow:
> 1) There is a keyspace "abc" with table "def", on disk, it will look like
> /my/data/abc/def-123445/...
> 2) I take a backup, all SSTables are
[jira] [Updated] (CASSANDRA-16335) Expose data dirs in ColumnFamilyStoreMBean
[
https://issues.apache.org/jira/browse/CASSANDRA-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-16335:
--
Description:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
backup and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /my/data/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of the
is not used anymore.
I was trying to get this information from CFSMB but that information is not
exposed.
Is there any way how to retrieve via JMX where a table actually stores its data?
I have put this together: https://github.com/apache/cassandra/pull/850/files
was:
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
back and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /my/data/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of the
is not used anymore.
I was trying to get this information from CFSMB but that information is not
exposed.
Is there any way how to retrieve via JMX where a table actually stores its data?
I have put this together: https://github.com/apache/cassandra/pull/850/files
> Expose data dirs in ColumnFamilyStoreMBean
> ---
>
> Key: CASSANDRA-16335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16335
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>
> As of now, I am not currently aware of any way how to get the information
> where a CF stores its data. While this might look like a detail, it is
> important for backup and restore purposes. Lets consider this workflow:
> 1) There is a keyspace "abc" with table "def", on disk, it will look
[jira] [Created] (CASSANDRA-16335) Expose data dirs in ColumnFamilyStoreMBean
Stefan Miklosovic created CASSANDRA-16335:
-
Summary: Expose data dirs in ColumnFamilyStoreMBean
Key: CASSANDRA-16335
URL: https://issues.apache.org/jira/browse/CASSANDRA-16335
Project: Cassandra
Issue Type: Improvement
Reporter: Stefan Miklosovic
Assignee: Stefan Miklosovic
As of now, I am not currently aware of any way how to get the information where
a CF stores its data. While this might look like a detail, it is important for
back and restore purposes. Lets consider this workflow:
1) There is a keyspace "abc" with table "def", on disk, it will look like
/my/data/abc/def-123445/...
2) I take a backup, all SSTables are restored somewhere under same path
/backups/abc/def-12345/
3) I delete this table by CQL, data ends up in "dropped"
4) I create this table again, but now it will generate other ID - like
/my/data/abc/def-6789/...
5) I want to restore /my/data/abc/def-123445/... but right now there are two
structures -
{code:java}
├── data
│ ├── abc
│ │ ├── def-12345...
│ │ │ ├── backups
│ │ │ └── snapshots
│ │ │ └── dropped-1607699318139-ghi
│ │ │ ├── manifest.json
│ │ │ ├── na-1-big-CompressionInfo.db
│ │ │ ├── na-1-big-Data.db
│ │ │ ├── na-1-big-Digest.crc32
│ │ │ ├── na-1-big-Filter.db
│ │ │ ├── na-1-big-Index.db
│ │ │ ├── na-1-big-Statistics.db
│ │ │ ├── na-1-big-Summary.db
│ │ │ ├── na-1-big-TOC.txt
│ │ │ └── schema.cql
│ │ └── def-6789...
│ │ ├── backups
│ │ ├── na-1-big-CompressionInfo.db
│ │ ├── na-1-big-Data.db
│ │ ├── na-1-big-Digest.crc32
│ │ ├── na-1-big-Filter.db
│ │ ├── na-1-big-Index.db
│ │ ├── na-1-big-Statistics.db
│ │ ├── na-1-big-Summary.db
│ │ └── na-1-big-TOC.txt
{code}
The question now is, what directory I should restore this to? Sure, into the
"active" one, but I can not possibly know which one it is, because one of the
is not used anymore.
I was trying to get this information from CFSMB but that information is not
exposed.
Is there any way how to retrieve via JMX where a table actually stores its data?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16300) Cassandra 2.2 doesn't satisfy Java 1.7 language level
[ https://issues.apache.org/jira/browse/CASSANDRA-16300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-16300: -- Status: Ready to Commit (was: Review In Progress) > Cassandra 2.2 doesn't satisfy Java 1.7 language level > - > > Key: CASSANDRA-16300 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16300 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 2.2.x > > Attachments: junitreport.zip > > Time Spent: 10m > Remaining Estimate: 0h > > I think that 2.2 needs to satisfy Java 1.7 language level. However, there are > a few places where newer syntax is used: > * > [MessagingService|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/net/MessagingService.java#L779] > * > [ExecutorUtils|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/utils/ExecutorUtils.java#L36-L47] > * > [Throwables|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/utils/Throwables.java#L36] > * > [YamlConfigurationLoader|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L142] > * > [YamlConfigurationLoaderTest|https://github.com/apache/cassandra/blob/cassandra-2.2/test/unit/org/apache/cassandra/config/YamlConfigurationLoaderTest.java#L38] > If I'm right and we really need 1.7 language level we should fix those, which > seems quite straightforward. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16311) Extend the exclusion of replica filtering protection to other indices instead of just SASI
[ https://issues.apache.org/jira/browse/CASSANDRA-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247946#comment-17247946 ] Stefan Miklosovic commented on CASSANDRA-16311: --- Please proceed. Thanks. > Extend the exclusion of replica filtering protection to other indices instead > of just SASI > -- > > Key: CASSANDRA-16311 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16311 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SASI >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > There was a check introduced in CASSANDRA-8272 telling if an index is a SASI > index and if it is, replica filtering protection was not triggered. > There might be other custom index implementations which also do not support > filtering protection and they do not have to be SASI indices neccessarily, > however it is not possible to exclude them. > > PR trunk > [https://github.com/apache/cassandra/pull/844] > PR 3.11 > [https://github.com/apache/cassandra/pull/847] > PR 3.0 > https://github.com/apache/cassandra/pull/848 > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16300) Cassandra 2.2 doesn't satisfy Java 1.7 language level
[ https://issues.apache.org/jira/browse/CASSANDRA-16300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247939#comment-17247939 ] Brandon Williams commented on CASSANDRA-16300: -- LGTM +1 > Cassandra 2.2 doesn't satisfy Java 1.7 language level > - > > Key: CASSANDRA-16300 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16300 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 2.2.x > > Attachments: junitreport.zip > > Time Spent: 10m > Remaining Estimate: 0h > > I think that 2.2 needs to satisfy Java 1.7 language level. However, there are > a few places where newer syntax is used: > * > [MessagingService|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/net/MessagingService.java#L779] > * > [ExecutorUtils|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/utils/ExecutorUtils.java#L36-L47] > * > [Throwables|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/utils/Throwables.java#L36] > * > [YamlConfigurationLoader|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L142] > * > [YamlConfigurationLoaderTest|https://github.com/apache/cassandra/blob/cassandra-2.2/test/unit/org/apache/cassandra/config/YamlConfigurationLoaderTest.java#L38] > If I'm right and we really need 1.7 language level we should fix those, which > seems quite straightforward. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[ https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247938#comment-17247938 ] Marcus Eriksson edited comment on CASSANDRA-16328 at 12/11/20, 2:05 PM: bq. When I look through the tests run I only see indev_3_11_x_To_indev_trunk tests. I would have thought we should be including all indev_3_0_x_To_indev_trunk as well? looks like we don't specify indev_3_0_x to trunk: https://github.com/apache/cassandra-dtest/blob/trunk/upgrade_tests/upgrade_manifest.py#L154 I'm +1 if adding indev_trunk there makes 3.0 -> trunk tests run was (Author: krummas): bq. When I look through the tests run I only see indev_3_11_x_To_indev_trunk tests. I would have thought we should be including all indev_3_0_x_To_indev_trunk as well? looks like we don't specify indev_3_0_x to trunk: https://github.com/apache/cassandra-dtest/blob/trunk/upgrade_tests/upgrade_manifest.py#L154 > python upgrade tests include tests which are not impacted by the version > under test > --- > > Key: CASSANDRA-16328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16328 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 0.5h > Remaining Estimate: 0h > > When we try to run the upgrade tests for trunk, we get tests such as > upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x > to run, which do not include the trunk source. Since these tests are costly > and can be flaky, running the unneeded tests for a release increases the cost > and can make it harder to see if any new failures. > See > https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[ https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247938#comment-17247938 ] Marcus Eriksson commented on CASSANDRA-16328: - bq. When I look through the tests run I only see indev_3_11_x_To_indev_trunk tests. I would have thought we should be including all indev_3_0_x_To_indev_trunk as well? looks like we don't specify indev_3_0_x to trunk: https://github.com/apache/cassandra-dtest/blob/trunk/upgrade_tests/upgrade_manifest.py#L154 > python upgrade tests include tests which are not impacted by the version > under test > --- > > Key: CASSANDRA-16328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16328 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 0.5h > Remaining Estimate: 0h > > When we try to run the upgrade tests for trunk, we get tests such as > upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x > to run, which do not include the trunk source. Since these tests are costly > and can be flaky, running the unneeded tests for a release increases the cost > and can make it harder to see if any new failures. > See > https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 04/10: Tweaked Dockerfile and run.sh to use the new HarryRunnerJvm class.
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit 3ca3bd09c87d9ac147b0b2393d6ec758b6dd04a0
Author: Gianluca Righetto
AuthorDate: Tue Dec 8 10:07:09 2020 -0300
Tweaked Dockerfile and run.sh to use the new HarryRunnerJvm class.
---
docker/Dockerfile.local | 4 +++-
docker/run.sh | 4 ++--
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/docker/Dockerfile.local b/docker/Dockerfile.local
index 20e0454..caaa551 100644
--- a/docker/Dockerfile.local
+++ b/docker/Dockerfile.local
@@ -26,7 +26,9 @@ RUN mkdir -p /cassandra/harry
COPY ./harry-core/target/lib/* /opt/harry/lib/
COPY ./harry-core/target/*.jar /opt/harry/lib/
COPY ./harry-runner/target/lib/* /opt/harry/lib/
-COPY ./harry-runner/target/*.jar /opt/harry/
+COPY ./harry-runner/target/*.jar /opt/harry/lib/
+COPY ./harry-integration/target/lib/ /opt/harry/lib/
+COPY ./harry-integration/target/*.jar /opt/harry/
COPY ./test/conf/logback-dtest.xml /opt/harry/test/conf/logback-dtest.xml
COPY ./docker/run.sh /opt/harry/
diff --git a/docker/run.sh b/docker/run.sh
index 25a8296..cde13f4 100755
--- a/docker/run.sh
+++ b/docker/run.sh
@@ -72,9 +72,9 @@ while true; do
--add-opens java.base/jdk.internal.util.jar=ALL-UNNAMED \
--add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED \
-Dorg.apache.cassandra.test.logback.configurationFile=file:///opt/harry/test/conf/logback-dtest.xml
\
- -cp /opt/harry/lib/*:/opt/harry/harry-runner-0.0.1-SNAPSHOT.jar \
+ -cp /opt/harry/lib/*:/opt/harry/harry-integration-0.0.1-SNAPSHOT.jar \
-Dharry.root=${HARRY_DIR} \
- harry.runner.HarryRunner
+ harry.runner.HarryRunnerJvm
if [ $? -ne 0 ]; then
if [ -e "failure.dump" ]; then
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 01/10: Removed harry-core dependency on the in in-jvm dtest libs (cassandra-dtest-shaded and dtest-api); Moved in-jvm tests to the harry-integration module; Created a new module: har
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit 7a0ef9fe5b5c9341329c5067a6d0644342d89aa6
Author: Gianluca Righetto
AuthorDate: Sun Dec 6 21:13:05 2020 -0300
Removed harry-core dependency on the in in-jvm dtest libs
(cassandra-dtest-shaded and dtest-api); Moved in-jvm tests to the
harry-integration module; Created a new module: harry-integration-external for
connections with external clusters.
---
harry-core/pom.xml | 12 -
.../pom.xml| 14 +-
.../model/sut/external}/ExternalClusterSut.java| 5 +-
harry-integration/pom.xml | 2 +-
.../test/harry/op/RowVisitorTest.java | 5 --
pom.xml| 9 ++--
test/conf/cassandra.yaml | 53 ++
7 files changed, 63 insertions(+), 37 deletions(-)
diff --git a/harry-core/pom.xml b/harry-core/pom.xml
index 3c53a7d..6ca4ef0 100755
--- a/harry-core/pom.xml
+++ b/harry-core/pom.xml
@@ -59,18 +59,6 @@
-org.apache.cassandra
-cassandra-dtest-shaded
-test
-
-
-
-org.apache.cassandra
-dtest-api
-test
-
-
-
junit
junit
test
diff --git a/harry-integration/pom.xml b/harry-integration-external/pom.xml
similarity index 83%
copy from harry-integration/pom.xml
copy to harry-integration-external/pom.xml
index 99bacdb..3e4e72e 100755
--- a/harry-integration/pom.xml
+++ b/harry-integration-external/pom.xml
@@ -29,8 +29,8 @@
harry-parent
-harry-integration
-Harry Integration
+harry-integration-external
+Harry Integration - External
@@ -40,16 +40,6 @@
-org.apache.cassandra
-cassandra-dtest-shaded
-
-
-
-org.apache.cassandra
-dtest-api
-
-
-
junit
junit
test
diff --git a/harry-integration/src/harry/model/sut/ExternalClusterSut.java
b/harry-integration-external/src/harry/model/sut/external/ExternalClusterSut.java
similarity index 97%
rename from harry-integration/src/harry/model/sut/ExternalClusterSut.java
rename to
harry-integration-external/src/harry/model/sut/external/ExternalClusterSut.java
index 061d751..146a162 100644
--- a/harry-integration/src/harry/model/sut/ExternalClusterSut.java
+++
b/harry-integration-external/src/harry/model/sut/external/ExternalClusterSut.java
@@ -16,15 +16,13 @@
* limitations under the License.
*/
-package harry.model.sut;
+package harry.model.sut.external;
import java.util.List;
import java.util.concurrent.CompletableFuture;
-import java.util.concurrent.Executor;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
-import java.util.function.Consumer;
import com.google.common.util.concurrent.FutureCallback;
import com.google.common.util.concurrent.Futures;
@@ -36,6 +34,7 @@ import com.datastax.driver.core.QueryOptions;
import com.datastax.driver.core.ResultSet;
import com.datastax.driver.core.Row;
import com.datastax.driver.core.Session;
+import harry.model.sut.SystemUnderTest;
public class ExternalClusterSut implements SystemUnderTest
{
diff --git a/harry-integration/pom.xml b/harry-integration/pom.xml
index 99bacdb..3bc873a 100755
--- a/harry-integration/pom.xml
+++ b/harry-integration/pom.xml
@@ -30,7 +30,7 @@
harry-integration
-Harry Integration
+Harry Integration - InJVM
diff --git a/harry-core/test/harry/op/RowVisitorTest.java
b/harry-integration/test/harry/op/RowVisitorTest.java
similarity index 96%
rename from harry-core/test/harry/op/RowVisitorTest.java
rename to harry-integration/test/harry/op/RowVisitorTest.java
index c2db8ed..e58c05b 100644
--- a/harry-core/test/harry/op/RowVisitorTest.java
+++ b/harry-integration/test/harry/op/RowVisitorTest.java
@@ -18,7 +18,6 @@
package harry.op;
-import java.util.function.Consumer;
import java.util.function.Supplier;
import org.junit.Assert;
@@ -27,12 +26,8 @@ import org.junit.Test;
import harry.ddl.SchemaGenerators;
import harry.ddl.SchemaSpec;
-import harry.generators.Generator;
-import harry.generators.PcgRSUFast;
import harry.generators.RandomGenerator;
-import harry.generators.Surjections;
import harry.generators.distribution.Distribution;
-import harry.model.OpSelectorsTest;
import harry.runner.DefaultRowVisitor;
import harry.model.clock.OffsetClock;
import harry.model.OpSelectors;
diff --git a/pom.xml b/pom.xml
index 0bcf9c5..f582caa 100755
--- a/pom.xml
+++ b/pom.xml
@@ -32,15 +32,16 @@
harry-core
harry-integration
+
[cassandra-harry] 07/10: Use `logger.error` instead of `printStackTrace` in HarryRunner.
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit ede0115320a438dce9b5d19039782cacc0268a77
Author: Gianluca Righetto
AuthorDate: Wed Dec 9 17:05:35 2020 -0300
Use `logger.error` instead of `printStackTrace` in HarryRunner.
---
harry-runner/src/harry/runner/HarryRunner.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/harry-runner/src/harry/runner/HarryRunner.java
b/harry-runner/src/harry/runner/HarryRunner.java
index a18885b..3adf9c0 100644
--- a/harry-runner/src/harry/runner/HarryRunner.java
+++ b/harry-runner/src/harry/runner/HarryRunner.java
@@ -72,7 +72,7 @@ public interface HarryRunner
}).get(run.snapshot.run_time_unit.toSeconds(run.snapshot.run_time)
+ 30,
TimeUnit.SECONDS);
if (result instanceof Throwable)
-((Throwable) result).printStackTrace();
+logger.error("Execution failed", result);
}
catch (Throwable e)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 08/10: Moved HarryRunner interface to harry-core and removed the harry-runner module.
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit 62534abb0b022249ec118c3cac647153d2754c05
Author: Gianluca Righetto
AuthorDate: Wed Dec 9 19:09:20 2020 -0300
Moved HarryRunner interface to harry-core and removed the harry-runner
module.
---
.../src/harry/runner/HarryRunner.java | 20 --
.../src/harry/util}/ThrowingRunnable.java | 2 +-
harry-runner/pom.xml | 44 --
pom.xml| 1 -
4 files changed, 8 insertions(+), 59 deletions(-)
diff --git a/harry-runner/src/harry/runner/HarryRunner.java
b/harry-core/src/harry/runner/HarryRunner.java
similarity index 95%
rename from harry-runner/src/harry/runner/HarryRunner.java
rename to harry-core/src/harry/runner/HarryRunner.java
index 3adf9c0..56e2350 100644
--- a/harry-runner/src/harry/runner/HarryRunner.java
+++ b/harry-core/src/harry/runner/HarryRunner.java
@@ -18,6 +18,13 @@
package harry.runner;
+import harry.core.Configuration;
+import harry.core.Run;
+import harry.corruptor.*;
+import harry.util.ThrowingRunnable;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
import java.io.File;
import java.io.FileNotFoundException;
import java.util.Random;
@@ -26,19 +33,6 @@ import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import harry.core.Configuration;
-import harry.core.Run;
-import harry.corruptor.AddExtraRowCorruptor;
-import harry.corruptor.ChangeValueCorruptor;
-import harry.corruptor.HideRowCorruptor;
-import harry.corruptor.HideValueCorruptor;
-import harry.corruptor.QueryResponseCorruptor;
-import harry.model.ExhaustiveChecker;
-import harry.model.OpSelectors;
-
public interface HarryRunner
{
diff --git a/harry-runner/src/harry/runner/ThrowingRunnable.java
b/harry-core/src/harry/util/ThrowingRunnable.java
similarity index 93%
rename from harry-runner/src/harry/runner/ThrowingRunnable.java
rename to harry-core/src/harry/util/ThrowingRunnable.java
index 37b..58d7397 100644
--- a/harry-runner/src/harry/runner/ThrowingRunnable.java
+++ b/harry-core/src/harry/util/ThrowingRunnable.java
@@ -1,4 +1,4 @@
-package harry.runner;
+package harry.util;
public interface ThrowingRunnable {
void run() throws Throwable;
diff --git a/harry-runner/pom.xml b/harry-runner/pom.xml
deleted file mode 100755
index c5c401c..000
--- a/harry-runner/pom.xml
+++ /dev/null
@@ -1,44 +0,0 @@
-
-
-
-http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;>
-
-4.0.0
-jar
-
-
-org.apache.cassandra
-0.0.1-SNAPSHOT
-harry-parent
-
-
-harry-runner
-Harry Runner
-
-
-
-org.apache.cassandra
-harry-core
-${project.parent.version}
-
-
-
-
-
diff --git a/pom.xml b/pom.xml
index 35e1712..efb1cf2 100755
--- a/pom.xml
+++ b/pom.xml
@@ -33,7 +33,6 @@
harry-core
harry-integration
harry-integration-external
-harry-runner
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 09/10: Added license header to new files.
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit e491ca84690514330bf22ad1657c43af10442fc7
Author: Gianluca Righetto
AuthorDate: Wed Dec 9 19:19:08 2020 -0300
Added license header to new files.
---
harry-core/src/harry/util/ThrowingRunnable.java| 18 ++
.../src/harry/runner/external/HarryRunnerExternal.java | 18 ++
harry-integration/src/harry/runner/HarryRunnerJvm.java | 18 ++
3 files changed, 54 insertions(+)
diff --git a/harry-core/src/harry/util/ThrowingRunnable.java
b/harry-core/src/harry/util/ThrowingRunnable.java
index 58d7397..8f0c8ae 100644
--- a/harry-core/src/harry/util/ThrowingRunnable.java
+++ b/harry-core/src/harry/util/ThrowingRunnable.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package harry.util;
public interface ThrowingRunnable {
diff --git
a/harry-integration-external/src/harry/runner/external/HarryRunnerExternal.java
b/harry-integration-external/src/harry/runner/external/HarryRunnerExternal.java
index c926693..5462568 100644
---
a/harry-integration-external/src/harry/runner/external/HarryRunnerExternal.java
+++
b/harry-integration-external/src/harry/runner/external/HarryRunnerExternal.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package harry.runner.external;
import harry.core.Configuration;
diff --git a/harry-integration/src/harry/runner/HarryRunnerJvm.java
b/harry-integration/src/harry/runner/HarryRunnerJvm.java
index 71c6620..38bf264 100644
--- a/harry-integration/src/harry/runner/HarryRunnerJvm.java
+++ b/harry-integration/src/harry/runner/HarryRunnerJvm.java
@@ -1,3 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package harry.runner;
import harry.core.Configuration;
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 03/10: Temporarily disabled `ExhaustiveCheckerIntegrationTest#testDetectsOverwrittenRow`.
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit d79455b19c3206129827058859d5bef74c523bb2
Author: Gianluca Righetto
AuthorDate: Mon Dec 7 19:08:51 2020 -0300
Temporarily disabled
`ExhaustiveCheckerIntegrationTest#testDetectsOverwrittenRow`.
---
.../test/harry/model/ExhaustiveCheckerIntegrationTest.java | 2 ++
1 file changed, 2 insertions(+)
diff --git
a/harry-integration/test/harry/model/ExhaustiveCheckerIntegrationTest.java
b/harry-integration/test/harry/model/ExhaustiveCheckerIntegrationTest.java
index be45f91..4a38e7b 100644
--- a/harry-integration/test/harry/model/ExhaustiveCheckerIntegrationTest.java
+++ b/harry-integration/test/harry/model/ExhaustiveCheckerIntegrationTest.java
@@ -22,6 +22,7 @@ import java.util.concurrent.CompletableFuture;
import java.util.function.Consumer;
import org.junit.Assert;
+import org.junit.Ignore;
import org.junit.Test;
import harry.core.Configuration;
@@ -122,6 +123,7 @@ public class ExhaustiveCheckerIntegrationTest extends
IntegrationTestBase
@Test
+@Ignore
public void testDetectsOverwrittenRow()
{
negativeTest((run) -> {
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 06/10: Fixed Java var names to match YAML props.
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit 884b3d1225692d652682fbfdd2d35b1d06000f2f
Author: Gianluca Righetto
AuthorDate: Wed Dec 9 15:42:38 2020 -0300
Fixed Java var names to match YAML props.
---
harry-core/src/harry/core/Configuration.java | 32 ++--
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/harry-core/src/harry/core/Configuration.java
b/harry-core/src/harry/core/Configuration.java
index e01eac4..a3c287f 100644
--- a/harry-core/src/harry/core/Configuration.java
+++ b/harry-core/src/harry/core/Configuration.java
@@ -429,40 +429,40 @@ public class Configuration
@JsonTypeName("debug_approximate_monotonic")
public static class DebugApproximateMonotonicClockConfiguration implements
ClockConfiguration
{
-public final long startTimeMicros;
-public final int historySize;
+public final long start_time_micros;
+public final int history_size;
public final long[] history;
public final long lts;
public final int idx;
-public final long epochPeriod;
-public final TimeUnit epochTimeUnit;
+public final long epoch_period;
+public final TimeUnit epoch_time_unit;
@JsonCreator
-public
DebugApproximateMonotonicClockConfiguration(@JsonProperty("start_time_micros")
long startTimeMicros,
-
@JsonProperty("history_size") int historySize,
+public
DebugApproximateMonotonicClockConfiguration(@JsonProperty("start_time_micros")
long start_time_micros,
+
@JsonProperty("history_size") int history_size,
@JsonProperty("history") long[] history,
@JsonProperty("lts") long lts,
@JsonProperty("idx") int idx,
-
@JsonProperty("epoch_period") long epochPeriod,
-
@JsonProperty("epoch_time_unit") TimeUnit epochTimeUnit)
+
@JsonProperty("epoch_period") long epoch_period,
+
@JsonProperty("epoch_time_unit") TimeUnit epoch_time_unit)
{
-this.startTimeMicros = startTimeMicros;
-this.historySize = historySize;
+this.start_time_micros = start_time_micros;
+this.history_size = history_size;
this.history = history;
this.lts = lts;
this.idx = idx;
-this.epochPeriod = epochPeriod;
-this.epochTimeUnit = epochTimeUnit;
+this.epoch_period = epoch_period;
+this.epoch_time_unit = epoch_time_unit;
}
public OpSelectors.MonotonicClock make()
{
-return ApproximateMonotonicClock.forDebug(startTimeMicros,
- historySize,
+return ApproximateMonotonicClock.forDebug(start_time_micros,
+ history_size,
lts,
idx,
- epochPeriod,
- epochTimeUnit,
+ epoch_period,
+ epoch_time_unit,
history);
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 10/10: Removed references to harry-runner from Dockerfile.
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git commit 41324f5ccaf1d9149240db8e7c1e0cd69d406e81 Author: Gianluca Righetto AuthorDate: Wed Dec 9 21:41:26 2020 -0300 Removed references to harry-runner from Dockerfile. --- docker/Dockerfile.local | 2 -- 1 file changed, 2 deletions(-) diff --git a/docker/Dockerfile.local b/docker/Dockerfile.local index 5bc3943..21ce8fc 100644 --- a/docker/Dockerfile.local +++ b/docker/Dockerfile.local @@ -25,8 +25,6 @@ RUN mkdir -p /cassandra/harry COPY ./harry-core/target/lib/* /opt/harry/lib/ COPY ./harry-core/target/*.jar /opt/harry/lib/ -COPY ./harry-runner/target/lib/* /opt/harry/lib/ -COPY ./harry-runner/target/*.jar /opt/harry/lib/ COPY ./harry-integration/target/lib/ /opt/harry/lib/ COPY ./harry-integration/target/*.jar /opt/harry/ COPY ./test/conf/logback-dtest.xml /opt/harry/test/conf/logback-dtest.xml - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] branch trunk updated (7360458 -> 41324f5)
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git.
from 7360458 Update jackson dependency to 2.11.3 to force yaml to 1.26
new 7a0ef9f Removed harry-core dependency on the in in-jvm dtest libs
(cassandra-dtest-shaded and dtest-api); Moved in-jvm tests to the
harry-integration module; Created a new module: harry-integration-external for
connections with external clusters.
new 4de2deb Allow for YAML-based configuration for execution against an
external cluster; Build an uber jar for harry-integration-external.
new d79455b Temporarily disabled
`ExhaustiveCheckerIntegrationTest#testDetectsOverwrittenRow`.
new 3ca3bd0 Tweaked Dockerfile and run.sh to use the new HarryRunnerJvm
class.
new 9c5eb00 Fixed broken & missing props in example YAML; Added new
example config for external cluster; Added run scripts to help with getting
started; Fixed object mapping for `ApproximateMonotonicClockConfiguration`;
Removed hardcoded configuration for in-JVM execution;
new 884b3d1 Fixed Java var names to match YAML props.
new ede0115 Use `logger.error` instead of `printStackTrace` in
HarryRunner.
new 62534ab Moved HarryRunner interface to harry-core and removed the
harry-runner module.
new e491ca8 Added license header to new files.
new 41324f5 Removed references to harry-runner from Dockerfile.
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
conf/example.yaml | 10 +-
conf/{example.yaml => external.yaml} | 19 ++--
docker/Dockerfile.local| 5 +-
docker/run.sh | 5 +-
harry-core/pom.xml | 12 ---
harry-core/src/harry/core/Configuration.java | 40
.../src/harry/runner/HarryRunner.java | 106 -
.../src/harry/util/ThrowingRunnable.java | 23 ++---
.../pom.xml| 50 --
.../model/sut/external}/ExternalClusterSut.java| 62
.../harry/runner/external/HarryRunnerExternal.java | 26 ++---
harry-integration/pom.xml | 10 +-
.../src/harry/model/sut/InJvmSut.java | 2 +-
.../src/harry/runner/HarryRunnerJvm.java | 27 +++---
.../src/harry/runner/Reproduce.java| 2 +-
.../model/ExhaustiveCheckerIntegrationTest.java| 2 +
.../test/harry/op/RowVisitorTest.java | 5 -
pom.xml| 17 +++-
run-external.sh| 3 +
docker/run.sh => run-jvm.sh| 68 ++---
test/conf/cassandra.yaml | 53 +++
21 files changed, 301 insertions(+), 246 deletions(-)
copy conf/{example.yaml => external.yaml} (93%)
rename {harry-runner => harry-core}/src/harry/runner/HarryRunner.java (60%)
copy harry-integration/test/harry/model/TestBaseImpl.java =>
harry-core/src/harry/util/ThrowingRunnable.java (71%)
rename {harry-runner => harry-integration-external}/pom.xml (50%)
rename {harry-integration/src/harry/model/sut =>
harry-integration-external/src/harry/model/sut/external}/ExternalClusterSut.java
(71%)
copy harry-core/src/harry/model/DoNothingModel.java =>
harry-integration-external/src/harry/runner/external/HarryRunnerExternal.java
(62%)
copy harry-core/src/harry/model/DoNothingModel.java =>
harry-integration/src/harry/runner/HarryRunnerJvm.java (63%)
rename {harry-runner => harry-integration}/src/harry/runner/Reproduce.java
(98%)
rename {harry-core => harry-integration}/test/harry/op/RowVisitorTest.java
(96%)
create mode 100755 run-external.sh
copy docker/run.sh => run-jvm.sh (50%)
create mode 100644 test/conf/cassandra.yaml
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 05/10: Fixed broken & missing props in example YAML; Added new example config for external cluster; Added run scripts to help with getting started; Fixed object mapping for `Approxim
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit 9c5eb00b2f88dd63a40c21d4ff7b618b5d2c1457
Author: Gianluca Righetto
AuthorDate: Tue Dec 8 17:05:31 2020 -0300
Fixed broken & missing props in example YAML; Added new example config for
external cluster; Added run scripts to help with getting started; Fixed object
mapping for `ApproximateMonotonicClockConfiguration`; Removed hardcoded
configuration for in-JVM execution;
---
conf/example.yaml | 10 +++-
conf/external.yaml | 12 ++--
docker/Dockerfile.local| 1 +
docker/run.sh | 3 +-
harry-core/src/harry/core/Configuration.java | 1 +
.../model/sut/external/ExternalClusterSut.java | 2 +-
.../harry/runner/external/HarryRunnerExternal.java | 23 ++--
.../src/harry/model/sut/InJvmSut.java | 2 +-
.../src/harry/runner/HarryRunnerJvm.java | 13 +++--
harry-integration/src/harry/runner/Reproduce.java | 2 +-
harry-runner/src/harry/runner/HarryRunner.java | 55 -
run-external.sh| 3 +
docker/run.sh => run-jvm.sh| 68 ++
13 files changed, 69 insertions(+), 126 deletions(-)
diff --git a/conf/example.yaml b/conf/example.yaml
index de8c77d..3a3c5ab 100644
--- a/conf/example.yaml
+++ b/conf/example.yaml
@@ -31,9 +31,9 @@ truncate_table: false
# to map it back to the logical timestamp of the operation that wrote this
value.
clock:
approximate_monotonic:
-historySize: 7300
-epochPeriod: 1
-epochTimeUnit: "SECONDS"
+history_size: 7300
+epoch_length: 1
+epoch_time_unit: "SECONDS"
# Runner is a is a component that schedules operations that change the cluster
(system under test)
# and model state.
@@ -88,3 +88,7 @@ clustering_descriptor_selector:
DELETE_COLUMN: 1
column_mask_bitsets: null
max_partition_size: 100
+
+# Default Row Visitor
+row_visitor:
+ default: {}
\ No newline at end of file
diff --git a/conf/external.yaml b/conf/external.yaml
index 742b624..770dda9 100644
--- a/conf/external.yaml
+++ b/conf/external.yaml
@@ -30,10 +30,10 @@ truncate_table: false
# be taken to map a real-time timestamp from the value retrieved from the
database in order
# to map it back to the logical timestamp of the operation that wrote this
value.
clock:
- debug_approximate_monotonic:
-historySize: 7300
-epochPeriod: 1
-epochTimeUnit: "SECONDS"
+ approximate_monotonic:
+history_size: 7300
+epoch_length: 1
+epoch_time_unit: "SECONDS"
# Runner is a is a component that schedules operations that change the cluster
(system under test)
# and model state.
@@ -89,3 +89,7 @@ clustering_descriptor_selector:
DELETE_COLUMN: 1
column_mask_bitsets: null
max_partition_size: 100
+
+# Default Row Visitor
+row_visitor:
+ default: {}
\ No newline at end of file
diff --git a/docker/Dockerfile.local b/docker/Dockerfile.local
index caaa551..5bc3943 100644
--- a/docker/Dockerfile.local
+++ b/docker/Dockerfile.local
@@ -30,6 +30,7 @@ COPY ./harry-runner/target/*.jar /opt/harry/lib/
COPY ./harry-integration/target/lib/ /opt/harry/lib/
COPY ./harry-integration/target/*.jar /opt/harry/
COPY ./test/conf/logback-dtest.xml /opt/harry/test/conf/logback-dtest.xml
+COPY ./conf/example.yaml /opt/harry/example.yaml
COPY ./docker/run.sh /opt/harry/
WORKDIR /opt/harry
diff --git a/docker/run.sh b/docker/run.sh
index cde13f4..490b21f 100755
--- a/docker/run.sh
+++ b/docker/run.sh
@@ -74,7 +74,8 @@ while true; do
-Dorg.apache.cassandra.test.logback.configurationFile=file:///opt/harry/test/conf/logback-dtest.xml
\
-cp /opt/harry/lib/*:/opt/harry/harry-integration-0.0.1-SNAPSHOT.jar \
-Dharry.root=${HARRY_DIR} \
- harry.runner.HarryRunnerJvm
+ harry.runner.HarryRunnerJvm \
+ /opt/harry/example.yaml
if [ $? -ne 0 ]; then
if [ -e "failure.dump" ]; then
diff --git a/harry-core/src/harry/core/Configuration.java
b/harry-core/src/harry/core/Configuration.java
index 0c37f74..e01eac4 100644
--- a/harry-core/src/harry/core/Configuration.java
+++ b/harry-core/src/harry/core/Configuration.java
@@ -68,6 +68,7 @@ public class Configuration
.disable(YAMLGenerator.Feature.CANONICAL_OUTPUT)
.enable(YAMLGenerator.Feature.INDENT_ARRAYS));
mapper.registerSubtypes(Configuration.DebugApproximateMonotonicClockConfiguration.class);
+
mapper.registerSubtypes(Configuration.ApproximateMonotonicClockConfiguration.class);
mapper.registerSubtypes(Configuration.ConcurrentRunnerConfig.class);
mapper.registerSubtypes(Configuration.ExhaustiveCheckerConfig.class);
diff
[cassandra-harry] 02/10: Allow for YAML-based configuration for execution against an external cluster; Build an uber jar for harry-integration-external.
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit 4de2debdcab63443738d8da936db54e3195ed45f
Author: Gianluca Righetto
AuthorDate: Mon Dec 7 13:14:40 2020 -0300
Allow for YAML-based configuration for execution against an external
cluster; Build an uber jar for harry-integration-external.
---
conf/external.yaml | 91 ++
harry-core/src/harry/core/Configuration.java | 7 +-
harry-integration-external/pom.xml | 39 +-
.../model/sut/external/ExternalClusterSut.java | 59 ++
.../harry/runner/external/HarryRunnerExternal.java | 33
harry-integration/pom.xml | 8 +-
.../src/harry/runner/HarryRunnerJvm.java | 18 +
.../src/harry/runner/Reproduce.java| 0
harry-runner/pom.xml | 11 ---
harry-runner/src/harry/runner/HarryRunner.java | 35 ++---
.../src/harry/runner/ThrowingRunnable.java | 15
pom.xml| 7 ++
12 files changed, 268 insertions(+), 55 deletions(-)
diff --git a/conf/external.yaml b/conf/external.yaml
new file mode 100644
index 000..742b624
--- /dev/null
+++ b/conf/external.yaml
@@ -0,0 +1,91 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+seed: 1596731732524
+
+# Default schema provider generates random schema
+schema_provider:
+ default: {}
+
+drop_schema: false
+create_schema: true
+truncate_table: false
+
+# Clock is a component responsible for mapping _logical_ timestamps to
_real-time_ ones.
+#
+# When reproducing test failures, and for validation purposes, a snapshot of
such clock can
+# be taken to map a real-time timestamp from the value retrieved from the
database in order
+# to map it back to the logical timestamp of the operation that wrote this
value.
+clock:
+ debug_approximate_monotonic:
+historySize: 7300
+epochPeriod: 1
+epochTimeUnit: "SECONDS"
+
+# Runner is a is a component that schedules operations that change the cluster
(system under test)
+# and model state.
+runner:
+ concurrent:
+writer_threads: 2
+round_robin_validator_threads: 1
+recent_partition_validator_threads: 1
+
+run_time: 2
+run_time_unit: "HOURS"
+
+# System under test: a Cassandra node or cluster. Default implementation is
in_jvm (in-jvm DTest cluster).
+# Harry also supports external clusters.
+system_under_test:
+ external:
+contact_points: 127.0.0.1
+port: 9042
+username: null
+password: null
+
+# Model is responsible for tracking logical timestamps that
+model:
+ exhaustive_checker:
+max_seen_lts: 19
+max_complete_lts: 16
+
+# Partition descriptor selector controls how partitions is selected based on
the current logical
+# timestamp. Default implementation is a sliding window of partition
descriptors that will visit
+# one partition after the other in the window `slide_after_repeats` times.
After that will
+# retire one partition descriptor, and pick one instead of it.
+partition_descriptor_selector:
+ default:
+window_size: 10
+slide_after_repeats: 100
+
+# Clustering descriptor selector controls how clusterings are picked within
the partition:
+# how many rows there can be in a partition, how many rows will be visited for
a logical timestamp,
+# how many operations there will be in batch, what kind of operations there
will and how often
+# each kind of operation is going to occur.
+clustering_descriptor_selector:
+ default:
+modifications_per_lts:
+ type: "constant"
+ constant: 10
+rows_per_modification:
+ type: "constant"
+ constant: 10
+operation_kind_weights:
+ WRITE: 97
+ DELETE_RANGE: 1
+ DELETE_ROW: 1
+ DELETE_COLUMN: 1
+column_mask_bitsets: null
+max_partition_size: 100
diff --git a/harry-core/src/harry/core/Configuration.java
b/harry-core/src/harry/core/Configuration.java
index 3a68053..0c37f74 100644
--- a/harry-core/src/harry/core/Configuration.java
+++
[jira] [Commented] (CASSANDRA-16311) Extend the exclusion of replica filtering protection to other indices instead of just SASI
[
https://issues.apache.org/jira/browse/CASSANDRA-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247888#comment-17247888
]
Andres de la Peña commented on CASSANDRA-16311:
---
[~stefan.miklosovic] thanks. I'm running a final CI round here:
||branch||utest||python-dtest||java-dtest||
|3.0|[129|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-test/129/]|[279|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/279/]|[124|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-jvm-dtest/124/]|
|3.11|[130|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-test/130/]|[280|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/280/]|[125|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-jvm-dtest/125/]|
|trunk|[131|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-test/131/]|[281|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/281/]|[126|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-jvm-dtest/126/]|
If it's ok with you I'll add the following to the JavaDoc of the new method
during commit:
{code:java}
Replica filtering protection might need to run the query row filter in the
coordinator to detect stale results. An index implementation will be compatible
with this protection mechanism if it returns the same results for the row
filter as CQL will return with {@code ALLOW FILTERING} and without using the
index. This means that index implementations using custom query syntax or
applying transformations to the indexed data won't support it.
See CASSANDRA-8272 for further details.
{code}
> Extend the exclusion of replica filtering protection to other indices instead
> of just SASI
> --
>
> Key: CASSANDRA-16311
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16311
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index, Feature/SASI
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> There was a check introduced in CASSANDRA-8272 telling if an index is a SASI
> index and if it is, replica filtering protection was not triggered.
> There might be other custom index implementations which also do not support
> filtering protection and they do not have to be SASI indices neccessarily,
> however it is not possible to exclude them.
>
> PR trunk
> [https://github.com/apache/cassandra/pull/844]
> PR 3.11
> [https://github.com/apache/cassandra/pull/847]
> PR 3.0
> https://github.com/apache/cassandra/pull/848
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16334) Replica failure causes timeout on multi-DC write
[
https://issues.apache.org/jira/browse/CASSANDRA-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-16334:
Description:
Inserting a mutation larger than {{max_mutation_size_in_kb}} correctly throws a
write error on a single DC keyspace with RF=3:
{noformat}
cassandra.WriteFailure: Error from server: code=1500 [Replica(s) failed to
execute write] message="Operation failed - received 0 responses and 3 failures:
UNKNOWN from /127.0.0.3:7000, UNKNOWN from /127.0.0.2:7000, UNKNOWN from
/127.0.0.1:7000" info={'consistency': 'LOCAL_ONE', 'required_responses': 1,
'received_responses': 0, 'failures': 3}
{noformat}
The same insert wrongly causes a timeout on a keyspace with 2 dcs (RF=3 each):
{noformat}
cassandra.WriteTimeout: Error from server: code=1100 [Coordinator node timed
out waiting for replica nodes' responses] message="Operation timed out -
received only 0 responses." info={'consistency': 'LOCAL_ONE',
'required_responses': 1, 'received_responses': 0}
{noformat}
Reproduction steps:
{noformat}
# Setup cluster
ccm create -n 3:3 test
for i in {1..6}; do echo 'max_mutation_size_in_kb: 1000' >>
~/.ccm/test/node$i/conf/cassandra.yaml; done
ccm start
# Create schema
ccm node1 cqlsh
CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
'dc1': 3, 'dc2': 3};
CREATE TABLE test.test (key int PRIMARY KEY, val blob);
exit;
# Insert data
python
from cassandra.cluster import Cluster
session = cluster.connect('test')
blob = f = open("2mbBlob", "rb").read().hex()
session.execute("INSERT INTO test (key, val) VALUES (1, textAsBlob('" + blob +
"'))")
{noformat}
Reproduced in 3.11, trunk.
was:
Inserting a mutation larger than {{max_mutation_size_in_kb}} correctly throws a
write error on a single DC keyspace with RF=3:
{noformat}
cassandra.WriteFailure: Error from server: code=1500 [Replica(s) failed to
execute write] message="Operation failed - received 0 responses and 3 failures:
UNKNOWN from /127.0.0.3:7000, UNKNOWN from /127.0.0.2:7000, UNKNOWN from
/127.0.0.1:7000" info={'consistency': 'LOCAL_ONE', 'required_responses': 1,
'received_responses': 0, 'failures': 3}
{noformat}
The same insert wrongly causes a timeout on a keyspace with 2 dcs (RF=3 each):
{noformat}
cassandra.WriteTimeout: Error from server: code=1100 [Coordinator node timed
out waiting for replica nodes' responses] message="Operation timed out -
received only 0 responses." info={'consistency': 'LOCAL_ONE',
'required_responses': 1, 'received_responses': 0}
{noformat}
Reproduction steps:
{noformat}
# Setup cluster
ccm create -n 3:3 test
for i in {1..9}; do echo 'max_mutation_size_in_kb: 1000' >>
~/.ccm/test/node$i/conf/cassandra.yaml; done
ccm start
# Create schema
ccm node1 cqlsh
CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
'dc1': 3, 'dc2': 3};
CREATE TABLE test.test (key int PRIMARY KEY, val blob);
exit;
# Insert data
python
from cassandra.cluster import Cluster
session = cluster.connect('test')
blob = f = open("2mbBlob", "rb").read().hex()
session.execute("INSERT INTO test (key, val) VALUES (1, textAsBlob('" + blob +
"'))")
{noformat}
Reproduced in 3.11, trunk.
> Replica failure causes timeout on multi-DC write
>
>
> Key: CASSANDRA-16334
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16334
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Messaging/Internode
>Reporter: Paulo Motta
>Priority: Normal
>
> Inserting a mutation larger than {{max_mutation_size_in_kb}} correctly throws
> a write error on a single DC keyspace with RF=3:
> {noformat}
> cassandra.WriteFailure: Error from server: code=1500 [Replica(s) failed to
> execute write] message="Operation failed - received 0 responses and 3
> failures: UNKNOWN from /127.0.0.3:7000, UNKNOWN from /127.0.0.2:7000, UNKNOWN
> from /127.0.0.1:7000" info={'consistency': 'LOCAL_ONE', 'required_responses':
> 1, 'received_responses': 0, 'failures': 3}
> {noformat}
> The same insert wrongly causes a timeout on a keyspace with 2 dcs (RF=3 each):
> {noformat}
> cassandra.WriteTimeout: Error from server: code=1100 [Coordinator node timed
> out waiting for replica nodes' responses] message="Operation timed out -
> received only 0 responses." info={'consistency': 'LOCAL_ONE',
> 'required_responses': 1, 'received_responses': 0}
> {noformat}
> Reproduction steps:
> {noformat}
> # Setup cluster
> ccm create -n 3:3 test
> for i in {1..6}; do echo 'max_mutation_size_in_kb: 1000' >>
> ~/.ccm/test/node$i/conf/cassandra.yaml; done
> ccm start
> # Create schema
> ccm node1 cqlsh
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'dc1': 3, 'dc2': 3};
> CREATE TABLE test.test (key int PRIMARY KEY, val blob);
> exit;
> # Insert data
>
[jira] [Updated] (CASSANDRA-16334) Replica failure causes timeout on multi-DC write
[
https://issues.apache.org/jira/browse/CASSANDRA-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-16334:
Bug Category: Parent values: Correctness(12982)Level 1 values: API /
Semantic Implementation(12988)
Complexity: Normal
Component/s: Messaging/Internode
Consistency/Coordination
Discovered By: User Report
Severity: Normal
Status: Open (was: Triage Needed)
> Replica failure causes timeout on multi-DC write
>
>
> Key: CASSANDRA-16334
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16334
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Messaging/Internode
>Reporter: Paulo Motta
>Priority: Normal
>
> Inserting a mutation larger than {{max_mutation_size_in_kb}} correctly throws
> a write error on a single DC keyspace with RF=3:
> {noformat}
> cassandra.WriteFailure: Error from server: code=1500 [Replica(s) failed to
> execute write] message="Operation failed - received 0 responses and 3
> failures: UNKNOWN from /127.0.0.3:7000, UNKNOWN from /127.0.0.2:7000, UNKNOWN
> from /127.0.0.1:7000" info={'consistency': 'LOCAL_ONE', 'required_responses':
> 1, 'received_responses': 0, 'failures': 3}
> {noformat}
> The same insert wrongly causes a timeout on a keyspace with 2 dcs (RF=3 each):
> {noformat}
> cassandra.WriteTimeout: Error from server: code=1100 [Coordinator node timed
> out waiting for replica nodes' responses] message="Operation timed out -
> received only 0 responses." info={'consistency': 'LOCAL_ONE',
> 'required_responses': 1, 'received_responses': 0}
> {noformat}
> Reproduction steps:
> {noformat}
> # Setup cluster
> ccm create -n 3:3 test
> for i in {1..9}; do echo 'max_mutation_size_in_kb: 1000' >>
> ~/.ccm/test/node$i/conf/cassandra.yaml; done
> ccm start
> # Create schema
> ccm node1 cqlsh
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'dc1': 3, 'dc2': 3};
> CREATE TABLE test.test (key int PRIMARY KEY, val blob);
> exit;
> # Insert data
> python
> from cassandra.cluster import Cluster
> session = cluster.connect('test')
> blob = f = open("2mbBlob", "rb").read().hex()
> session.execute("INSERT INTO test (key, val) VALUES (1, textAsBlob('" + blob
> + "'))")
> {noformat}
> Reproduced in 3.11, trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16334) Replica failure causes timeout on multi-DC write
Paulo Motta created CASSANDRA-16334:
---
Summary: Replica failure causes timeout on multi-DC write
Key: CASSANDRA-16334
URL: https://issues.apache.org/jira/browse/CASSANDRA-16334
Project: Cassandra
Issue Type: Bug
Reporter: Paulo Motta
Inserting a mutation larger than {{max_mutation_size_in_kb}} correctly throws a
write error on a single DC keyspace with RF=3:
{noformat}
cassandra.WriteFailure: Error from server: code=1500 [Replica(s) failed to
execute write] message="Operation failed - received 0 responses and 3 failures:
UNKNOWN from /127.0.0.3:7000, UNKNOWN from /127.0.0.2:7000, UNKNOWN from
/127.0.0.1:7000" info={'consistency': 'LOCAL_ONE', 'required_responses': 1,
'received_responses': 0, 'failures': 3}
{noformat}
The same insert wrongly causes a timeout on a keyspace with 2 dcs (RF=3 each):
{noformat}
cassandra.WriteTimeout: Error from server: code=1100 [Coordinator node timed
out waiting for replica nodes' responses] message="Operation timed out -
received only 0 responses." info={'consistency': 'LOCAL_ONE',
'required_responses': 1, 'received_responses': 0}
{noformat}
Reproduction steps:
{noformat}
# Setup cluster
ccm create -n 3:3 test
for i in {1..9}; do echo 'max_mutation_size_in_kb: 1000' >>
~/.ccm/test/node$i/conf/cassandra.yaml; done
ccm start
# Create schema
ccm node1 cqlsh
CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
'dc1': 3, 'dc2': 3};
CREATE TABLE test.test (key int PRIMARY KEY, val blob);
exit;
# Insert data
python
from cassandra.cluster import Cluster
session = cluster.connect('test')
blob = f = open("2mbBlob", "rb").read().hex()
session.execute("INSERT INTO test (key, val) VALUES (1, textAsBlob('" + blob +
"'))")
{noformat}
Reproduced in 3.11, trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[ https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247859#comment-17247859 ] Michael Semb Wever edited comment on CASSANDRA-16328 at 12/11/20, 11:42 AM: bq. … RUN_STATIC_UPGRADE_MATRIX … What is the purpose of this variable? When would we want/need to use it? If we have no use for it, can we just remove it altogether? Here's the patch for the same in ci-cassandra.a.o in-sync - cassandra-builds patch: https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:mck/16328 - example CI run: https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-dtest-upgrade/9/ When I look through the tests run I only see *indev_3_11_x_To_indev_trunk* tests. I would have thought we should be including all *indev_3_0_x_To_indev_trunk* as well? was (Author: michaelsembwever): bq. … RUN_STATIC_UPGRADE_MATRIX … What is the purpose of this variable? When would we want/need to use it? If we have no use for it, can we just remove it altogether? Here's the patch for the same in ci-cassandra.a.o in-sync - cassandra-builds patch: https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:mck/16328 - example CI run: https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-dtest-upgrade/9/ When I look through the tests run I only see *indev_3_11_x_To_indev_trunk* tests. I would have thought we should be including all *indev_3_0_x_To_indev_trunk* as well? > python upgrade tests include tests which are not impacted by the version > under test > --- > > Key: CASSANDRA-16328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16328 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 0.5h > Remaining Estimate: 0h > > When we try to run the upgrade tests for trunk, we get tests such as > upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x > to run, which do not include the trunk source. Since these tests are costly > and can be flaky, running the unneeded tests for a release increases the cost > and can make it harder to see if any new failures. > See > https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[ https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247859#comment-17247859 ] Michael Semb Wever commented on CASSANDRA-16328: bq. … RUN_STATIC_UPGRADE_MATRIX … What is the purpose of this variable? When would we want/need to use it? If we have no use for it, can we just remove it altogether? Here's the patch for the same in ci-cassandra.a.o in-sync - cassandra-builds patch: https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:mck/16328 - example CI run: https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-dtest-upgrade/9/ When I look through the tests run I only see *indev_3_11_x_To_indev_trunk* tests. I would have thought we should be including all *indev_3_0_x_To_indev_trunk* as well? > python upgrade tests include tests which are not impacted by the version > under test > --- > > Key: CASSANDRA-16328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16328 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 0.5h > Remaining Estimate: 0h > > When we try to run the upgrade tests for trunk, we get tests such as > upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x > to run, which do not include the trunk source. Since these tests are costly > and can be flaky, running the unneeded tests for a release increases the cost > and can make it harder to see if any new failures. > See > https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16290) Consistency can be violated when bootstrap or decommission is resumed after node restart
[
https://issues.apache.org/jira/browse/CASSANDRA-16290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta reassigned CASSANDRA-16290:
---
Assignee: Paulo Motta
> Consistency can be violated when bootstrap or decommission is resumed after
> node restart
>
>
> Key: CASSANDRA-16290
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16290
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> Since CASSANDRA-12008, successfully transferred ranges during decommission
> are saved on the {{system.transferred_ranges}} table. This allow skipping
> ranges already transferred when a failed decommission is retried with
> {{nodetool decommission}}.
> If instead of resuming the decommission, an operator restarts the node, waits
> N minutes and then performs a new decommission, the previously transferred
> ranges will be skipped during streaming, and any writes received by the
> decommissioned node during these N minutes will not be replicated to the new
> range owner, what violates consistency.
> This issue is analogous to the issue mentioned [on this
> comment|https://issues.apache.org/jira/browse/CASSANDRA-8838?focusedCommentId=16900234=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16900234]
> for resumable bootstrap (CASSANDRA-8838).
> In order to prevent consistency violations we should clear the
> {{system.transferred_ranges}} state during node restart, and maybe a system
> property to disable it. While we're at this, we should change the default of
> {{-Dcassandra.reset_bootstrap_progress}} to {{true}} to clear the
> {{system.available_ranges}} state by default when a bootstrapping node is
> restarted.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16333) Document provide_overlapping_tombstones compaction option
[ https://issues.apache.org/jira/browse/CASSANDRA-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-16333: Change Category: Operability Complexity: Low Hanging Fruit Component/s: Documentation/Website Status: Open (was: Triage Needed) > Document provide_overlapping_tombstones compaction option > - > > Key: CASSANDRA-16333 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16333 > Project: Cassandra > Issue Type: Improvement > Components: Documentation/Website >Reporter: Paulo Motta >Priority: Normal > > This option was added on CASSANDRA-7019 but it's not documented. We should > add it to > https://cassandra.apache.org/doc/latest/operating/compaction/index.html. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16333) Document provide_overlapping_tombstones compaction option
Paulo Motta created CASSANDRA-16333: --- Summary: Document provide_overlapping_tombstones compaction option Key: CASSANDRA-16333 URL: https://issues.apache.org/jira/browse/CASSANDRA-16333 Project: Cassandra Issue Type: Improvement Reporter: Paulo Motta This option was added on CASSANDRA-7019 but it's not documented. We should add it to https://cassandra.apache.org/doc/latest/operating/compaction/index.html. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch trunk updated: A more intuitve layout of devbranch builds in nightlies.a.o
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/trunk by this push:
new da5f40a A more intuitve layout of devbranch builds in nightlies.a.o
da5f40a is described below
commit da5f40aa6f1bf39fb917ddf1623bc8b8b73cf0fd
Author: Mick Semb Wever
AuthorDate: Sat Nov 21 19:14:20 2020 +0100
A more intuitve layout of devbranch builds in nightlies.a.o
---
jenkins-dsl/cassandra_job_dsl_seed.groovy | 32 ---
jenkins-dsl/cassandra_pipeline.groovy | 2 +-
2 files changed, 26 insertions(+), 8 deletions(-)
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 828acb7..32a4bbf 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -420,6 +420,15 @@ matrixJob('Cassandra-template-cqlsh-tests') {
""")
}
publishers {
+publishOverSsh {
+server('Nightlies') {
+transferSet {
+sourceFiles("**/cqlshlib.xml,**/nosetests.xml, **/*.head")
+remoteDirectory("cassandra/\${JOB_NAME}/\${BUILD_NUMBER}/")
+}
+}
+failOnError(false)
+}
archiveArtifacts {
pattern('**/cqlshlib.xml,**/nosetests.xml, **/*.head')
allowEmpty()
@@ -638,7 +647,7 @@ matrixJob('Cassandra-devbranch-artifacts') {
}
compressBuildLog()
logRotator {
-numToKeep(10)
+numToKeep(90)
artifactNumToKeep(10)
}
wrappers {
@@ -690,7 +699,7 @@ matrixJob('Cassandra-devbranch-artifacts') {
server('Nightlies') {
transferSet {
sourceFiles("build/apache-cassandra-*.tar.gz,
build/apache-cassandra-*.jar, build/apache-cassandra-*.pom,
build/cassandra*.deb, build/cassandra*.rpm")
-remoteDirectory("cassandra/\${JOB_NAME}/\${BUILD_NUMBER}/")
+
remoteDirectory("cassandra/devbranch/Cassandra-devbranch-artifacts/\${BUILD_NUMBER}/\${JOB_NAME}/")
}
}
failOnError(false)
@@ -723,7 +732,7 @@ testTargets.each {
}
compressBuildLog()
logRotator {
-numToKeep(10)
+numToKeep(90)
artifactNumToKeep(10)
}
wrappers {
@@ -782,7 +791,7 @@ testTargets.each {
server('Nightlies') {
transferSet {
sourceFiles("build/test/logs/**")
-
remoteDirectory("cassandra/\${JOB_NAME}/\${BUILD_NUMBER}/")
+
remoteDirectory("cassandra/devbranch/Cassandra-devbranch-${targetName}/\${BUILD_NUMBER}/\${JOB_NAME}/")
}
}
failOnError(false)
@@ -825,7 +834,7 @@ dtestTargets.each {
compressBuildLog()
compressBuildLog()
logRotator {
-numToKeep(10)
+numToKeep(90)
artifactNumToKeep(10)
}
wrappers {
@@ -899,7 +908,7 @@ dtestTargets.each {
server('Nightlies') {
transferSet {
sourceFiles("**/test_stdout.txt.xz,**/ccm_logs.tar.xz")
-
remoteDirectory("cassandra/\${JOB_NAME}/\${BUILD_NUMBER}/")
+
remoteDirectory("cassandra/devbranch/Cassandra-devbranch-${targetName}/\${BUILD_NUMBER}/\${JOB_NAME}/")
}
}
failOnError(false)
@@ -934,7 +943,7 @@ matrixJob('Cassandra-devbranch-cqlsh-tests') {
concurrentBuild()
compressBuildLog()
logRotator {
-numToKeep(10)
+numToKeep(90)
artifactNumToKeep(10)
}
wrappers {
@@ -993,6 +1002,15 @@ matrixJob('Cassandra-devbranch-cqlsh-tests') {
shell('./pylib/cassandra-cqlsh-tests.sh $WORKSPACE')
}
publishers {
+publishOverSsh {
+server('Nightlies') {
+transferSet {
+sourceFiles("**/test_stdout.txt.xz,**/ccm_logs.tar.xz")
+
remoteDirectory("cassandra/devbranch/Cassandra-devbranch-cqlsh-tests/\${BUILD_NUMBER}/\${JOB_NAME}/")
+}
+}
+failOnError(false)
+}
archiveArtifacts {
pattern('**/cqlshlib.xml,**/nosetests.xml, **/*.head')
allowEmpty()
diff --git a/jenkins-dsl/cassandra_pipeline.groovy
b/jenkins-dsl/cassandra_pipeline.groovy
index 1f1d807..9830fc2 100644
--- a/jenkins-dsl/cassandra_pipeline.groovy
+++ b/jenkins-dsl/cassandra_pipeline.groovy
@@ -286,7 +286,7 @@ pipeline {
post {
always {
archiveArtifacts artifacts: 'cassandra-test-report.txt.xz',
fingerprint: true
-
[jira] [Commented] (CASSANDRA-16311) Extend the exclusion of replica filtering protection to other indices instead of just SASI
[ https://issues.apache.org/jira/browse/CASSANDRA-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247809#comment-17247809 ] Stefan Miklosovic commented on CASSANDRA-16311: --- trunk Jenkins run https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/252/ > Extend the exclusion of replica filtering protection to other indices instead > of just SASI > -- > > Key: CASSANDRA-16311 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16311 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SASI >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > There was a check introduced in CASSANDRA-8272 telling if an index is a SASI > index and if it is, replica filtering protection was not triggered. > There might be other custom index implementations which also do not support > filtering protection and they do not have to be SASI indices neccessarily, > however it is not possible to exclude them. > > PR trunk > [https://github.com/apache/cassandra/pull/844] > PR 3.11 > [https://github.com/apache/cassandra/pull/847] > PR 3.0 > https://github.com/apache/cassandra/pull/848 > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
