[
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849218#comment-17849218
]
Stefan Miklosovic edited comment on CASSANDRA-12937 at 5/24/24 8:36 AM:
------------------------------------------------------------------------
Seems reasonably clean ...
What I have not done is that the idea [~jlewandowski] had with "if some config
parameter is not in cql statement just merge the values from cassandra.yaml"
because it is quite tricky to get that right. We would need to know what values
were specfied and then diffing what is not there and then validating that such
combination makes sense (and if it does not, should we fail otherwise valid CQL
statement just because we happened to merge values from cassandra.yaml and that
combination was not right? I do not think so).
Let's just go with a simple case of "if compression is not specified just take
the defaults from cassandra.yaml" rather then trying to merge the configs ...
Too much of a hassle, might come as an improvement if somebody is really after
that.
I will try to come up with more tests and I think that sometimes next week this
should be all completed and ready for review again.
[CASSANDRA-12937-squashed|https://github.com/instaclustr/cassandra/tree/CASSANDRA-12937-squashed]
{noformat}
java17_pre-commit_tests
✓ j17_build 4m 7s
✓ j17_cqlsh_dtests_py311 7m 11s
✓ j17_cqlsh_dtests_py311_vnode 7m 18s
✓ j17_cqlsh_dtests_py38 6m 58s
✓ j17_cqlsh_dtests_py38_vnode 7m 1s
✓ j17_cqlshlib_cython_tests 7m 26s
✓ j17_cqlshlib_tests 6m 50s
✓ j17_unit_tests 17m 36s
✓ j17_utests_oa 15m 39s
✕ j17_dtests 37m 48s
scrub_test.TestScrub test_standalone_scrub_essential_files_only
topology_test.TestTopology test_movement
✕ j17_dtests_latest 35m 36s
offline_tools_test.TestOfflineTools test_sstableverify
scrub_test.TestScrub test_standalone_scrub_essential_files_only
configuration_test.TestConfiguration test_change_durable_writes
✕ j17_dtests_vnode 35m 11s
scrub_test.TestScrub test_standalone_scrub_essential_files_only
✕ j17_jvm_dtests 28m 15s
✕ j17_jvm_dtests_latest_vnode 27m 59s
org.apache.cassandra.fuzz.harry.integration.model.ConcurrentQuiescentCheckerIntegrationTest
testConcurrentReadWriteWorkload
✕ j17_utests_latest 14m 34s
org.apache.cassandra.tcm.DiscoverySimulationTest discoveryTest
java17_separate_tests
java11_pre-commit_tests
java11_separate_tests
{noformat}
[java17_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4350/workflows/a68e4fb0-bd7a-4758-841c-6b4b0fe22865]
[java17_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4350/workflows/fa57a86d-d120-4304-bbdf-a6cf8fefc4d2]
[java11_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4350/workflows/91afc77c-54fe-4369-9cb9-ababa3568e16]
[java11_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4350/workflows/71add260-9c68-4d87-9d5a-99863a01bb3f]
was (Author: smiklosovic):
Seems reasonably clean ...
What I have not done is that the idea Jacek had with "if some config parameter
is not in cql statement just merge the values from cassandra.yaml" because it
is quite tricky to get that right. We would need to know what values were
specfied and then diffing what is not there and then validating that such
combination makes sense (and if it does not, should we fail otherwise valid CQL
statement just because we happened to merge values from cassandra.yaml and that
combination was not right? I do not think so).
Let's just go with a simple case of "if compression is not specified just take
the defaults from cassandra.yaml" rather then trying to merge the configs ...
Too much of a hassle, might come as an improvement if somebody is really after
that.
I will try to come up with more tests and I think that sometimes next week this
should be all completed and ready for review again.
[CASSANDRA-12937-squashed|https://github.com/instaclustr/cassandra/tree/CASSANDRA-12937-squashed]
{noformat}
java17_pre-commit_tests
✓ j17_build 4m 7s
✓ j17_cqlsh_dtests_py311 7m 11s
✓ j17_cqlsh_dtests_py311_vnode 7m 18s
✓ j17_cqlsh_dtests_py38 6m 58s
✓ j17_cqlsh_dtests_py38_vnode 7m 1s
✓ j17_cqlshlib_cython_tests 7m 26s
✓ j17_cqlshlib_tests 6m 50s
✓ j17_unit_tests 17m 36s
✓ j17_utests_oa 15m 39s
✕ j17_dtests 37m 48s
scrub_test.TestScrub test_standalone_scrub_essential_files_only
topology_test.TestTopology test_movement
✕ j17_dtests_latest 35m 36s
offline_tools_test.TestOfflineTools test_sstableverify
scrub_test.TestScrub test_standalone_scrub_essential_files_only
configuration_test.TestConfiguration test_change_durable_writes
✕ j17_dtests_vnode 35m 11s
scrub_test.TestScrub test_standalone_scrub_essential_files_only
✕ j17_jvm_dtests 28m 15s
✕ j17_jvm_dtests_latest_vnode 27m 59s
org.apache.cassandra.fuzz.harry.integration.model.ConcurrentQuiescentCheckerIntegrationTest
testConcurrentReadWriteWorkload
✕ j17_utests_latest 14m 34s
org.apache.cassandra.tcm.DiscoverySimulationTest discoveryTest
java17_separate_tests
java11_pre-commit_tests
java11_separate_tests
{noformat}
[java17_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4350/workflows/a68e4fb0-bd7a-4758-841c-6b4b0fe22865]
[java17_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4350/workflows/fa57a86d-d120-4304-bbdf-a6cf8fefc4d2]
[java11_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4350/workflows/91afc77c-54fe-4369-9cb9-ababa3568e16]
[java11_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4350/workflows/71add260-9c68-4d87-9d5a-99863a01bb3f]
> Default setting (yaml) for SSTable compression
> ----------------------------------------------
>
> Key: CASSANDRA-12937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12937
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
> Reporter: Michael Semb Wever
> Assignee: Stefan Miklosovic
> Priority: Low
> Labels: AdventCalendar2021
> Fix For: 5.x
>
> Time Spent: 8h 20m
> Remaining Estimate: 0h
>
> In many situations the choice of compression for sstables is more relevant to
> the disks attached than to the schema and data.
> This issue is to add to cassandra.yaml a default value for sstable
> compression that new tables will inherit (instead of the defaults found in
> {{CompressionParams.DEFAULT}}.
> Examples where this can be relevant are filesystems that do on-the-fly
> compression (btrfs, zfs) or specific disk configurations or even specific C*
> versions (see CASSANDRA-10995 ).
> +Additional information for newcomers+
> Some new fields need to be added to {{cassandra.yaml}} to allow specifying
> the field required for defining the default compression parameters. In
> {{DatabaseDescriptor}} a new {{CompressionParams}} field should be added for
> the default compression. This field should be initialized in
> {{DatabaseDescriptor.applySimpleConfig()}}. At the different places where
> {{CompressionParams.DEFAULT}} was used the code should call
> {{DatabaseDescriptor#getDefaultCompressionParams}} that should return some
> copy of configured {{CompressionParams}}.
> Some unit test using {{OverrideConfigurationLoader}} should be used to test
> that the table schema use the new default when a new table is created (see
> CreateTest for some example).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
