[jira] [Commented] (CASSANDRA-18533) Move format-specific sstable options into the format configuration
[
https://issues.apache.org/jira/browse/CASSANDRA-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783923#comment-17783923
]
Maxwell Guo commented on CASSANDRA-18533:
-
initial pr for this ,may need to do some optimize, but I think we can make this
matter move forward a step further
[pr|https://github.com/apache/cassandra/pull/2879]
[java17|https://app.circleci.com/pipelines/github/Maxwell-Guo/cassandra/519/workflows/519686e7-d2b6-458a-a325-d04fa43842f3]
[java11|https://app.circleci.com/pipelines/github/Maxwell-Guo/cassandra/519/workflows/10d096f7-baf7-45ea-b8b5-49c6923881c8]
> Move format-specific sstable options into the format configuration
> --
>
> Key: CASSANDRA-18533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18533
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
>
> This mainly concerns cassandra yaml settings:
> - {{column_index_size}}, which should also be renamed to
> {{row_index_granularity}}
> - {{column_index_cache_size}}
> - {{index_summary_capacity}}
> - {{index_summary_resize_interval}}
> and possibly
> - {{key_cache_size}}, {{key_cache_keys_to_save}}, {{key_cache_save_period}},
> {{key_cache_migrate_during_compaction}}
> - {{sstable_preemptive_open_interval}}
> Existing settings should be deprecated but still picked up if defined.
> At this point we will not consider table-level options that make better sense
> as format parameters ({{min/max_index_interval}}, {{bloom_filter_fp_chance}},
> {{crc_check_chance}} and possibly {{compression}}), because we do not yet
> support per-table format selection/configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18533) Move format-specific sstable options into the format configuration
[
https://issues.apache.org/jira/browse/CASSANDRA-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782190#comment-17782190
]
Maxwell Guo commented on CASSANDRA-18533:
-
[~blambov]I have update the upper comment , would mind to take a look again ?
thanks.
> Move format-specific sstable options into the format configuration
> --
>
> Key: CASSANDRA-18533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18533
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
>
> This mainly concerns cassandra yaml settings:
> - {{column_index_size}}, which should also be renamed to
> {{row_index_granularity}}
> - {{column_index_cache_size}}
> - {{index_summary_capacity}}
> - {{index_summary_resize_interval}}
> and possibly
> - {{key_cache_size}}, {{key_cache_keys_to_save}}, {{key_cache_save_period}},
> {{key_cache_migrate_during_compaction}}
> - {{sstable_preemptive_open_interval}}
> Existing settings should be deprecated but still picked up if defined.
> At this point we will not consider table-level options that make better sense
> as format parameters ({{min/max_index_interval}}, {{bloom_filter_fp_chance}},
> {{crc_check_chance}} and possibly {{compression}}), because we do not yet
> support per-table format selection/configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18533) Move format-specific sstable options into the format configuration
[
https://issues.apache.org/jira/browse/CASSANDRA-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782193#comment-17782193
]
Branimir Lambov commented on CASSANDRA-18533:
-
I would keep it simple and not add a common settings entry under options. If
necessary, the user can copy the value to both.
> Move format-specific sstable options into the format configuration
> --
>
> Key: CASSANDRA-18533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18533
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
>
> This mainly concerns cassandra yaml settings:
> - {{column_index_size}}, which should also be renamed to
> {{row_index_granularity}}
> - {{column_index_cache_size}}
> - {{index_summary_capacity}}
> - {{index_summary_resize_interval}}
> and possibly
> - {{key_cache_size}}, {{key_cache_keys_to_save}}, {{key_cache_save_period}},
> {{key_cache_migrate_during_compaction}}
> - {{sstable_preemptive_open_interval}}
> Existing settings should be deprecated but still picked up if defined.
> At this point we will not consider table-level options that make better sense
> as format parameters ({{min/max_index_interval}}, {{bloom_filter_fp_chance}},
> {{crc_check_chance}} and possibly {{compression}}), because we do not yet
> support per-table format selection/configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18533) Move format-specific sstable options into the format configuration
[
https://issues.apache.org/jira/browse/CASSANDRA-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782184#comment-17782184
]
Branimir Lambov commented on CASSANDRA-18533:
-
1. Yes, precisely.
2. The key cache is constructed in a completely separate portion of the code,
isn't it? Ignore the key cache settings (except migration), I don't think
changing this is something we can do at the moment.
3. Although it is not at the moment, the row index granularity in particular
should be a table-level property -- there's no real reason to use one setting
for all tables, and there's an advantage to be had by making it configurable.
However, things like the key cache size or index summary capacity are something
to be shared, not just between tables but also potentially between formats; I
don't want to get into a complicated solution for this, I would either ignore
any table-level modification for these (with a warning) or check that the value
is the same among all tables. This, along with format variations (e.g.
"bti-fast"), is also out of scope for this ticket.
> Move format-specific sstable options into the format configuration
> --
>
> Key: CASSANDRA-18533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18533
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
>
> This mainly concerns cassandra yaml settings:
> - {{column_index_size}}, which should also be renamed to
> {{row_index_granularity}}
> - {{column_index_cache_size}}
> - {{index_summary_capacity}}
> - {{index_summary_resize_interval}}
> and possibly
> - {{key_cache_size}}, {{key_cache_keys_to_save}}, {{key_cache_save_period}},
> {{key_cache_migrate_during_compaction}}
> - {{sstable_preemptive_open_interval}}
> Existing settings should be deprecated but still picked up if defined.
> At this point we will not consider table-level options that make better sense
> as format parameters ({{min/max_index_interval}}, {{bloom_filter_fp_chance}},
> {{crc_check_chance}} and possibly {{compression}}), because we do not yet
> support per-table format selection/configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18533) Move format-specific sstable options into the format configuration
[
https://issues.apache.org/jira/browse/CASSANDRA-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782170#comment-17782170
]
Maxwell Guo commented on CASSANDRA-18533:
-
ping [~blambov]
> Move format-specific sstable options into the format configuration
> --
>
> Key: CASSANDRA-18533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18533
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
>
> This mainly concerns cassandra yaml settings:
> - {{column_index_size}}, which should also be renamed to
> {{row_index_granularity}}
> - {{column_index_cache_size}}
> - {{index_summary_capacity}}
> - {{index_summary_resize_interval}}
> and possibly
> - {{key_cache_size}}, {{key_cache_keys_to_save}}, {{key_cache_save_period}},
> {{key_cache_migrate_during_compaction}}
> - {{sstable_preemptive_open_interval}}
> Existing settings should be deprecated but still picked up if defined.
> At this point we will not consider table-level options that make better sense
> as format parameters ({{min/max_index_interval}}, {{bloom_filter_fp_chance}},
> {{crc_check_chance}} and possibly {{compression}}), because we do not yet
> support per-table format selection/configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18533) Move format-specific sstable options into the format configuration
[
https://issues.apache.org/jira/browse/CASSANDRA-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779554#comment-17779554
]
Maxwell Guo commented on CASSANDRA-18533:
-
[~blambov]Hi, I have some questions here.
1.we are going to put the format-specific options into the [option
map|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/format/AbstractSSTableFormat.java#L27]
here , am I right ?
2.it seems key_cache_size, key_cache_keys_to_save, key_cache_save_period, are
not sstable level but just for key cache which is db level, should I remove
them?
3.I am a little confusing now for table level configuration in CASSANDRA-18534
and options here. if we are going to define sstable option like [this
|https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L1029],then
we may mix the table level options in CASSANDRA-18534 and the options here,
feeling not clear as we mix the table level configuration together with those
non-table level, should we separate them at yaml level, though the original
yaml do not define a configuration as table level or not .if we define sstable
format like this :
{code:java}
sstable:
selected_format: big
format.options:
- bit-fast:
row_index_granularity: 1kiB
bloom_filter_fp_chance: 0.01
- big-small:
row_index_granularity: 1kiB
bloom_filter_fp_chance: 0.01
{code}
what about change to
{code:java}
{code:java}
sstable:
selected_format: big
format.options:
- bit-fast:
row_index_granularity: 1kiB
table-level:
bloom_filter_fp_chance: 0.01
{code}
{code}
> Move format-specific sstable options into the format configuration
> --
>
> Key: CASSANDRA-18533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18533
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
>
> This mainly concerns cassandra yaml settings:
> - {{column_index_size}}, which should also be renamed to
> {{row_index_granularity}}
> - {{column_index_cache_size}}
> - {{index_summary_capacity}}
> - {{index_summary_resize_interval}}
> and possibly
> - {{key_cache_size}}, {{key_cache_keys_to_save}}, {{key_cache_save_period}},
> {{key_cache_migrate_during_compaction}}
> - {{sstable_preemptive_open_interval}}
> Existing settings should be deprecated but still picked up if defined.
> At this point we will not consider table-level options that make better sense
> as format parameters ({{min/max_index_interval}}, {{bloom_filter_fp_chance}},
> {{crc_check_chance}} and possibly {{compression}}), because we do not yet
> support per-table format selection/configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18533) Move format-specific sstable options into the format configuration
[
https://issues.apache.org/jira/browse/CASSANDRA-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767594#comment-17767594
]
Branimir Lambov commented on CASSANDRA-18533:
-
Absolutely.
> Move format-specific sstable options into the format configuration
> --
>
> Key: CASSANDRA-18533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18533
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Branimir Lambov
>Priority: Normal
>
> This mainly concerns cassandra yaml settings:
> - {{column_index_size}}, which should also be renamed to
> {{row_index_granularity}}
> - {{column_index_cache_size}}
> - {{index_summary_capacity}}
> - {{index_summary_resize_interval}}
> and possibly
> - {{key_cache_size}}, {{key_cache_keys_to_save}}, {{key_cache_save_period}},
> {{key_cache_migrate_during_compaction}}
> - {{sstable_preemptive_open_interval}}
> Existing settings should be deprecated but still picked up if defined.
> At this point we will not consider table-level options that make better sense
> as format parameters ({{min/max_index_interval}}, {{bloom_filter_fp_chance}},
> {{crc_check_chance}} and possibly {{compression}}), because we do not yet
> support per-table format selection/configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18533) Move format-specific sstable options into the format configuration
[
https://issues.apache.org/jira/browse/CASSANDRA-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767388#comment-17767388
]
Maxwell Guo commented on CASSANDRA-18533:
-
[~blambov] I assigin this to myself , seems have some relation with
CASSANDRA-18534.
it's that ok ?
> Move format-specific sstable options into the format configuration
> --
>
> Key: CASSANDRA-18533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18533
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Branimir Lambov
>Priority: Normal
>
> This mainly concerns cassandra yaml settings:
> - {{column_index_size}}, which should also be renamed to
> {{row_index_granularity}}
> - {{column_index_cache_size}}
> - {{index_summary_capacity}}
> - {{index_summary_resize_interval}}
> and possibly
> - {{key_cache_size}}, {{key_cache_keys_to_save}}, {{key_cache_save_period}},
> {{key_cache_migrate_during_compaction}}
> - {{sstable_preemptive_open_interval}}
> Existing settings should be deprecated but still picked up if defined.
> At this point we will not consider table-level options that make better sense
> as format parameters ({{min/max_index_interval}}, {{bloom_filter_fp_chance}},
> {{crc_check_chance}} and possibly {{compression}}), because we do not yet
> support per-table format selection/configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
