This may be an off-base comparison, but this reminds me of struggles we've had getting to 0 failing unit tests before and the debates on fencing off a snapshot of the current "failure set" so you can have a set point where no further degradation is allowed in a primary data set.
All of which is to say - maybe at the end of the spreadsheet, we take "unclaimed" items and move them to their own InternalConfig.java or something and add an ant target that a) disallows further addition to InternalConfig.java w/out throwing an error / needing whitelist update, and b) disallows further regression in the Config.java <-> cassandra.yaml relationship for non-annotated fields. That way we can at least halt the progression of the disease even if we're stymied on cleaning up some of the existing symptoms. On Mon, Jan 27, 2025, at 1:38 PM, Štefan Miklošovič wrote: > Indeed, we need to balance that and thoughtfully choose what is going to be > added and what not. However, we should not hide something which is meant to > be tweaked by a user. The config is intimidating mostly because everything is > just in one file. I merely remember discussions a few years ago which were > about splitting cassandra.yaml into multiple files which would be focused > just on one subsystem / would cover some logically isolated domain. > > Anyway, I think the main goal of this effort for now would be to at least map > where we are at. Some of them are genuinely missing. E.g. guardrails, how is > a user meant to know about that if it is not even documented ... > > On Mon, Jan 27, 2025 at 6:16 PM Chris lohfink <cnl...@gmail.com> wrote: >> Might be a bit of a balance between exposing what people actually are likely >> to need to modify vs having a super intimidating config file. It's already >> nearly 2000 lines. Personally I'd rather see some auto-documentation or >> something that's in the docs >> <https://cassandra.apache.org/doc/latest/cassandra/managing/configuration/cass_yaml_file.html> >> than an effort to manually add another 1000 lines. >> >> Chris >> >> On Fri, Jan 24, 2025 at 9:41 AM Dmitry Konstantinov <netud...@gmail.com> >> wrote: >>> Maybe I missed some patterns but it looks like a pretty good estimation, I >>> did like 10 random checks manually to verify :-) >>> I will try to make an ant target with a similar logic (hopefully, during >>> the weekend) >>> I will create a ticket to track this activity (to share attachments there >>> to not overload the thread with such outputs in future). >>> >>> On Fri, 24 Jan 2025 at 15:37, Štefan Miklošovič <smikloso...@apache.org> >>> wrote: >>>> Oh my god, 112? :DD I was thinking it would be less than 10. >>>> >>>> Anyway, I think we need to integrate this to some ant target. If you >>>> expanded on this, that would be great. >>>> >>>> On Fri, Jan 24, 2025 at 4:31 PM Dmitry Konstantinov <netud...@gmail.com> >>>> wrote: >>>>> A very primitive implementation of the 1st idea below: >>>>> >>>>> String configUrl = >>>>> "file:///Users/dmitry/IdeaProjects/cassandra-trunk/conf/cassandra.yaml"; >>>>> Field[] allFields = Config.class.getFields(); >>>>> List<String> topLevelPropertyNames = new ArrayList<>(); >>>>> for(Field field : allFields) >>>>> { >>>>> if (!Modifier.*isStatic*(field.getModifiers())) >>>>> { >>>>> topLevelPropertyNames.add(field.getName()); >>>>> } >>>>> } >>>>> >>>>> URL url = new URL(configUrl); >>>>> List<String> lines = Files.*readAllLines*(Paths.*get*(url.toURI())); >>>>> >>>>> int missedCount = 0; >>>>> for (String propertyName : topLevelPropertyNames) >>>>> { >>>>> boolean found = false; >>>>> for (String line : lines) >>>>> { >>>>> if (line.startsWith(propertyName + ":") >>>>> || line.startsWith("#" + propertyName + ":") >>>>> || line.startsWith("# " + propertyName + ":")) { >>>>> found = true; >>>>> break; >>>>> } >>>>> } >>>>> if (!found) >>>>> { >>>>> missedCount++; >>>>> System.*out*.println(propertyName); >>>>> } >>>>> } >>>>> System.*out*.println("Total missed:" + missedCount); >>>>> >>>>> It prints the following config property names which are defined in >>>>> Config.java but not present as "property" or "# property " in a file: >>>>> permissions_cache_max_entries >>>>> roles_cache_max_entries >>>>> credentials_cache_max_entries >>>>> auto_bootstrap >>>>> force_new_prepared_statement_behaviour >>>>> use_deterministic_table_id >>>>> repair_request_timeout >>>>> stream_transfer_task_timeout >>>>> cms_await_timeout >>>>> cms_default_max_retries >>>>> cms_default_retry_backoff >>>>> epoch_aware_debounce_inflight_tracker_max_size >>>>> metadata_snapshot_frequency >>>>> available_processors >>>>> repair_session_max_tree_depth >>>>> use_offheap_merkle_trees >>>>> internode_max_message_size >>>>> native_transport_max_message_size >>>>> native_transport_max_request_data_in_flight_per_ip >>>>> native_transport_max_request_data_in_flight >>>>> native_transport_receive_queue_capacity >>>>> min_free_space_per_drive >>>>> max_space_usable_for_compactions_in_percentage >>>>> reject_repair_compaction_threshold >>>>> concurrent_index_builders >>>>> max_streaming_retries >>>>> commitlog_max_compression_buffers_in_pool >>>>> max_mutation_size >>>>> dynamic_snitch >>>>> failure_detector >>>>> use_creation_time_for_hint_ttl >>>>> key_cache_migrate_during_compaction >>>>> key_cache_invalidate_after_sstable_deletion >>>>> paxos_cache_size >>>>> file_cache_round_up >>>>> disk_optimization_estimate_percentile >>>>> disk_optimization_page_cross_chance >>>>> purgeable_tobmstones_metric_granularity >>>>> windows_timer_interval >>>>> otc_coalescing_strategy >>>>> otc_coalescing_window_us >>>>> otc_coalescing_enough_coalesced_messages >>>>> otc_backlog_expiration_interval_ms >>>>> scripted_user_defined_functions_enabled >>>>> user_defined_functions_threads_enabled >>>>> allow_insecure_udfs >>>>> allow_extra_insecure_udfs >>>>> user_defined_functions_warn_timeout >>>>> user_defined_functions_fail_timeout >>>>> user_function_timeout_policy >>>>> back_pressure_enabled >>>>> back_pressure_strategy >>>>> repair_command_pool_full_strategy >>>>> repair_command_pool_size >>>>> block_for_peers_timeout_in_secs >>>>> block_for_peers_in_remote_dcs >>>>> skip_stream_disk_space_check >>>>> snapshot_on_repaired_data_mismatch >>>>> validation_preview_purge_head_start >>>>> initial_range_tombstone_list_allocation_size >>>>> range_tombstone_list_growth_factor >>>>> snapshot_on_duplicate_row_detection >>>>> check_for_duplicate_rows_during_reads >>>>> check_for_duplicate_rows_during_compaction >>>>> autocompaction_on_startup_enabled >>>>> auto_optimise_inc_repair_streams >>>>> auto_optimise_full_repair_streams >>>>> auto_optimise_preview_repair_streams >>>>> consecutive_message_errors_threshold >>>>> internode_error_reporting_exclusions >>>>> compact_tables_enabled >>>>> vector_type_enabled >>>>> intersect_filtering_query_warned >>>>> intersect_filtering_query_enabled >>>>> streaming_slow_events_log_timeout >>>>> repair_state_expires >>>>> repair_state_size >>>>> paxos_variant >>>>> skip_paxos_repair_on_topology_change >>>>> paxos_purge_grace_period >>>>> paxos_on_linearizability_violations >>>>> paxos_state_purging >>>>> paxos_repair_enabled >>>>> paxos_topology_repair_no_dc_checks >>>>> paxos_topology_repair_strict_each_quorum >>>>> skip_paxos_repair_on_topology_change_keyspaces >>>>> paxos_contention_wait_randomizer >>>>> paxos_contention_min_wait >>>>> paxos_contention_max_wait >>>>> paxos_contention_min_delta >>>>> paxos_repair_parallelism >>>>> sstable_read_rate_persistence_enabled >>>>> client_request_size_metrics_enabled >>>>> max_top_size_partition_count >>>>> max_top_tombstone_partition_count >>>>> min_tracked_partition_size >>>>> min_tracked_partition_tombstone_count >>>>> top_partitions_enabled >>>>> severity_during_decommission >>>>> progress_barrier_min_consistency_level >>>>> progress_barrier_default_consistency_level >>>>> progress_barrier_timeout >>>>> progress_barrier_backoff >>>>> discovery_timeout >>>>> unsafe_tcm_mode >>>>> cql_start_time >>>>> native_transport_throw_on_overload >>>>> native_transport_queue_max_item_age_threshold >>>>> native_transport_min_backoff_on_queue_overload >>>>> native_transport_max_backoff_on_queue_overload >>>>> native_transport_timeout >>>>> enforce_native_deadline_for_hints >>>>> Total missed:112 >>>>> >>>>> >>>>> On Fri, 24 Jan 2025 at 15:10, Štefan Miklošovič <smikloso...@apache.org> >>>>> wrote: >>>>>> It should also work the other way around. If there is a property which >>>>>> is commented out in yaml and it is not in Config.java, that should fail >>>>>> as well. If it is not commented out and it is not in Config.java, that >>>>>> will fail in runtime as it fails on unrecognized property. >>>>>> >>>>>> This will be used in practice very rarely as we seldom remove the >>>>>> properties in Config but if we do and a property is commented out, we >>>>>> should not ship a dead property name, even commented out. >>>>>> >>>>>> On Fri, Jan 24, 2025 at 3:51 PM Paulo Motta <pa...@apache.org> wrote: >>>>>>> > > If "# my_cool_property: true" is NOT in cassandra.yaml, we might >>>>>>> > indeed add it, also commented out. I think it would be quite easy to >>>>>>> > check against yaml if there is a line starting on "# >>>>>>> > my_cool_property" or just on "my_cool_property". Both cases would >>>>>>> > satisfy the check. >>>>>>> >>>>>>> Makes sense, I think this would be good to have as a lint or test to >>>>>>> easily catch overlooks during review. >>>>>>> >>>>>>> On Fri, Jan 24, 2025 at 9:44 AM Štefan Miklošovič >>>>>>> <smikloso...@apache.org> wrote: >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Jan 24, 2025 at 3:27 PM Paulo Motta <pa...@apache.org> wrote: >>>>>>>>> > from time to time I see configuration properties in Config.java and >>>>>>>>> > they are clearly not in cassandra.yaml. Not every property in >>>>>>>>> > Config is in cassandra.yaml. I would like to know if there is some >>>>>>>>> > specific reason behind that. >>>>>>>>> >>>>>>>>> I think one of the original reasons was to "hide" advanced configs >>>>>>>>> that are not meant to be updated, unless in very niche circumstances. >>>>>>>>> However I think this has been extrapolated to non-advanced settings. >>>>>>>>> >>>>>>>>> > Question related to that is if we could not have a build-time check >>>>>>>>> > that all properties in Config have to be in cassandra.yaml and fail >>>>>>>>> > the build if a property in Config does not have its counterpart in >>>>>>>>> > yaml. >>>>>>>>> >>>>>>>>> Are you saying every configuration property should be commented-out, >>>>>>>>> or do you think that every Config property should be specified in >>>>>>>>> cassandra.yaml with their default uncomented ? One issue with that is >>>>>>>>> that you could cause user confusion if you "reveal" a niche/advanced >>>>>>>>> config that is not meant to be updated. I think this would be >>>>>>>>> addressed by the @HiddenInYaml flag you are proposing in a later post. >>>>>>>> >>>>>>>> Yes, then can stay hidden, but we should annotate it with @Hidden or >>>>>>>> similar. As of now, if that property is not in yaml, we just don't >>>>>>>> know if it was forgotten to be added or if we have not added it on >>>>>>>> purpose. >>>>>>>> >>>>>>>> They can keep being commented out if they currently are. Imagine a >>>>>>>> property in Config.java >>>>>>>> >>>>>>>> public boolean my_cool_property = true; >>>>>>>> >>>>>>>> and then this in cassandra.yaml >>>>>>>> >>>>>>>> # my_cool_property: true >>>>>>>> >>>>>>>> It is completely ok. >>>>>>>> >>>>>>>> If "# my_cool_property: true" is NOT in cassandra.yaml, we might >>>>>>>> indeed add it, also commented out. I think it would be quite easy to >>>>>>>> check against yaml if there is a line starting on "# my_cool_property" >>>>>>>> or just on "my_cool_property". Both cases would satisfy the check. >>>>>>>> >>>>>>>> >>>>>>>>> > There are dozens of properties in Config and I have a strong >>>>>>>>> > suspicion that we missed to publish some to yaml so users do not >>>>>>>>> > even know such a property exists and as of now we do not even know >>>>>>>>> > which they are. >>>>>>>>> >>>>>>>>> I believe this is a problem. I think most properties should be in >>>>>>>>> cassandra.yaml, unless they are very advanced or not meant to be >>>>>>>>> updated. >>>>>>>>> >>>>>>>>> Another tangential issue is that there are features/settings that >>>>>>>>> don't even have a Config entry, but are just controlled by JVM >>>>>>>>> properties. >>>>>>>>> >>>>>>>>> I think that we should attempt to unify Config and jvm properties >>>>>>>>> under a predictable structure. For example, if there is a YAML config >>>>>>>>> enable_user_defined_functions, then there should be a respective JVM >>>>>>>>> flag -Dcassandra.enable_user_defined_functions, and vice versa. >>>>>>>> >>>>>>>> Yeah, good idea. >>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Jan 24, 2025 at 9:16 AM Štefan Miklošovič >>>>>>>>> <smikloso...@apache.org> wrote: >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> from time to time I see configuration properties in Config.java and >>>>>>>>>> they are clearly not in cassandra.yaml. Not every property in Config >>>>>>>>>> is in cassandra.yaml. I would like to know if there is some specific >>>>>>>>>> reason behind that. >>>>>>>>>> >>>>>>>>>> Question related to that is if we could not have a build-time check >>>>>>>>>> that all properties in Config have to be in cassandra.yaml and fail >>>>>>>>>> the build if a property in Config does not have its counterpart in >>>>>>>>>> yaml. >>>>>>>>>> >>>>>>>>>> There are dozens of properties in Config and I have a strong >>>>>>>>>> suspicion that we missed to publish some to yaml so users do not >>>>>>>>>> even know such a property exists and as of now we do not even know >>>>>>>>>> which they are. >>>>> >>>>> >>>>> -- >>>>> Dmitry Konstantinov >>> >>> >>> -- >>> Dmitry Konstantinov
