simges opened a new issue, #67934: URL: https://github.com/apache/doris/issues/67934
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version v3.1.4 ### What's Wrong? <html> <body> <!--StartFragment--><html><head></head><body><h1>[Config] Inconsistent defaults for storage flood stage thresholds between FE and BE</h1><h3>Description</h3><p>Apache Doris defines <code inline="">storage_flood_stage_usage_percent</code> and <code inline="">storage_flood_stage_left_capacity_bytes</code> in both FE and BE configuration. The two components use the same parameter names, but the default value of <code inline="">storage_flood_stage_usage_percent</code> is inconsistent.</p><p>According to the official documentation:</p> Parameter | FE default | BE default -- | -- | -- storage_flood_stage_usage_percent | 95 | 90 storage_flood_stage_left_capacity_bytes | 1 GiB | 1 GiB <p>References:</p><ul><li><p><a href="https://github.com/apache/doris/blob/master/fe/fe-common/src/main/java/org/apache/doris/common/Config.java";>FE Config.java</a></p></li><li><p><a href="https://doris.apache.org/docs/4.x/admin-manual/config/fe-config/";>FE configuration documentation</a></p></li><li><p><a href="https://doris.apache.org/docs/4.x/admin-manual/config/be-config/";>BE configuration documentation</a></p></li></ul><h3>Problem</h3><p>The same-named configuration parameters have different defaults in FE and BE. This can cause users to misunderstand the effective disk flood-stage threshold when configuring a Doris cluster.</p><p>For example, with a disk usage of 92%, the FE and BE have different default behavior:</p><ul><li><p>FE: the disk is below the default 95% usage threshold.</p></li><li><p>BE: the disk is above the default 90% usage threshold.</p></li></ul><p>As a result, BE may enter its flood-stage protection while FE does not consider the disk to have reached the c orresponding usage threshold. Then BE will start returning errors which will result in query failures.</p><p>This can make the behavior of disk protection difficult to predict and complicate cluster configuration and troubleshooting.</p></body></html><!--EndFragment--> </body> ### What You Expected? The default values and descriptions of the corresponding FE and BE parameters should be consistent, or the documentation should clearly explain why they intentionally differ. ### How to Reproduce? - set config::storage_flood_stage_usage_percent 92% as hard-coded. - set storage_flood_stage_left_capacity_bytes to a high value enough for satisy this condition in data_dir.cpp: bool DataDir::reach_capacity_limit(int64_t incoming_data_size) { double used_pct = get_usage(incoming_data_size); int64_t left_bytes = _available_bytes - incoming_data_size; if (used_pct >= config::storage_flood_stage_usage_percent / 100.0 && left_bytes <= config::storage_flood_stage_left_capacity_bytes) { LOG(WARNING) << "reach capacity limit. used pct: " << used_pct << ", left bytes: " << left_bytes << ", path: " << _path; return true; } return false; } - then the backend will return an error. It is still treated as a query execution candidate, though, since exceedLimit continues to return false: public boolean exceedLimit(boolean floodStage) { if (LOG.isDebugEnabled()) { LOG.debug("flood stage: {}, diskAvailableCapacityB: {}, totalCapacityB: {}", floodStage, diskAvailableCapacityB, totalCapacityB); } if (floodStage) { return diskAvailableCapacityB < Config.storage_flood_stage_left_capacity_bytes && this.getUsedPct() > (Config.storage_flood_stage_usage_percent / 100.0); } else { return diskAvailableCapacityB < Config.storage_min_left_capacity_bytes || this.getUsedPct() > (Config.storage_high_watermark_usage_percent / 100.0); } } ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
