zclllyybb commented on issue #64877: URL: https://github.com/apache/doris/issues/64877#issuecomment-4806473522
Breakwater-GitHub-Analysis-Slot: slot_f82afc887ff0 This content is generated by AI for reference only. Initial analysis for apache/doris#64877: The reported behavior is reproducible from the current FE code path, and it is not something that an upgrade to 4.0 would change if the same `dynamic_partition.*` settings are kept. What the current code does: - `dynamic_partition.end = -1` is rejected by design. `DynamicPartitionUtil.checkEnd()` only allows `end = 0` when the table is also an auto-partition table; negative values are invalid. - When `dynamic_partition.create_history_partition` is false, `DynamicPartitionScheduler` starts dynamic partition creation from offset `idx = 0`. - The scheduler loop is inclusive: `idx <= dynamicPartitionProperty.getEnd()`. Therefore `dynamic_partition.end = 0` creates exactly the current-period partition, for this DDL the partition `[2026-06-26 00:00:00, 2026-06-27 00:00:00)`. - I checked the same mechanism on upstream master, branch-3.0, and branch-4.0. Branch-3.0 and branch-4.0 also have a regression case for Auto Partition + Dynamic Partition with `dynamic_partition.end = 0` that expects one partition after table creation. So this looks like a documentation/contract mismatch rather than a random scheduler failure. The 3.x Auto Partition documentation says the example uses `dynamic_partition.end = 0` for "Dynamic Partition No Partition Creation", but the implementation and tests interpret it as "no future partition creation"; the current partition is still created by Dynamic Partition. For the user's question: - Upgrading to 4.0 does not make `dynamic_partition.end = 0` create zero partitions. The branch-4.0 code still starts from `idx = 0` and loops through `end` inclusively. - If the goal is "Auto Partition creates partitions only when data arrives, and old partitions are recycled automatically", the 4.x path is to use `partition.retention_count` on an AUTO RANGE PARTITION table instead of Dynamic Partition. In branch-4.0 this path skips dynamic add-partition generation and only recycles historical auto partitions. It also cannot be used together with `dynamic_partition.*` properties, so the dynamic partition properties must be removed. - On 3.x, I did not find an equivalent `partition.retention_count` path in branch-3.0/branch-3.1. The practical choices are to accept the single current-period partition, disable Dynamic Partition and manage lifecycle separately, or upgrade to a version that supports `partition.retention_count` if that lifecycle model fits. Suggested maintainer action: 1. If the intended contract is "end=0 means no future partition creation", update the 3.x Auto Partition docs/example wording from "no partition creation" to "no future partition creation; the current-period partition may still be created by Dynamic Partition". 2. If the intended contract is actually "Auto + Dynamic with end=0 should create zero dynamic partitions", this needs a FE behavior change in the dynamic scheduler, for example skipping add-partition generation for auto-partition tables when `create_history_partition=false` and `end=0`, plus updating the existing regression test that currently expects one partition. Useful additional information from the reporter would be: - Exact Doris version, for example `select version();`, not only "doris version 3". - Whether they require zero partitions immediately after `CREATE TABLE`, or only want to avoid future pre-created partitions. - Outputs of `SHOW PARTITIONS FROM testdb.dws_province_sale_di;`, `SHOW DYNAMIC PARTITION TABLES;`, and `SHOW CREATE TABLE testdb.dws_province_sale_di;` after creation and after one dynamic partition scheduler interval. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
