github-actions[bot] commented on code in PR #65219:
URL: https://github.com/apache/doris/pull/65219#discussion_r3576937851
##########
fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java:
##########
@@ -237,6 +243,17 @@ public class PropertyAnalyzer {
public static final String ENABLE_UNIQUE_KEY_MERGE_ON_WRITE =
"enable_unique_key_merge_on_write";
public static final String ENABLE_UNIQUE_KEY_SKIP_BITMAP_COLUMN =
"enable_unique_key_skip_bitmap_column";
private static final Logger LOG =
LogManager.getLogger(PropertyAnalyzer.class);
+ /** Formatter for cooldown/baseTime strings that carry an explicit timezone
+ * offset (e.g. "+00:00"). Accepts optional fractional seconds up to
+ * nanosecond precision so that TIMESTAMPTZ(p) values with 0 ≤ p ≤ 6
Review Comment:
This formatter keeps the JDK default SMART resolver, so invalid
offset-bearing dates can be normalized into a valid instant. For example, with
this exact formatter on JDK 17, `2027-02-29 00:00:00+00:00` parses as
`2027-02-28T00:00Z`; the old `DateLiteralUtils` path is strict and would have
fallen back to `MAX_COOLDOWN_TIME_MS`. Please build the offset formatter with
strict calendar validation, e.g. `uuuu-MM-dd HH:mm:ss` plus
`.withResolverStyle(ResolverStyle.STRICT)`, and add a negative test for an
invalid future offset date.
##########
fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java:
##########
@@ -409,10 +426,24 @@ public static DataProperty
analyzeDataProperty(Map<String, String> properties, f
}
} else if (key.equalsIgnoreCase(PROPERTIES_STORAGE_COOLDOWN_TIME))
{
try {
- DateLiteral dateLiteral =
DateLiteralUtils.createDateLiteral(value,
- ScalarType.getDefaultDateType(Type.DATETIME));
- cooldownTimestamp =
dateLiteral.unixTimestamp(TimeUtils.getTimeZone());
- } catch (AnalysisException e) {
+ // A value with an explicit timezone offset (e.g.
+ // "2027-01-01 00:00:00+00:00" or, for TIMESTAMPTZ(6),
+ // "2027-01-01 00:00:00.000000+00:00") represents an
+ // unambiguous UTC instant. Parse it directly as a
+ // ZonedDateTime and convert to epoch millis, bypassing
+ // the DATETIME-based DateLiteralUtils.createDateLiteral()
+ // path which uses Instant.now() for the initial offset
Review Comment:
This branch catches every value ending in `+/-HH:MM`, but the new formatter
only accepts the full `yyyy-MM-dd HH:mm:ss[.fraction]+HH:MM` shape. Previously
the legacy parser stripped the offset and accepted forms like `2027-01-01
00:00+00:00` (seconds default to 0) and compact datetime keys such as
`20270101000000+00:00`; now they throw here and the cooldown becomes
`MAX_COOLDOWN_TIME_MS`. Either make the instant-aware parser cover the same
accepted shapes, or only route the generated TIMESTAMPTZ shape through this
narrower parser.
##########
fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java:
##########
@@ -511,12 +589,13 @@ private static String convertToUtcTimestamp(Column
column, String border, TimeZo
}
private Range<PartitionKey> getClosedRange(Database db, OlapTable
olapTable, Column partitionColumn,
- String partitionFormat, String lowerBorderOfReservedHistory,
String upperBorderOfReservedHistory) {
+ String partitionFormat, String lowerBorderOfReservedHistory,
String upperBorderOfReservedHistory,
+ TimeZone borderTimeZone) {
Range<PartitionKey> reservedHistoryPartitionKeyRange = null;
lowerBorderOfReservedHistory = convertToUtcTimestamp(partitionColumn,
lowerBorderOfReservedHistory,
-
olapTable.getTableProperty().getDynamicPartitionProperty().getTimeZone());
+ borderTimeZone);
upperBorderOfReservedHistory = convertToUtcTimestamp(partitionColumn,
upperBorderOfReservedHistory,
-
olapTable.getTableProperty().getDynamicPartitionProperty().getTimeZone());
+ borderTimeZone);
PartitionValue lowerBorderPartitionValue = new
PartitionValue(lowerBorderOfReservedHistory);
Review Comment:
`getAddPartitionOp()` captures its `DynamicPartitionTimeContext` earlier in
the pass, and this drop path captures a second one. If an hourly TIMESTAMPTZ
table crosses a UTC hour between those calls, add generation can still build an
add op for a missing history partition that is already outside the later drop
cutoff; the drop phase cannot remove a partition that is not present yet, and
the stale add is applied afterward, leaving an expired partition until the next
scheduler interval. Capture one time context per table scheduling pass and pass
it into both add and drop generation so the whole pass uses one UTC instant.
##########
fe/fe-core/src/test/java/org/apache/doris/catalog/DynamicPartitionTableTest.java:
##########
@@ -2095,15 +2094,29 @@ public void testTimeStampTzDynamicPartition() throws
Exception {
Assert.assertTrue("Upper key must be UTC with +00:00 suffix: "
+ upperStr,
upperStr.contains("+00:00"));
- // With time_zone = "+00:00", partition boundaries must be
midnight UTC
- // (hour = 00). If the scheduler incorrectly used session
timezone
- // (Asia/Shanghai, UTC+8), the hour would be 16 instead of 00.
+ // Partition boundaries must be at UTC midnight (hour=00)
+ // regardless of time_zone.
String lowerHour = lowerStr.substring(11, 13);
- Assert.assertEquals("Lower bound hour should be 00 (midnight
UTC), proving"
- + " dynamic_partition.time_zone was used: " + lowerStr,
+ Assert.assertEquals("Lower bound must be UTC midnight (00): "
+ lowerStr,
"00", lowerHour);
+ String upperHour = upperStr.substring(11, 13);
+ Assert.assertEquals("Upper bound must be UTC midnight (00): "
+ upperStr,
+ "00", upperHour);
}
+ // Verify the current partition (idx=0) name matches the UTC
Review Comment:
This samples UTC after scheduling and then searches the whole
seven-partition table for the expected/current-or-previous name. That can pass
while validating the wrong partition: with `start=-3,end=3`, a regression that
names idx=0 from the configured timezone can still leave an adjacent historical
partition with the expected UTC name. Please derive the generated idx=0
partition from the stored ranges and assert that partition's name and lower
bound directly.
##########
fe/fe-core/src/test/java/org/apache/doris/catalog/DynamicPartitionTableTest.java:
##########
@@ -2313,4 +2514,805 @@ public void
testAutoPartitionRetentionTimestampTzCutoffNormalized() throws Excep
connectContext.getSessionVariable().setTimeZone(originalSessionTz);
}
}
+
+ @Test
+ public void testTimeStampTzGetHistoricalPartitionsRangeBased() throws
Exception {
+ // Verify that getHistoricalPartitions correctly identifies the current
+ // partition by range lower bound (currentUtcBorder) rather than by
name.
+ // When the configured timezone (e.g. Asia/Shanghai) is a day ahead of
UTC,
+ // the scheduler computes nowPartitionName from the TZ date (e.g.
"p20260707")
+ // while currentUtcBorder is from the UTC date ("2026-07-06
00:00:00+00:00").
+ // Name-based comparison would fail to exclude the real current
partition
+ // (p20260706), treating it as historical. Range-based comparison
correctly
+ // excludes it by matching the stored partition lower bound.
+
+ String createSql = "CREATE TABLE test.`tstz_hist_parts` (\n"
+ + " `k1` TIMESTAMPTZ NULL COMMENT \"\",\n"
+ + " `k2` int NULL COMMENT \"\"\n"
+ + ") ENGINE=OLAP\n"
+ + "DUPLICATE KEY(`k1`, `k2`)\n"
+ + "PARTITION BY RANGE(`k1`)\n"
+ + "()\n"
+ + "DISTRIBUTED BY HASH(`k2`) BUCKETS 3\n"
+ + "PROPERTIES (\n"
+ + "\"replication_num\" = \"1\",\n"
+ + "\"dynamic_partition.enable\" = \"true\",\n"
+ + "\"dynamic_partition.start\" = \"-3\",\n"
+ + "\"dynamic_partition.end\" = \"3\",\n"
+ + "\"dynamic_partition.create_history_partition\" =
\"true\",\n"
+ + "\"dynamic_partition.time_unit\" = \"day\",\n"
+ + "\"dynamic_partition.prefix\" = \"p\",\n"
+ + "\"dynamic_partition.buckets\" = \"1\",\n"
+ + "\"dynamic_partition.time_zone\" = \"Asia/Shanghai\"\n"
+ + ");";
+ createTable(createSql);
+
+ Database db =
Env.getCurrentInternalCatalog().getDbOrAnalysisException("test");
+ OlapTable table = (OlapTable)
db.getTableOrAnalysisException("tstz_hist_parts");
+
+ Env.getCurrentEnv().getDynamicPartitionScheduler()
+ .executeDynamicPartitionFirstTime(db.getId(), table.getId());
+
+ int totalPartitions = table.getPartitionNames().size();
+ Assert.assertEquals(7, totalPartitions);
+
+ RangePartitionInfo info = (RangePartitionInfo)
table.getPartitionInfo();
+ DynamicPartitionProperty prop =
table.getTableProperty().getDynamicPartitionProperty();
+ String partitionFormat = DynamicPartitionUtil.getPartitionFormat(
+ info.getPartitionColumns().get(0));
+ ZonedDateTime utcNow = ZonedDateTime.now(ZoneOffset.UTC);
+ // getPartitionRangeString with DATETIME_FORMAT returns "yyyy-MM-dd
HH:mm:ss"
+ // (e.g. "2026-07-06 00:00:00"). convertToUtcTimestamp appends
"+00:00".
+ String utcBorderDateTime =
DynamicPartitionUtil.getPartitionRangeString(prop, utcNow, 0, partitionFormat);
+ String currentUtcBorder = utcBorderDateTime + "+00:00";
+
+ // Use a deliberately mismatched nowPartitionName to simulate the
scenario
+ // where the configured TZ and UTC disagree on the calendar date.
+ // Name-based comparison would NOT exclude any partition with this
name.
+ String wrongNowPartitionName = "p_nonexistent";
+
+ // 1. Without currentUtcBorder: name-based cannot identify the current
partition.
+ List<Partition> historicalNoUtc =
DynamicPartitionScheduler.getHistoricalPartitions(
+ table, wrongNowPartitionName, null);
+ boolean currentFoundByName = false;
+ for (Partition p : historicalNoUtc) {
+ RangePartitionItem item = (RangePartitionItem)
info.getItem(p.getId());
+ String lowerBound =
item.getItems().lowerEndpoint().getKeys().get(0).getStringValue();
+ if (lowerBound.equals(currentUtcBorder)) {
+ currentFoundByName = true;
+ break;
+ }
+ }
+ Assert.assertTrue("Name-based should fail to exclude current
partition: "
+ + "nowPartitionName='" + wrongNowPartitionName + "' does not
match any partition",
+ currentFoundByName);
+
+ // 2. With currentUtcBorder: range-based correctly excludes the
current partition.
+ List<Partition> historicalWithUtc =
DynamicPartitionScheduler.getHistoricalPartitions(
+ table, wrongNowPartitionName, currentUtcBorder);
+ boolean currentFoundByRange = false;
+ for (Partition p : historicalWithUtc) {
+ RangePartitionItem item = (RangePartitionItem)
info.getItem(p.getId());
+ String lowerBound =
item.getItems().lowerEndpoint().getKeys().get(0).getStringValue();
+ if (lowerBound.equals(currentUtcBorder)) {
+ currentFoundByRange = true;
+ break;
+ }
+ }
+ Assert.assertFalse("Range-based should correctly exclude the current
partition",
+ currentFoundByRange);
+ Assert.assertEquals("Should exclude exactly the current partition",
+ totalPartitions - 1, historicalWithUtc.size());
+ }
+
+ @Test
+ public void testTimeStampTzDynamicPartitionMonthUnit() throws Exception {
+ // Verify TIMESTAMPTZ dynamic partition with MONTH unit:
+ // boundaries are at UTC midnight (day=01 00:00:00+00:00),
+ // names use the configured timezone's calendar month.
+ String originalTimeZone =
connectContext.getSessionVariable().getTimeZone();
+ try {
+ connectContext.getSessionVariable().setTimeZone("America/Chicago");
+
+ String createOlapTblStmt = "CREATE TABLE
test.`timestamptz_dynamic_month` (\n"
+ + " `k1` TIMESTAMPTZ NULL COMMENT \"\",\n"
+ + " `k2` int NULL COMMENT \"\"\n"
+ + ") ENGINE=OLAP\n"
+ + "DUPLICATE KEY(`k1`, `k2`)\n"
+ + "PARTITION BY RANGE(`k1`)\n"
+ + "()\n"
+ + "DISTRIBUTED BY HASH(`k2`) BUCKETS 3\n"
+ + "PROPERTIES (\n"
+ + "\"replication_num\" = \"1\",\n"
+ + "\"dynamic_partition.enable\" = \"true\",\n"
+ + "\"dynamic_partition.start\" = \"-3\",\n"
+ + "\"dynamic_partition.end\" = \"3\",\n"
+ + "\"dynamic_partition.create_history_partition\" =
\"true\",\n"
+ + "\"dynamic_partition.time_unit\" = \"month\",\n"
+ + "\"dynamic_partition.prefix\" = \"p\",\n"
+ + "\"dynamic_partition.buckets\" = \"1\",\n"
+ + "\"dynamic_partition.time_zone\" = \"Asia/Shanghai\"\n"
+ + ");";
+ createTable(createOlapTblStmt);
+
+ Database db =
Env.getCurrentInternalCatalog().getDbOrAnalysisException("test");
+ OlapTable table = (OlapTable)
db.getTableOrAnalysisException("timestamptz_dynamic_month");
+ Assert.assertTrue(table.dynamicPartitionExists());
+
+ Env.getCurrentEnv().getDynamicPartitionScheduler()
+ .executeDynamicPartitionFirstTime(db.getId(),
table.getId());
+
+ int partitionCount = table.getPartitionNames().size();
+ Assert.assertEquals(7, partitionCount);
+
+ // Verify partition boundaries are UTC midnight and names use
+ // configured timezone (Asia/Shanghai).
+ RangePartitionInfo partitionInfo = (RangePartitionInfo)
table.getPartitionInfo();
+ for (Map.Entry<Long, PartitionItem> entry :
partitionInfo.getIdToItem(false).entrySet()) {
+ RangePartitionItem item = (RangePartitionItem)
entry.getValue();
+ String partitionName =
table.getPartition(entry.getKey()).getName();
+ Assert.assertTrue("Partition name should start with 'p': " +
partitionName,
+ partitionName.startsWith("p"));
+ // Month partition names: p + yyyyMM → length 7
+ Assert.assertEquals("Month partition name length: " +
partitionName,
+ 7, partitionName.length());
+
+ // Verify range validity
+ Range<PartitionKey> range = item.getItems();
+ Assert.assertTrue("lower must be < upper",
+ range.lowerEndpoint().compareTo(range.upperEndpoint())
< 0);
+
+ // Partition boundaries must be UTC timestamps with +00:00
suffix.
+ List<LiteralExpr> lowerKeys = range.lowerEndpoint().getKeys();
+ Assert.assertEquals(1, lowerKeys.size());
+ String lowerStr = lowerKeys.get(0).getStringValue();
+ Assert.assertTrue("Lower key must be UTC with +00:00 suffix: "
+ lowerStr,
+ lowerStr.contains("+00:00"));
+
+ List<LiteralExpr> upperKeys = range.upperEndpoint().getKeys();
+ Assert.assertEquals(1, upperKeys.size());
+ String upperStr = upperKeys.get(0).getStringValue();
+ Assert.assertTrue("Upper key must be UTC with +00:00 suffix: "
+ upperStr,
+ upperStr.contains("+00:00"));
+
+ // Partition boundaries must be at UTC midnight (hour=00)
+ // regardless of time_zone. Day should be 01 (first of month).
+ String lowerDay = lowerStr.substring(8, 10);
+ Assert.assertEquals("Lower bound must be day 01 for month
unit: " + lowerStr,
+ "01", lowerDay);
+ String lowerHour = lowerStr.substring(11, 13);
+ Assert.assertEquals("Lower bound must be UTC midnight (00): "
+ lowerStr,
+ "00", lowerHour);
+ String upperHour = upperStr.substring(11, 13);
+ Assert.assertEquals("Upper bound must be UTC midnight (00): "
+ upperStr,
+ "00", upperHour);
+ }
+
+ // Verify the current partition (idx=0) name matches the UTC
+ // calendar month, since both names and values are UTC-based.
+ // Accept either the current or previous month in case the wall
+ // clock rolled over between scheduler execution and this
assertion.
+ ZonedDateTime utcNow = ZonedDateTime.now(ZoneOffset.UTC);
+ String expectedName = "p" +
DateTimeFormatter.ofPattern("yyyyMM").format(utcNow);
+ String prevExpectedName = "p" +
DateTimeFormatter.ofPattern("yyyyMM")
+ .format(utcNow.minusMonths(1));
+ Assert.assertTrue("Should find month partition '" + expectedName +
"' or '" + prevExpectedName
+ + "' named per UTC",
+ table.getPartitionNames().contains(expectedName)
+ || table.getPartitionNames().contains(prevExpectedName));
+ } finally {
+ connectContext.getSessionVariable().setTimeZone(originalTimeZone);
+ }
+ }
+
+ @Test
+ public void testTimeStampTzDynamicPartitionYearUnit() throws Exception {
+ // Verify TIMESTAMPTZ dynamic partition with YEAR unit:
+ // boundaries are at UTC midnight (month=01, day=01 00:00:00+00:00),
+ // names use the configured timezone's calendar year.
+ String originalTimeZone =
connectContext.getSessionVariable().getTimeZone();
+ try {
+ connectContext.getSessionVariable().setTimeZone("America/Chicago");
+
+ String createOlapTblStmt = "CREATE TABLE
test.`timestamptz_dynamic_year` (\n"
+ + " `k1` TIMESTAMPTZ NULL COMMENT \"\",\n"
+ + " `k2` int NULL COMMENT \"\"\n"
+ + ") ENGINE=OLAP\n"
+ + "DUPLICATE KEY(`k1`, `k2`)\n"
+ + "PARTITION BY RANGE(`k1`)\n"
+ + "()\n"
+ + "DISTRIBUTED BY HASH(`k2`) BUCKETS 3\n"
+ + "PROPERTIES (\n"
+ + "\"replication_num\" = \"1\",\n"
+ + "\"dynamic_partition.enable\" = \"true\",\n"
+ + "\"dynamic_partition.start\" = \"-3\",\n"
+ + "\"dynamic_partition.end\" = \"3\",\n"
+ + "\"dynamic_partition.create_history_partition\" =
\"true\",\n"
+ + "\"dynamic_partition.time_unit\" = \"year\",\n"
+ + "\"dynamic_partition.prefix\" = \"p\",\n"
+ + "\"dynamic_partition.buckets\" = \"1\",\n"
+ + "\"dynamic_partition.time_zone\" = \"Asia/Shanghai\"\n"
+ + ");";
+ createTable(createOlapTblStmt);
+
+ Database db =
Env.getCurrentInternalCatalog().getDbOrAnalysisException("test");
+ OlapTable table = (OlapTable)
db.getTableOrAnalysisException("timestamptz_dynamic_year");
+ Assert.assertTrue(table.dynamicPartitionExists());
+
+ Env.getCurrentEnv().getDynamicPartitionScheduler()
+ .executeDynamicPartitionFirstTime(db.getId(),
table.getId());
+
+ int partitionCount = table.getPartitionNames().size();
+ Assert.assertEquals(7, partitionCount);
+
+ // Verify partition boundaries are UTC midnight and names use
+ // configured timezone (Asia/Shanghai).
+ RangePartitionInfo partitionInfo = (RangePartitionInfo)
table.getPartitionInfo();
+ for (Map.Entry<Long, PartitionItem> entry :
partitionInfo.getIdToItem(false).entrySet()) {
+ RangePartitionItem item = (RangePartitionItem)
entry.getValue();
+ String partitionName =
table.getPartition(entry.getKey()).getName();
+ Assert.assertTrue("Partition name should start with 'p': " +
partitionName,
+ partitionName.startsWith("p"));
+ // Year partition names: p + yyyy → length 5
+ Assert.assertEquals("Year partition name length: " +
partitionName,
+ 5, partitionName.length());
+
+ // Verify range validity
+ Range<PartitionKey> range = item.getItems();
+ Assert.assertTrue("lower must be < upper",
+ range.lowerEndpoint().compareTo(range.upperEndpoint())
< 0);
+
+ // Partition boundaries must be UTC timestamps with +00:00
suffix.
+ List<LiteralExpr> lowerKeys = range.lowerEndpoint().getKeys();
+ Assert.assertEquals(1, lowerKeys.size());
+ String lowerStr = lowerKeys.get(0).getStringValue();
+ Assert.assertTrue("Lower key must be UTC with +00:00 suffix: "
+ lowerStr,
+ lowerStr.contains("+00:00"));
+
+ List<LiteralExpr> upperKeys = range.upperEndpoint().getKeys();
+ Assert.assertEquals(1, upperKeys.size());
+ String upperStr = upperKeys.get(0).getStringValue();
+ Assert.assertTrue("Upper key must be UTC with +00:00 suffix: "
+ upperStr,
+ upperStr.contains("+00:00"));
+
+ // Partition boundaries must be at UTC midnight (hour=00)
+ // regardless of time_zone. Month should be 01, day should be
01.
+ String lowerMonth = lowerStr.substring(5, 7);
+ Assert.assertEquals("Lower bound must be month 01 for year
unit: " + lowerStr,
+ "01", lowerMonth);
+ String lowerDay = lowerStr.substring(8, 10);
+ Assert.assertEquals("Lower bound must be day 01 for year unit:
" + lowerStr,
+ "01", lowerDay);
+ String lowerHour = lowerStr.substring(11, 13);
+ Assert.assertEquals("Lower bound must be UTC midnight (00): "
+ lowerStr,
+ "00", lowerHour);
+ String upperHour = upperStr.substring(11, 13);
+ Assert.assertEquals("Upper bound must be UTC midnight (00): "
+ upperStr,
+ "00", upperHour);
+ }
+
+ // Verify the current partition (idx=0) name matches the UTC
+ // calendar year, since both names and values are UTC-based.
+ // Accept either the current or previous year in case the wall
+ // clock rolled over between scheduler execution and this
assertion.
+ ZonedDateTime utcNow = ZonedDateTime.now(ZoneOffset.UTC);
+ String expectedName = "p" +
DateTimeFormatter.ofPattern("yyyy").format(utcNow);
+ String prevExpectedName = "p" + DateTimeFormatter.ofPattern("yyyy")
+ .format(utcNow.minusYears(1));
+ Assert.assertTrue("Should find year partition '" + expectedName +
"' or '" + prevExpectedName
+ + "' named per UTC",
+ table.getPartitionNames().contains(expectedName)
+ || table.getPartitionNames().contains(prevExpectedName));
+ } finally {
+ connectContext.getSessionVariable().setTimeZone(originalTimeZone);
+ }
+ }
+
+ @Test
+ public void testTimeStampTzReservedHistoryPeriodsUtcAligned() throws
Exception {
+ // Verify that reserved_history_periods uses UTC-midnight boundaries
+ // (via borderTimeZone from getDropPartitionOpForDynamic) consistently
+ // with the main drop cutoff and add-partition boundaries.
+ // Without the fix, getClosedRange() hardcoded the configured timezone
+ // (e.g. Asia/Shanghai) for convertToUtcTimestamp(), shifting reserved
+ // ranges by the UTC offset and causing mismatches with UTC-aligned
+ // partition boundaries.
+ String originalTimeZone =
connectContext.getSessionVariable().getTimeZone();
+ try {
+ connectContext.getSessionVariable().setTimeZone("America/Chicago");
+
+ String createSql = "CREATE TABLE test.`tstz_reserved_hist` (\n"
+ + " `k1` TIMESTAMPTZ NULL COMMENT \"\",\n"
+ + " `k2` int NULL COMMENT \"\"\n"
+ + ") ENGINE=OLAP\n"
+ + "DUPLICATE KEY(`k1`, `k2`)\n"
+ + "PARTITION BY RANGE(`k1`)\n"
+ + "()\n"
+ + "DISTRIBUTED BY HASH(`k2`) BUCKETS 3\n"
+ + "PROPERTIES (\n"
+ + "\"replication_num\" = \"1\",\n"
+ + "\"dynamic_partition.enable\" = \"true\",\n"
+ + "\"dynamic_partition.start\" = \"-3\",\n"
+ + "\"dynamic_partition.end\" = \"3\",\n"
+ + "\"dynamic_partition.create_history_partition\" =
\"false\",\n"
+ + "\"dynamic_partition.time_unit\" = \"day\",\n"
+ + "\"dynamic_partition.prefix\" = \"p\",\n"
+ + "\"dynamic_partition.buckets\" = \"1\",\n"
+ + "\"dynamic_partition.time_zone\" = \"Asia/Shanghai\",\n"
+ + "\"dynamic_partition.reserved_history_periods\" =
\"[2019-06-01,2020-08-01]\"\n"
+ + ");";
Review Comment:
This partition is well inside both the fixed UTC reserved range and the old
Asia/Shanghai-shifted range. Under the old behavior `[2019-06-01,2020-08-01]`
would convert roughly to `[2019-05-31 16:00:00+00:00,2020-07-31
16:00:00+00:00]`, which still intersects `[2020-01-01,2020-02-01)`, while
`p_old` is outside both ranges. The test would pass without the
`borderTimeZone` fix; please add a boundary partition that is retained only by
the corrected UTC range, for example one ending between `2020-07-31
16:00:00+00:00` and `2020-08-01 00:00:00+00:00`.
##########
fe/fe-core/src/test/java/org/apache/doris/catalog/DynamicPartitionTableTest.java:
##########
@@ -2313,4 +2514,805 @@ public void
testAutoPartitionRetentionTimestampTzCutoffNormalized() throws Excep
connectContext.getSessionVariable().setTimeZone(originalSessionTz);
}
}
+
+ @Test
+ public void testTimeStampTzGetHistoricalPartitionsRangeBased() throws
Exception {
+ // Verify that getHistoricalPartitions correctly identifies the current
+ // partition by range lower bound (currentUtcBorder) rather than by
name.
+ // When the configured timezone (e.g. Asia/Shanghai) is a day ahead of
UTC,
+ // the scheduler computes nowPartitionName from the TZ date (e.g.
"p20260707")
+ // while currentUtcBorder is from the UTC date ("2026-07-06
00:00:00+00:00").
+ // Name-based comparison would fail to exclude the real current
partition
+ // (p20260706), treating it as historical. Range-based comparison
correctly
+ // excludes it by matching the stored partition lower bound.
+
+ String createSql = "CREATE TABLE test.`tstz_hist_parts` (\n"
+ + " `k1` TIMESTAMPTZ NULL COMMENT \"\",\n"
+ + " `k2` int NULL COMMENT \"\"\n"
+ + ") ENGINE=OLAP\n"
+ + "DUPLICATE KEY(`k1`, `k2`)\n"
+ + "PARTITION BY RANGE(`k1`)\n"
+ + "()\n"
+ + "DISTRIBUTED BY HASH(`k2`) BUCKETS 3\n"
+ + "PROPERTIES (\n"
+ + "\"replication_num\" = \"1\",\n"
+ + "\"dynamic_partition.enable\" = \"true\",\n"
+ + "\"dynamic_partition.start\" = \"-3\",\n"
+ + "\"dynamic_partition.end\" = \"3\",\n"
+ + "\"dynamic_partition.create_history_partition\" =
\"true\",\n"
+ + "\"dynamic_partition.time_unit\" = \"day\",\n"
+ + "\"dynamic_partition.prefix\" = \"p\",\n"
+ + "\"dynamic_partition.buckets\" = \"1\",\n"
+ + "\"dynamic_partition.time_zone\" = \"Asia/Shanghai\"\n"
+ + ");";
+ createTable(createSql);
+
+ Database db =
Env.getCurrentInternalCatalog().getDbOrAnalysisException("test");
+ OlapTable table = (OlapTable)
db.getTableOrAnalysisException("tstz_hist_parts");
+
+ Env.getCurrentEnv().getDynamicPartitionScheduler()
+ .executeDynamicPartitionFirstTime(db.getId(), table.getId());
+
+ int totalPartitions = table.getPartitionNames().size();
+ Assert.assertEquals(7, totalPartitions);
+
+ RangePartitionInfo info = (RangePartitionInfo)
table.getPartitionInfo();
+ DynamicPartitionProperty prop =
table.getTableProperty().getDynamicPartitionProperty();
+ String partitionFormat = DynamicPartitionUtil.getPartitionFormat(
+ info.getPartitionColumns().get(0));
+ ZonedDateTime utcNow = ZonedDateTime.now(ZoneOffset.UTC);
+ // getPartitionRangeString with DATETIME_FORMAT returns "yyyy-MM-dd
HH:mm:ss"
+ // (e.g. "2026-07-06 00:00:00"). convertToUtcTimestamp appends
"+00:00".
+ String utcBorderDateTime =
DynamicPartitionUtil.getPartitionRangeString(prop, utcNow, 0, partitionFormat);
+ String currentUtcBorder = utcBorderDateTime + "+00:00";
+
+ // Use a deliberately mismatched nowPartitionName to simulate the
scenario
+ // where the configured TZ and UTC disagree on the calendar date.
+ // Name-based comparison would NOT exclude any partition with this
name.
+ String wrongNowPartitionName = "p_nonexistent";
+
+ // 1. Without currentUtcBorder: name-based cannot identify the current
partition.
+ List<Partition> historicalNoUtc =
DynamicPartitionScheduler.getHistoricalPartitions(
+ table, wrongNowPartitionName, null);
+ boolean currentFoundByName = false;
+ for (Partition p : historicalNoUtc) {
+ RangePartitionItem item = (RangePartitionItem)
info.getItem(p.getId());
+ String lowerBound =
item.getItems().lowerEndpoint().getKeys().get(0).getStringValue();
+ if (lowerBound.equals(currentUtcBorder)) {
+ currentFoundByName = true;
+ break;
+ }
+ }
+ Assert.assertTrue("Name-based should fail to exclude current
partition: "
+ + "nowPartitionName='" + wrongNowPartitionName + "' does not
match any partition",
+ currentFoundByName);
+
+ // 2. With currentUtcBorder: range-based correctly excludes the
current partition.
+ List<Partition> historicalWithUtc =
DynamicPartitionScheduler.getHistoricalPartitions(
+ table, wrongNowPartitionName, currentUtcBorder);
+ boolean currentFoundByRange = false;
+ for (Partition p : historicalWithUtc) {
+ RangePartitionItem item = (RangePartitionItem)
info.getItem(p.getId());
+ String lowerBound =
item.getItems().lowerEndpoint().getKeys().get(0).getStringValue();
+ if (lowerBound.equals(currentUtcBorder)) {
+ currentFoundByRange = true;
+ break;
+ }
+ }
+ Assert.assertFalse("Range-based should correctly exclude the current
partition",
+ currentFoundByRange);
+ Assert.assertEquals("Should exclude exactly the current partition",
+ totalPartitions - 1, historicalWithUtc.size());
+ }
+
+ @Test
+ public void testTimeStampTzDynamicPartitionMonthUnit() throws Exception {
+ // Verify TIMESTAMPTZ dynamic partition with MONTH unit:
+ // boundaries are at UTC midnight (day=01 00:00:00+00:00),
+ // names use the configured timezone's calendar month.
+ String originalTimeZone =
connectContext.getSessionVariable().getTimeZone();
+ try {
+ connectContext.getSessionVariable().setTimeZone("America/Chicago");
+
+ String createOlapTblStmt = "CREATE TABLE
test.`timestamptz_dynamic_month` (\n"
+ + " `k1` TIMESTAMPTZ NULL COMMENT \"\",\n"
+ + " `k2` int NULL COMMENT \"\"\n"
+ + ") ENGINE=OLAP\n"
+ + "DUPLICATE KEY(`k1`, `k2`)\n"
+ + "PARTITION BY RANGE(`k1`)\n"
+ + "()\n"
+ + "DISTRIBUTED BY HASH(`k2`) BUCKETS 3\n"
+ + "PROPERTIES (\n"
+ + "\"replication_num\" = \"1\",\n"
+ + "\"dynamic_partition.enable\" = \"true\",\n"
+ + "\"dynamic_partition.start\" = \"-3\",\n"
+ + "\"dynamic_partition.end\" = \"3\",\n"
+ + "\"dynamic_partition.create_history_partition\" =
\"true\",\n"
+ + "\"dynamic_partition.time_unit\" = \"month\",\n"
+ + "\"dynamic_partition.prefix\" = \"p\",\n"
+ + "\"dynamic_partition.buckets\" = \"1\",\n"
+ + "\"dynamic_partition.time_zone\" = \"Asia/Shanghai\"\n"
+ + ");";
+ createTable(createOlapTblStmt);
+
+ Database db =
Env.getCurrentInternalCatalog().getDbOrAnalysisException("test");
+ OlapTable table = (OlapTable)
db.getTableOrAnalysisException("timestamptz_dynamic_month");
+ Assert.assertTrue(table.dynamicPartitionExists());
+
+ Env.getCurrentEnv().getDynamicPartitionScheduler()
+ .executeDynamicPartitionFirstTime(db.getId(),
table.getId());
+
+ int partitionCount = table.getPartitionNames().size();
+ Assert.assertEquals(7, partitionCount);
+
+ // Verify partition boundaries are UTC midnight and names use
+ // configured timezone (Asia/Shanghai).
+ RangePartitionInfo partitionInfo = (RangePartitionInfo)
table.getPartitionInfo();
+ for (Map.Entry<Long, PartitionItem> entry :
partitionInfo.getIdToItem(false).entrySet()) {
+ RangePartitionItem item = (RangePartitionItem)
entry.getValue();
+ String partitionName =
table.getPartition(entry.getKey()).getName();
+ Assert.assertTrue("Partition name should start with 'p': " +
partitionName,
+ partitionName.startsWith("p"));
+ // Month partition names: p + yyyyMM → length 7
+ Assert.assertEquals("Month partition name length: " +
partitionName,
+ 7, partitionName.length());
+
+ // Verify range validity
+ Range<PartitionKey> range = item.getItems();
+ Assert.assertTrue("lower must be < upper",
+ range.lowerEndpoint().compareTo(range.upperEndpoint())
< 0);
+
+ // Partition boundaries must be UTC timestamps with +00:00
suffix.
+ List<LiteralExpr> lowerKeys = range.lowerEndpoint().getKeys();
+ Assert.assertEquals(1, lowerKeys.size());
+ String lowerStr = lowerKeys.get(0).getStringValue();
+ Assert.assertTrue("Lower key must be UTC with +00:00 suffix: "
+ lowerStr,
+ lowerStr.contains("+00:00"));
+
+ List<LiteralExpr> upperKeys = range.upperEndpoint().getKeys();
+ Assert.assertEquals(1, upperKeys.size());
+ String upperStr = upperKeys.get(0).getStringValue();
+ Assert.assertTrue("Upper key must be UTC with +00:00 suffix: "
+ upperStr,
+ upperStr.contains("+00:00"));
+
+ // Partition boundaries must be at UTC midnight (hour=00)
+ // regardless of time_zone. Day should be 01 (first of month).
+ String lowerDay = lowerStr.substring(8, 10);
+ Assert.assertEquals("Lower bound must be day 01 for month
unit: " + lowerStr,
+ "01", lowerDay);
+ String lowerHour = lowerStr.substring(11, 13);
+ Assert.assertEquals("Lower bound must be UTC midnight (00): "
+ lowerStr,
+ "00", lowerHour);
+ String upperHour = upperStr.substring(11, 13);
+ Assert.assertEquals("Upper bound must be UTC midnight (00): "
+ upperStr,
+ "00", upperHour);
+ }
+
+ // Verify the current partition (idx=0) name matches the UTC
+ // calendar month, since both names and values are UTC-based.
+ // Accept either the current or previous month in case the wall
+ // clock rolled over between scheduler execution and this
assertion.
+ ZonedDateTime utcNow = ZonedDateTime.now(ZoneOffset.UTC);
+ String expectedName = "p" +
DateTimeFormatter.ofPattern("yyyyMM").format(utcNow);
+ String prevExpectedName = "p" +
DateTimeFormatter.ofPattern("yyyyMM")
+ .format(utcNow.minusMonths(1));
+ Assert.assertTrue("Should find month partition '" + expectedName +
"' or '" + prevExpectedName
+ + "' named per UTC",
+ table.getPartitionNames().contains(expectedName)
+ || table.getPartitionNames().contains(prevExpectedName));
+ } finally {
+ connectContext.getSessionVariable().setTimeZone(originalTimeZone);
+ }
+ }
+
+ @Test
+ public void testTimeStampTzDynamicPartitionYearUnit() throws Exception {
+ // Verify TIMESTAMPTZ dynamic partition with YEAR unit:
+ // boundaries are at UTC midnight (month=01, day=01 00:00:00+00:00),
+ // names use the configured timezone's calendar year.
+ String originalTimeZone =
connectContext.getSessionVariable().getTimeZone();
+ try {
+ connectContext.getSessionVariable().setTimeZone("America/Chicago");
+
+ String createOlapTblStmt = "CREATE TABLE
test.`timestamptz_dynamic_year` (\n"
+ + " `k1` TIMESTAMPTZ NULL COMMENT \"\",\n"
+ + " `k2` int NULL COMMENT \"\"\n"
+ + ") ENGINE=OLAP\n"
+ + "DUPLICATE KEY(`k1`, `k2`)\n"
+ + "PARTITION BY RANGE(`k1`)\n"
+ + "()\n"
+ + "DISTRIBUTED BY HASH(`k2`) BUCKETS 3\n"
+ + "PROPERTIES (\n"
+ + "\"replication_num\" = \"1\",\n"
+ + "\"dynamic_partition.enable\" = \"true\",\n"
+ + "\"dynamic_partition.start\" = \"-3\",\n"
+ + "\"dynamic_partition.end\" = \"3\",\n"
+ + "\"dynamic_partition.create_history_partition\" =
\"true\",\n"
+ + "\"dynamic_partition.time_unit\" = \"year\",\n"
+ + "\"dynamic_partition.prefix\" = \"p\",\n"
+ + "\"dynamic_partition.buckets\" = \"1\",\n"
+ + "\"dynamic_partition.time_zone\" = \"Asia/Shanghai\"\n"
+ + ");";
+ createTable(createOlapTblStmt);
+
+ Database db =
Env.getCurrentInternalCatalog().getDbOrAnalysisException("test");
+ OlapTable table = (OlapTable)
db.getTableOrAnalysisException("timestamptz_dynamic_year");
+ Assert.assertTrue(table.dynamicPartitionExists());
+
+ Env.getCurrentEnv().getDynamicPartitionScheduler()
+ .executeDynamicPartitionFirstTime(db.getId(),
table.getId());
+
+ int partitionCount = table.getPartitionNames().size();
+ Assert.assertEquals(7, partitionCount);
+
+ // Verify partition boundaries are UTC midnight and names use
+ // configured timezone (Asia/Shanghai).
+ RangePartitionInfo partitionInfo = (RangePartitionInfo)
table.getPartitionInfo();
+ for (Map.Entry<Long, PartitionItem> entry :
partitionInfo.getIdToItem(false).entrySet()) {
+ RangePartitionItem item = (RangePartitionItem)
entry.getValue();
+ String partitionName =
table.getPartition(entry.getKey()).getName();
+ Assert.assertTrue("Partition name should start with 'p': " +
partitionName,
+ partitionName.startsWith("p"));
+ // Year partition names: p + yyyy → length 5
+ Assert.assertEquals("Year partition name length: " +
partitionName,
+ 5, partitionName.length());
+
+ // Verify range validity
+ Range<PartitionKey> range = item.getItems();
+ Assert.assertTrue("lower must be < upper",
+ range.lowerEndpoint().compareTo(range.upperEndpoint())
< 0);
+
+ // Partition boundaries must be UTC timestamps with +00:00
suffix.
+ List<LiteralExpr> lowerKeys = range.lowerEndpoint().getKeys();
+ Assert.assertEquals(1, lowerKeys.size());
+ String lowerStr = lowerKeys.get(0).getStringValue();
+ Assert.assertTrue("Lower key must be UTC with +00:00 suffix: "
+ lowerStr,
+ lowerStr.contains("+00:00"));
+
+ List<LiteralExpr> upperKeys = range.upperEndpoint().getKeys();
+ Assert.assertEquals(1, upperKeys.size());
+ String upperStr = upperKeys.get(0).getStringValue();
+ Assert.assertTrue("Upper key must be UTC with +00:00 suffix: "
+ upperStr,
+ upperStr.contains("+00:00"));
+
+ // Partition boundaries must be at UTC midnight (hour=00)
+ // regardless of time_zone. Month should be 01, day should be
01.
+ String lowerMonth = lowerStr.substring(5, 7);
+ Assert.assertEquals("Lower bound must be month 01 for year
unit: " + lowerStr,
+ "01", lowerMonth);
+ String lowerDay = lowerStr.substring(8, 10);
+ Assert.assertEquals("Lower bound must be day 01 for year unit:
" + lowerStr,
+ "01", lowerDay);
+ String lowerHour = lowerStr.substring(11, 13);
+ Assert.assertEquals("Lower bound must be UTC midnight (00): "
+ lowerStr,
+ "00", lowerHour);
+ String upperHour = upperStr.substring(11, 13);
+ Assert.assertEquals("Upper bound must be UTC midnight (00): "
+ upperStr,
+ "00", upperHour);
+ }
+
+ // Verify the current partition (idx=0) name matches the UTC
+ // calendar year, since both names and values are UTC-based.
+ // Accept either the current or previous year in case the wall
+ // clock rolled over between scheduler execution and this
assertion.
+ ZonedDateTime utcNow = ZonedDateTime.now(ZoneOffset.UTC);
+ String expectedName = "p" +
DateTimeFormatter.ofPattern("yyyy").format(utcNow);
+ String prevExpectedName = "p" + DateTimeFormatter.ofPattern("yyyy")
+ .format(utcNow.minusYears(1));
+ Assert.assertTrue("Should find year partition '" + expectedName +
"' or '" + prevExpectedName
+ + "' named per UTC",
+ table.getPartitionNames().contains(expectedName)
+ || table.getPartitionNames().contains(prevExpectedName));
+ } finally {
+ connectContext.getSessionVariable().setTimeZone(originalTimeZone);
+ }
+ }
+
+ @Test
+ public void testTimeStampTzReservedHistoryPeriodsUtcAligned() throws
Exception {
+ // Verify that reserved_history_periods uses UTC-midnight boundaries
+ // (via borderTimeZone from getDropPartitionOpForDynamic) consistently
+ // with the main drop cutoff and add-partition boundaries.
+ // Without the fix, getClosedRange() hardcoded the configured timezone
+ // (e.g. Asia/Shanghai) for convertToUtcTimestamp(), shifting reserved
+ // ranges by the UTC offset and causing mismatches with UTC-aligned
+ // partition boundaries.
+ String originalTimeZone =
connectContext.getSessionVariable().getTimeZone();
+ try {
+ connectContext.getSessionVariable().setTimeZone("America/Chicago");
+
+ String createSql = "CREATE TABLE test.`tstz_reserved_hist` (\n"
+ + " `k1` TIMESTAMPTZ NULL COMMENT \"\",\n"
+ + " `k2` int NULL COMMENT \"\"\n"
+ + ") ENGINE=OLAP\n"
+ + "DUPLICATE KEY(`k1`, `k2`)\n"
+ + "PARTITION BY RANGE(`k1`)\n"
+ + "()\n"
+ + "DISTRIBUTED BY HASH(`k2`) BUCKETS 3\n"
+ + "PROPERTIES (\n"
+ + "\"replication_num\" = \"1\",\n"
+ + "\"dynamic_partition.enable\" = \"true\",\n"
+ + "\"dynamic_partition.start\" = \"-3\",\n"
+ + "\"dynamic_partition.end\" = \"3\",\n"
+ + "\"dynamic_partition.create_history_partition\" =
\"false\",\n"
+ + "\"dynamic_partition.time_unit\" = \"day\",\n"
+ + "\"dynamic_partition.prefix\" = \"p\",\n"
+ + "\"dynamic_partition.buckets\" = \"1\",\n"
+ + "\"dynamic_partition.time_zone\" = \"Asia/Shanghai\",\n"
+ + "\"dynamic_partition.reserved_history_periods\" =
\"[2019-06-01,2020-08-01]\"\n"
+ + ");";
+ createTable(createSql);
+
+ Database db =
Env.getCurrentInternalCatalog().getDbOrAnalysisException("test");
+ OlapTable table = (OlapTable)
db.getTableOrAnalysisException("tstz_reserved_hist");
+
+ // Add partitions manually — must temporarily disable dynamic
partition.
+ alterTable("ALTER TABLE test.tstz_reserved_hist SET "
+ + "('dynamic_partition.enable' = 'false')");
+ // Add a partition that falls within the reserved period with
+ // UTC-midnight boundaries: [2020-01-01 00:00 UTC, 2020-02-01
00:00 UTC).
+ // This should be protected by the reserved period.
+ alterTable("ALTER TABLE test.tstz_reserved_hist ADD PARTITION
p_202001 VALUES "
+ + "[('2020-01-01 00:00:00+00:00'), ('2020-02-01
00:00:00+00:00'))");
+ // Add a partition entirely before the reserved period.
+ // This should be dropped by the start=-3 cutoff.
+ alterTable("ALTER TABLE test.tstz_reserved_hist ADD PARTITION
p_old VALUES "
+ + "[('2018-01-01 00:00:00+00:00'), ('2018-02-01
00:00:00+00:00'))");
+
+ Assert.assertTrue("p_202001 should exist before scheduling",
+ table.getPartitionNames().contains("p_202001"));
+ Assert.assertTrue("p_old should exist before scheduling",
+ table.getPartitionNames().contains("p_old"));
+
+ // Re-enable dynamic partition and run the scheduler.
+ alterTable("ALTER TABLE test.tstz_reserved_hist SET "
+ + "('dynamic_partition.enable' = 'true')");
+
+ Env.getCurrentEnv().getDynamicPartitionScheduler()
+ .executeDynamicPartitionFirstTime(db.getId(),
table.getId());
+
+ // p_202001 falls within reserved period [2019-06-01, 2020-08-01]
in
+ // UTC-midnight interpretation → must be kept.
+ Assert.assertTrue("p_202001 should be kept — within UTC-aligned
reserved history period",
+ table.getPartitionNames().contains("p_202001"));
+ // p_old is before the start=-3 cutoff and outside the reserved
+ // period → must be dropped.
+ Assert.assertFalse("p_old should be dropped — outside reserved
period and before start",
+ table.getPartitionNames().contains("p_old"));
+ } finally {
+ connectContext.getSessionVariable().setTimeZone(originalTimeZone);
+ }
+ }
+
+ @Test
+ public void testTimeStampTzHotPartitionCooldownUtcAligned() throws
Exception {
+ // Verify that hot-partition cooldown times are UTC-midnight aligned
+ // and not shifted by the configured timezone (America/Chicago, UTC-5).
+ // Without the fix, setStorageMediumProperty() used nowTz (configured
+ // timezone) which could place the cooldown boundary on a different
+ // calendar day than the partition's UTC range.
+ String originalTimeZone =
connectContext.getSessionVariable().getTimeZone();
+ try {
+ connectContext.getSessionVariable().setTimeZone("Asia/Shanghai");
+ changeBeDisk(TStorageMedium.SSD);
+
+ String createSql = "CREATE TABLE test.`tstz_hot_part_cooldown` (\n"
+ + " `k1` TIMESTAMPTZ NULL COMMENT \"\",\n"
+ + " `k2` int NULL COMMENT \"\"\n"
+ + ") ENGINE=OLAP\n"
+ + "DUPLICATE KEY(`k1`, `k2`)\n"
+ + "PARTITION BY RANGE(`k1`)\n"
+ + "()\n"
+ + "DISTRIBUTED BY HASH(`k2`) BUCKETS 3\n"
+ + "PROPERTIES (\n"
+ + "\"replication_num\" = \"1\",\n"
+ + "\"dynamic_partition.enable\" = \"true\",\n"
+ + "\"dynamic_partition.start\" = \"-3\",\n"
+ + "\"dynamic_partition.end\" = \"3\",\n"
+ + "\"dynamic_partition.create_history_partition\" =
\"true\",\n"
+ + "\"dynamic_partition.time_unit\" = \"day\",\n"
+ + "\"dynamic_partition.prefix\" = \"p\",\n"
+ + "\"dynamic_partition.buckets\" = \"1\",\n"
+ + "\"dynamic_partition.time_zone\" =
\"America/Chicago\",\n"
+ + "\"dynamic_partition.hot_partition_num\" = \"1\"\n"
+ + ");";
+ createTable(createSql);
+
+ Database db =
Env.getCurrentInternalCatalog().getDbOrAnalysisException("test");
+ OlapTable table = (OlapTable)
db.getTableOrAnalysisException("tstz_hot_part_cooldown");
+
+ Env.getCurrentEnv().getDynamicPartitionScheduler()
+ .executeDynamicPartitionFirstTime(db.getId(),
table.getId());
+
+ RangePartitionInfo partitionInfo = (RangePartitionInfo)
table.getPartitionInfo();
+ // Sort partitions by lower bound.
+ List<Map.Entry<Long, PartitionItem>> sortedEntries =
Lists.newArrayList(
+ partitionInfo.getIdToItem(false).entrySet());
+ sortedEntries.sort((a, b) -> {
+ RangePartitionItem ai = (RangePartitionItem) a.getValue();
+ RangePartitionItem bi = (RangePartitionItem) b.getValue();
+ return
ai.getItems().lowerEndpoint().compareTo(bi.getItems().lowerEndpoint());
+ });
+
+ Assert.assertEquals(7, sortedEntries.size());
+
+ // Partitions idx=-3,-2,-1 are before the hot range → HDD.
+ // Partitions idx=0..3 (4 partitions) are within
hot_partition_num=1:
+ // - idx=0: hot (offset + hotPartitionNum = 0+1 = 1 > 0) → SSD
+ // - idx=1,2,3: hot (offset + hotPartitionNum > 0) → SSD
+ // Because hot_partition_num=1 means only the current partition is
hot,
+ // but the logic counts from idx + hotPartitionNum > 0.
+ // Actually: the condition is `offset + hotPartitionNum <= 0` →
HDD.
+ // So idx=-3,-2,-1: -3+1=-2≤0 HDD, -2+1=-1≤0 HDD, -1+1=0≤0 HDD.
+ // idx=0,1,2,3: 0+1=1>0 SSD, etc.
+ // idx=-3,-2,-1 are HDD, idx=0,1,2,3 are SSD.
+
+ for (int i = 0; i < sortedEntries.size(); i++) {
+ Map.Entry<Long, PartitionItem> entry = sortedEntries.get(i);
+ RangePartitionItem item = (RangePartitionItem)
entry.getValue();
+ DataProperty dp =
partitionInfo.getDataProperty(entry.getKey());
+
+ if (i < 3) {
+ // Historical partitions: HDD
+ Assert.assertEquals("Historical partition should be HDD:
idx=" + (i - 3),
+ TStorageMedium.HDD, dp.getStorageMedium());
+ } else {
+ // Hot partitions: SSD
+ Assert.assertEquals("Hot partition should be SSD: idx=" +
(i - 3),
+ TStorageMedium.SSD, dp.getStorageMedium());
+
+ // Every hot partition must have a finite cooldown equal to
+ // that partition's upper endpoint (offset +
hotPartitionNum).
+ // Assert it is NOT the MAX fallback, which would indicate
+ // the TIMESTAMPTZ lifecycle string was rejected during
parse.
+ Assert.assertNotEquals("Hot partition must have a finite
cooldown: idx=" + (i - 3),
+ DataProperty.MAX_COOLDOWN_TIME_MS,
dp.getCooldownTimeMs());
+
+ ZonedDateTime cooldownUtc = ZonedDateTime.ofInstant(
+
java.time.Instant.ofEpochMilli(dp.getCooldownTimeMs()),
+ ZoneOffset.UTC);
+
+ // Parse the partition's upper bound as a UTC instant.
+ String upperStr =
item.getItems().upperEndpoint().getKeys().get(0).getStringValue();
+ ZonedDateTime upperUtc = ZonedDateTime.parse(upperStr,
+ DateTimeFormatter.ofPattern("yyyy-MM-dd
HH:mm:ssXXX"));
+
+ Assert.assertEquals("Cooldown must equal the partition
upper bound ("
+ + upperUtc + "): idx=" + (i - 3),
+ upperUtc.toInstant(), cooldownUtc.toInstant());
+ }
+
+ // Verify partition boundaries are UTC midnight.
+ List<LiteralExpr> lowerKeys =
item.getItems().lowerEndpoint().getKeys();
+ String lowerStr = lowerKeys.get(0).getStringValue();
+ Assert.assertTrue("Lower key must be UTC: " + lowerStr,
+ lowerStr.contains("+00:00"));
+ Assert.assertEquals("Lower bound must be at UTC midnight: " +
lowerStr,
+ "00", lowerStr.substring(11, 13));
+ }
+ } finally {
+ connectContext.getSessionVariable().setTimeZone(originalTimeZone);
+ }
+ }
+
+ @Test
+ public void testTimeStampTzHotPartitionCooldownDstMonth() throws Exception
{
+ // Regression: When the session timezone has DST (e.g. America/Chicago,
+ // UTC-5 in summer, UTC-6 in winter),
PropertyAnalyzer.analyzeDataProperty()
+ // parses cooldown values as DATETIME via
DateLiteralUtils.createDateLiteral()
+ // which applies the current Instant's offset during the initial shift
+ // while using the target date's offset in unixTimestamp(). With the
old
+ // convertToUtcTimestamp path (which appended "+00:00"), a cooldown
+ // boundary in a different DST period than NOW would shift by one hour.
+ // The fix stores the cooldown with an explicit +00:00 suffix, and
+ // PropertyAnalyzer now parses timezone-suffixed values directly as
+ // instants. This test uses MONTH unit with America/Chicago (DST) for
+ // both session and partition timezones so the cooldown boundary is
+ // likely to fall in a different DST period than the current date.
+ String originalTimeZone =
connectContext.getSessionVariable().getTimeZone();
+ try {
+ connectContext.getSessionVariable().setTimeZone("America/Chicago");
+ changeBeDisk(TStorageMedium.SSD);
+
+ String createSql = "CREATE TABLE test.`tstz_cooldown_dst_month`
(\n"
+ + " `k1` TIMESTAMPTZ NULL COMMENT \"\",\n"
+ + " `k2` int NULL COMMENT \"\"\n"
+ + ") ENGINE=OLAP\n"
+ + "DUPLICATE KEY(`k1`, `k2`)\n"
+ + "PARTITION BY RANGE(`k1`)\n"
+ + "()\n"
+ + "DISTRIBUTED BY HASH(`k2`) BUCKETS 3\n"
+ + "PROPERTIES (\n"
+ + "\"replication_num\" = \"1\",\n"
+ + "\"dynamic_partition.enable\" = \"true\",\n"
+ + "\"dynamic_partition.start\" = \"-3\",\n"
+ + "\"dynamic_partition.end\" = \"3\",\n"
+ + "\"dynamic_partition.create_history_partition\" =
\"true\",\n"
+ + "\"dynamic_partition.time_unit\" = \"month\",\n"
+ + "\"dynamic_partition.prefix\" = \"p\",\n"
+ + "\"dynamic_partition.buckets\" = \"1\",\n"
+ + "\"dynamic_partition.time_zone\" =
\"America/Chicago\",\n"
+ + "\"dynamic_partition.hot_partition_num\" = \"6\"\n"
+ + ");";
+ createTable(createSql);
+
+ Database db =
Env.getCurrentInternalCatalog().getDbOrAnalysisException("test");
+ OlapTable table = (OlapTable)
db.getTableOrAnalysisException("tstz_cooldown_dst_month");
+
+ Env.getCurrentEnv().getDynamicPartitionScheduler()
+ .executeDynamicPartitionFirstTime(db.getId(),
table.getId());
+
+ RangePartitionInfo partitionInfo = (RangePartitionInfo)
table.getPartitionInfo();
+ List<Map.Entry<Long, PartitionItem>> sortedEntries =
Lists.newArrayList(
+ partitionInfo.getIdToItem(false).entrySet());
+ sortedEntries.sort((a, b) -> {
+ RangePartitionItem ai = (RangePartitionItem) a.getValue();
+ RangePartitionItem bi = (RangePartitionItem) b.getValue();
+ return
ai.getItems().lowerEndpoint().compareTo(bi.getItems().lowerEndpoint());
+ });
+
+ Assert.assertEquals(7, sortedEntries.size());
+
+ // idx=-3,-2,-1: offset+6=3,2,1 >0 → SSD (hot)
+ // idx=0..3: offset+6=6,7,8,9 >0 → SSD (hot)
+ // All partitions should be SSD since hot_partition_num covers all.
+
+ // Derive the expected cooldown from the current partition's
+ // stored lower bound rather than sampling ZonedDateTime.now(),
+ // which can differ from the clock captured by the scheduler
+ // and cause spurious failures across UTC month boundaries.
+ RangePartitionItem currentItem = (RangePartitionItem)
sortedEntries.get(3).getValue();
+ ZonedDateTime currentLower = ZonedDateTime.parse(
+
currentItem.getItems().lowerEndpoint().getKeys().get(0).getStringValue(),
+ DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ssXXX"));
+
+ for (int i = 0; i < sortedEntries.size(); i++) {
+ int idx = i - 3; // idx=-3,-2,-1,0,1,2,3
+ Map.Entry<Long, PartitionItem> entry = sortedEntries.get(i);
+ RangePartitionItem item = (RangePartitionItem)
entry.getValue();
+ DataProperty dp =
partitionInfo.getDataProperty(entry.getKey());
+
+ Assert.assertEquals("All partitions should be SSD with
hot_partition_num=6",
+ TStorageMedium.SSD, dp.getStorageMedium());
+ Assert.assertNotEquals("Hot partition must have a finite
cooldown",
+ DataProperty.MAX_COOLDOWN_TIME_MS,
dp.getCooldownTimeMs());
+
+ // Cooldown is the lower bound of the (idx +
hotPartitionNum)-th
+ // partition. Derive it from the current partition's lower
bound
+ // (which shares the scheduler's clock) to avoid race
conditions.
+ ZonedDateTime cooldownUtc = ZonedDateTime.ofInstant(
+ java.time.Instant.ofEpochMilli(dp.getCooldownTimeMs()),
+ ZoneOffset.UTC);
+ ZonedDateTime expectedUtc = currentLower.plusMonths(idx + 6);
+ Assert.assertEquals("Cooldown must equal lower bound of
partition at offset "
+ + (idx + 6) + " (" + expectedUtc + "): idx=" + idx,
+ expectedUtc.toInstant(), cooldownUtc.toInstant());
+
+ // Verify partition boundaries are UTC midnight, day=01.
+ List<LiteralExpr> lowerKeys =
item.getItems().lowerEndpoint().getKeys();
+ String lowerStr = lowerKeys.get(0).getStringValue();
+ Assert.assertTrue("Lower key must be UTC: " + lowerStr,
+ lowerStr.contains("+00:00"));
+ Assert.assertEquals("Lower bound must be at UTC midnight: " +
lowerStr,
+ "00", lowerStr.substring(11, 13));
+ // Month boundaries must fall on the first day of the month.
+ Assert.assertEquals("Month lower bound day must be 01: " +
lowerStr,
+ "01", lowerStr.substring(8, 10));
+ }
+ } finally {
+ connectContext.getSessionVariable().setTimeZone(originalTimeZone);
+ }
+ }
+
+ @Test
+ public void testTimeStampTzHotPartitionCooldownDstFallback() throws
Exception {
+ // DST fall-back regression: on 2026-11-01 at 2:00 AM CDT,
America/Chicago
+ // clocks fall back to 1:00 AM CST. The hour 01:00–02:00 occurs twice,
+ // so a wall-clock string "01:00:00" without offset is ambiguous and
Java
+ // resolves it to the earlier offset (06:00 UTC instead of 07:00 UTC).
+ // The fix stores the cooldown as an unambiguous UTC timestamp with
+00:00
+ // suffix (e.g. "2026-11-01 07:00:00+00:00"), and PropertyAnalyzer now
+ // parses timezone-suffixed values directly as instants, bypassing the
+ // broken DATETIME path. This test uses HOUR unit with America/Chicago
+ // to ensure cooldown timestamps remain correct regardless of DST
status.
+ String originalTimeZone =
connectContext.getSessionVariable().getTimeZone();
+ try {
+ connectContext.getSessionVariable().setTimeZone("America/Chicago");
+ changeBeDisk(TStorageMedium.SSD);
+
+ String createSql = "CREATE TABLE test.`tstz_cooldown_dst_fallback`
(\n"
+ + " `k1` TIMESTAMPTZ NULL COMMENT \"\",\n"
+ + " `k2` int NULL COMMENT \"\"\n"
+ + ") ENGINE=OLAP\n"
+ + "DUPLICATE KEY(`k1`, `k2`)\n"
+ + "PARTITION BY RANGE(`k1`)\n"
+ + "()\n"
+ + "DISTRIBUTED BY HASH(`k2`) BUCKETS 3\n"
+ + "PROPERTIES (\n"
+ + "\"replication_num\" = \"1\",\n"
+ + "\"dynamic_partition.enable\" = \"true\",\n"
+ + "\"dynamic_partition.start\" = \"-3\",\n"
+ + "\"dynamic_partition.end\" = \"3\",\n"
+ + "\"dynamic_partition.create_history_partition\" =
\"true\",\n"
Review Comment:
The test describes the 2026-11-01 America/Chicago fall-back overlap, but
this call uses the scheduler's real current clock. Unless CI happens to run
during the repeated 01:00 hour, the old ambiguous-wall-time bug is not
exercised and the assertions only cover ordinary hourly cooldown arithmetic.
Please use a fixed/injectable clock for this scheduler path, or add a focused
conversion/parser test for the two distinct `America/Chicago` 01:00 instants
and their UTC offsets.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
