sundapeng commented on code in PR #8728:
URL: https://github.com/apache/paimon/pull/8728#discussion_r3610938024
##########
paimon-core/src/main/java/org/apache/paimon/utils/PartitionPathUtils.java:
##########
@@ -265,12 +357,45 @@ public static LinkedHashMap<String, String>
extractPartitionSpecFromPath(Path cu
return fullPartSpec;
}
+ /** Extract exactly the trailing key-value components for the declared
partition keys. */
+ @Nullable
+ static LinkedHashMap<String, String> extractPartitionSpecFromPath(
+ Path currPath, List<String> partitionKeys) {
+ String[] values = new String[partitionKeys.size()];
+ Path current = currPath;
+ for (int i = partitionKeys.size() - 1; i >= 0; i--) {
+ if (current == null) {
+ return null;
+ }
+ Matcher matcher =
PARTITION_NAME_PATTERN.matcher(current.getName());
+ if (!matcher.matches()
+ ||
!partitionKeys.get(i).equals(unescapePathName(matcher.group(1)))) {
+ return null;
+ }
+ values[i] = unescapePathName(matcher.group(2));
+ current = current.getParent();
+ }
+
+ LinkedHashMap<String, String> spec = new LinkedHashMap<>();
+ for (int i = 0; i < partitionKeys.size(); i++) {
+ spec.put(partitionKeys.get(i), values[i]);
+ }
+ return spec;
+ }
+
public static LinkedHashMap<String, String>
extractPartitionSpecFromPathOnlyValue(
Path currPath, List<String> partitionKeys) {
LinkedHashMap<String, String> fullPartSpec = new LinkedHashMap<>();
String[] split = currPath.toString().split(Path.SEPARATOR);
for (int i = 0; i < partitionKeys.size(); i++) {
- fullPartSpec.put(partitionKeys.get(i), split[split.length -
partitionKeys.size() + i]);
+ // Unescape the directory component so the extracted value is the
RAW partition value,
+ // consistent with the key=value branch
(extractPartitionSpecFromPath) and with the
+ // values the write path registers into a partition-managing
catalog. Without this,
+ // directories containing escaped characters (e.g. a%3Ab) would
round-trip to a
+ // different value than the one registered (a:b).
+ fullPartSpec.put(
+ partitionKeys.get(i),
+ unescapePathName(split[split.length - partitionKeys.size()
+ i]));
Review Comment:
🟡 **[minor] migration — value-only 分区值新增反转义，改变现有 format table 的分区 spec**
extractPartitionSpecFromPathOnlyValue 中新增的 `unescapePathName()` 改变了所有使用
`format-table.partition-only-value-in-path=true` 的现有 format table 的文件系统扫描返回的分区
spec 值。以前目录名 `a%3Ab` 产生分区值 `a%3Ab`；现在产生 `a:b`。
虽然这是正确性修复（对齐 key=value 分支），但对现有表是**行为变更**：
1. 缓存或存储了旧转义分区值的下游消费者（如 Hive metastore 同步、外部元数据存储）将出现不匹配
2. 使用旧转义表示的分区过滤器查询将停止匹配
3. 向外部系统注册分区（MSCK、Hive sync）将注册与之前不同的值，可能创建重复分区条目
这影响 searchPartSpecAndPaths 代码路径，用于所有非 managed format table 扫描，不仅是新的 managed
format table。
**建议**: 考虑将反转义行为置于表选项或版本标志之后，使现有表保留之前的读取语义直到显式迁移。或者在 release notes
中记录为已知行为变更，并提供下游元数据存储的迁移步骤。至少添加集成测试验证具有转义目录名（如 `a%3Ab`）的现有 value-only
表在升级后仍返回正确查询结果。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
