924060929 commented on code in PR #66530:
URL: https://github.com/apache/doris/pull/66530#discussion_r3773298800
##########
fe/fe-connector/fe-connector-hudi/src/main/java/org/apache/doris/connector/hudi/HudiScanPlanProvider.java:
##########
@@ -1009,4 +1028,89 @@ static Map<String, String>
storageHadoopConfig(ConnectorContext context) {
private ConnectorStorageContext storage() {
return context.getStorageContext();
}
+
+ /** Package-private for offline unit tests of key construction. */
+ static HudiScanReuseKey hudiScanReuseKey(HudiTableHandle handle) {
+ return new HudiScanReuseKey(handle);
+ }
+
+ /**
+ * Statement-scoped cache key for one Hudi scan.
+ *
+ * <p>Includes every input that changes the planned split list: table
identity, the snapshot
+ * instant, the incremental window (begin/end instant + incremental
options), the pruned
+ * partition set, the partition keys, and the JNI metadata carriers (input
format / serde).
+ * Session variables are statement-constant and deliberately absent.
+ */
+ static final class HudiScanReuseKey {
+ private final String dbName;
+ private final String tableName;
+ private final String basePath;
+ private final String queryInstant;
+ private final String beginInstant;
+ private final String endInstant;
+ private final Map<String, String> incrementalParams;
+ private final List<String> prunedPartitionPaths;
+ private final List<String> partitionKeyNames;
+ private final String inputFormat;
+ private final String serdeLib;
+
+ // Hudi scan-planning identity is fully captured by the handle and its
query/incremental
+ // parameters; request-level filter, columns, and countPushdown do not
affect split planning
+ // and are deliberately excluded from the key.
+ private HudiScanReuseKey(HudiTableHandle handle) {
+ // Catalog and query isolation are provided by the statement-scope
memo key.
+ this.dbName = handle.getDbName();
+ this.tableName = handle.getTableName();
+ this.basePath = handle.getBasePath();
+ this.queryInstant = handle.getQueryInstant();
+ this.beginInstant = handle.getBeginInstant();
+ this.endInstant = handle.getEndInstant();
+ this.incrementalParams = handle.getIncrementalParams() == null
+ ? Collections.emptyMap()
+ : Collections.unmodifiableMap(new
HashMap<>(handle.getIncrementalParams()));
+ this.prunedPartitionPaths = handle.getPrunedPartitionPaths() ==
null
Review Comment:
Addressed on the new head. `HudiScanReuseKey` now preserves `null` versus an
empty pruned-partition-path list, and the same-instant live-scope test plans
zero-pruned first followed by unpruned and verifies two distinct cache entries
and the expected non-empty unpruned result.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
