github-actions[bot] commented on code in PR #67186:
URL: https://github.com/apache/doris/pull/67186#discussion_r3921050264
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/scan/PluginDrivenScanNode.java:
##########
@@ -504,6 +505,68 @@ private void
appendConnectorScanProfiles(List<ConnectorScanProfile> profiles) {
writeScanProfilesInto(summaryProfile.getExecutionSummary(), profiles);
}
+ private void
collectAndAppendConnectorScanProfiles(ConnectorScanPlanProvider scanProvider) {
+ List<ConnectorScanProfile> profiles = onPluginClassLoader(scanProvider,
+ () -> scanProvider.collectScanProfiles(connectorSession));
+ appendConnectorScanProfiles(profiles);
+ }
+
Review Comment:
[P1] Close the connector statement scope for batch scans.
`FileQueryScanNode` enters this path through `SplitAssignment.init()` ->
`startSplit()`, so it bypasses `getSplits()`, the only place that currently
registers `statementScope::closeAll`. The session captured here can own Iceberg
`TableLease`/`TrackedTable` resources, but Arrow Flight sets
`returnResultFromLocal=false` and `StatementContext.close()` deliberately skips
its fallback close, relying on the query-finish callback. Because this batch
path never registers that callback, each deferred Flight batch query can leave
its scope and leases open. Please factor the callback registration into a
helper and invoke it before dispatch in both batch flavors, with a lifecycle
test that verifies closure after coordinator unregister.
##########
fe/fe-connector/fe-connector-hms/src/main/java/org/apache/doris/connector/hms/HmsClientConfig.java:
##########
@@ -85,6 +89,7 @@ public static String removedMetastoreTypeError(Map<String,
String> properties) {
private final Map<String, String> properties;
private final String confResources;
private final int poolSize;
+ private final int partitionBatchSize;
Review Comment:
[P2] Normalize the raw batch-size property before parsing. The catalog
binder trims values before binding this field, but both
`HiveCatalogProperties.of` and `HudiCatalogProperties.of` then pass the
original map to this constructor, where `Integer.parseInt(value)` is called
without trimming. A quoted SQL property such as
`hive.hms_partitions_batch_size_per_rpc = ' 5000 '` therefore binds
successfully and then throws while creating/replaying the catalog. Please trim
(or pass the typed field) and add Hive/Hudi whitespace tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
