github-actions[bot] commented on code in PR #66711:
URL: https://github.com/apache/doris/pull/66711#discussion_r3781710820
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/FileSplit.java:
##########
@@ -76,6 +80,12 @@ public Object getInfo() {
return null;
}
+ @Override
+ public Optional<String> getFileAffinityKey() {
+ return hosts.length == 0 && fileLength > length &&
fileAffinitySupported
Review Comment:
[P2] Keep pinned local TVFs out of same-file affinity
This predicate also becomes true for a splittable Parquet/ORC `local()` TVF:
`TVFScanNode` enables the scan-wide format flag, `FileSplitter` supplies empty
synthetic hosts, and `fileLength > length` holds. In the normal non-shared
form, `backend_id` has already constrained execution to the selected backend
location, but recording this key makes `AffinityAwareInstanceSplitter` treat
the entire file as one group. For a query over that one large file that is
non-serial before grouping, both legacy and Nereids consequently drop from
several fragment instances to one; the remaining scanner fan-out does not
restore the lost downstream pipeline parallelism. Please gate affinity off
using local-TVF/source provenance (production `safe_glob` paths can be
schemaless, so a `FILE_LOCAL`-only check is insufficient) and cover a split,
backend-pinned local-TVF plan.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
