github-actions[bot] commented on code in PR #69346:
URL: https://github.com/apache/doris/pull/69346#discussion_r4343983922
##########
fe/fe-core/src/main/java/org/apache/doris/tablefunction/LanceExternalSearchTableValuedFunction.java:
##########
@@ -108,6 +108,16 @@ protected
LanceExternalSearchTableValuedFunction(PreparedSearch prepared) {
this.offset = common.offset;
}
+ protected LanceExternalSearchTableValuedFunction(PreparedSearch prepared,
byte[] substraitFilter) {
+ this(prepared);
+ // Automatic rewrites must apply the original WHERE before choosing
search candidates.
+ searchRequest.setPrefilter(true);
+ if (substraitFilter.length > 0) {
+ searchRequest.setSearchFilter(new
TSearchFilter().setFormat(TSearchFilterFormat.SUBSTRAIT)
Review Comment:
[P2] Keep scalar fallback while old search readers remain eligible
During cloud smooth upgrade, an alive source BE from before this change
remains in `CloudComputeGroup.getBackendList()` and passes
`FederationBackendPolicy`'s query/load availability checks. For ordinary
BIGINT/FLOAT32 fields,
`LanceScanNode.checkAdditionalTypeBackendCompatibility()` also lets it through.
Thus this previously working plan can send a rewritten search split to that old
reader:
```text
TopN(d ASC, limit=1)
Project(id, l2_distance_approximate(embedding, [1,2]) AS d)
Filter(id = 9)
LanceScan(id BIGINT, embedding FixedSizeList<Float32>[2])
```
The bound-scan factory now puts the WHERE predicate in
`external_search_request.search_filter` as SUBSTRAIT, but the pre-change BE's
`_validate_external_search_request()` accepts only SQL and returns `unsupported
external search filter format`. Before this rewrite, the same BE handled the
ordinary scalar scan and its existing `lance_substrait_filter`;
`inner_product_approximate` regresses the same way. Gate the optional rewrite
on reader capability, retain scalar execution while incompatible BEs remain
eligible, or constrain scheduling to compatible readers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
