github-actions[bot] commented on code in PR #68227:
URL: https://github.com/apache/doris/pull/68227#discussion_r4055887250
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushDownFilterIntoSchemaScan.java:
##########
@@ -48,7 +48,7 @@
public class PushDownFilterIntoSchemaScan extends OneRewriteRuleFactory {
public static ImmutableSet<String> SUPPOPRT_FRONTEND_CONJUNCTS_TABLES =
- ImmutableSet.of("view_dependency", "sql_block_rule_status");
+ ImmutableSet.of("view_dependency", "sql_block_rule_status",
"table_stream_consumption");
Review Comment:
[P1] Resolve the end-to-end regression before enabling this table
The exact-head local P0 build 1051636 and cloud P0 build 1051638 both fail
`mtmv_p0.ivm.test_ivm_replace_stream_cleanup` at the first aggregate over
`table_stream_consumption` with only `WHERE DB_NAME = ...`; JDBC receives
`errCode = 2` with an empty detail instead of rows. This whitelist entry is
what newly activates the planner-to-BE-to-FE predicate route for that existing
query, while the added direct tests do not cover that route. Please use the FE
`Failed to fetchSchemaTableData` stack to fix the throwing step (or keep this
table out of the gate) and make the existing local and cloud regressions pass.
##########
fe/fe-core/src/main/java/org/apache/doris/tablefunction/MetadataGenerator.java:
##########
@@ -2329,8 +2336,56 @@ private static TFetchSchemaTableDataResult
streamMetadataResult(TSchemaTableRequ
private static TFetchSchemaTableDataResult
streamConsumptionMetadataResult(TSchemaTableRequestParams params) {
TFetchSchemaTableDataResult result = new TFetchSchemaTableDataResult();
List<TRow> dataBatch = Lists.newArrayList();
+ List<Expression> parsedConjuncts = Collections.emptyList();
+ if (params.isSetFrontendConjuncts()) {
+ try {
+ parsedConjuncts =
FrontendConjunctsUtils.convertToExpression(params.getFrontendConjuncts());
+ } catch (RuntimeException e) {
+ LOG.warn("Failed to convert frontend conjuncts for
table_stream_consumption; skip FE pruning", e);
+ }
+ }
+ List<Expression> conjuncts = parsedConjuncts;
try {
-
Env.getCurrentEnv().getTableStreamManager().fillStreamConsumptionValuesMetadataResult(dataBatch);
+ if (conjuncts.isEmpty()) {
+
Env.getCurrentEnv().getTableStreamManager().fillStreamConsumptionValuesMetadataResult(dataBatch);
+ result.setDataBatch(dataBatch);
+ result.setStatus(new TStatus(TStatusCode.OK));
+ return result;
+ }
+ // `DB_NAME='db1'`, `STREAM_ID=10`
+ List<Expression> streamConjuncts = Lists.newArrayList();
+ // `UNIT='p1'`, `DB_NAME='db1' OR UNIT='p1'`
+ List<Expression> unitConjuncts = Lists.newArrayList();
+ for (Expression conjunct : conjuncts) {
+ Set<String> referencedColumns = new HashSet<>();
+ for (UnboundSlot slot :
conjunct.<UnboundSlot>collectToList(UnboundSlot.class::isInstance)) {
+ List<String> nameParts = slot.getNameParts();
+ if (!nameParts.isEmpty()) {
+ referencedColumns.add(nameParts.get(nameParts.size() -
1).toUpperCase(Locale.ROOT));
+ }
+ }
+ if
(STREAM_CONSUMPTION_STREAM_COLUMNS.containsAll(referencedColumns)) {
+ streamConjuncts.add(conjunct);
+ } else if (referencedColumns.contains("UNIT")
+ &&
STREAM_CONSUMPTION_SELECTOR_COLUMNS.containsAll(referencedColumns)) {
+ unitConjuncts.add(conjunct);
+ }
+ }
+
Env.getCurrentEnv().getTableStreamManager().fillStreamConsumptionValuesMetadataResult(
+ dataBatch, (dbName, streamName, streamId, unit) -> {
+ List<Expression> currentConjuncts = unit == null ?
streamConjuncts : unitConjuncts;
+ if (currentConjuncts.isEmpty()) {
+ return true;
+ }
+ TreeMap<String, Object> values = new
TreeMap<>(String.CASE_INSENSITIVE_ORDER);
+ values.put("DB_NAME", dbName);
+ values.put("STREAM_NAME", streamName);
+ values.put("STREAM_ID", streamId);
+ if (unit != null) {
+ values.put("UNIT", unit);
+ }
+ return
!FrontendConjunctsUtils.isFiltered(currentConjuncts, values);
Review Comment:
[P1] Preserve the BIGINT type when evaluating STREAM_ID IN
For a stream whose ID fits in INT, `STREAM_ID IN (10000, 10001)` is analyzed
with BIGINT options, but the legacy-expression SQL round-trip prints bare
digits and reparses them as narrower integer literals. The selector inserts the
Java `long` as a `BigIntLiteral`; `FoldConstantRuleOnFE.visitInPredicate` then
uses class-sensitive `Literal.equals`, so even a matching ID folds to false and
this return prunes the stream before the retained BE filter can evaluate it.
Please preserve/coerce the option types or use numeric SQL equality semantics,
and add a serialized conversion-path test for a matching small-width stream ID.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
