VGalaxies opened a new pull request, #19346:
URL: https://github.com/apache/iotdb/pull/19346

   ## Summary
   Fixes two correctness issues in table-model subscription tag/column 
filtering:
   
   1. **Treat absent schema-evolved `TAG` columns in older `Tablet`s as SQL 
`NULL` (`TabletTagFilter`)**:
      - After `ALTER TABLE ... ADD COLUMN new_tag STRING TAG`, historical 
TsFiles and pre-alter WAL `Tablet`s do not include `new_tag` in their column 
schema.
      - Previously, `TabletTagFilter#resolveReferencedTagColumns` 
unconditionally threw `TagFilterEvaluationException.schemaBinding("Referenced 
TAG column is missing")` even when `validateBinding` had already proven against 
`TsTable` that `new_tag` is a valid `TAG` column.
      - When table schema binding is validated (`schemaBound == true`, covering 
both literal and bound wildcard topics), absent referenced `TAG` columns now 
evaluate as SQL `NULL` (handled by `getTagValue` when `columnIndex == null`, 
preserving SQL three-valued logic). Unbound matchers without a `TsTable` 
binding continue to fail closed.
   2. **Flatten same-operator `AND`/`OR` chains in `ColumnFilterParser`**:
      - `ColumnFilterParser.AstBuilder#visitLogicalBinary` previously 
constructed left-skewed binary trees via `LogicalExpression.and(left, right)` / 
`.or(left, right)`, causing flat chains of >31 predicates (e.g., 40 `OR` terms) 
to exceed `TagFilterValidator.MAX_AST_DEPTH = 32`.
      - Same-operator `LogicalExpression` children are now flattened into a 
single n-ary `LogicalExpression` (matching how `TagFilterValidator`, 
`TagFilterEvaluator`, `ColumnFilterValidator`, and `ColumnFilterEvaluator` 
already iterate `node.getTerms()`), while preserving operator precedence (`(a 
OR b) AND c`) and still rejecting genuinely deep nesting (>32 levels of `NOT 
(...)`).
   
   ## Concrete Before-and-After Examples
   - **Schema evolution (`ALTER TABLE weather ADD COLUMN new_tag STRING TAG`)** 
with a pre-alter `Tablet(region, device, temperature)`:
     - Before: filters such as `new_tag IS NULL` or `new_tag = "v1" OR region = 
"north"` threw `TagFilterEvaluationException: Referenced TAG column is missing`.
     - After: `new_tag` evaluates as SQL `NULL` — `new_tag IS NULL` retains all 
rows, `new_tag = "v1"` and `NOT (new_tag = "v1")` drop all rows under 3VL, and 
`new_tag = "v1" OR region = "north"` retains `"north"` rows; unbound matchers 
still throw on missing columns.
   - **Flat 40-term logical chain** (`region = "r0" OR region = "r1" OR ... OR 
region = "r39"`):
     - Before: parsed as a left-skewed tree of depth 40 and rejected with 
`Tag-filter AST depth exceeds maximum of 32`.
     - After: parsed as a single `LogicalExpression(OR, 40 terms)` of depth 2 
and evaluated accurately; 33-level nested `NOT (NOT (...))` is still rejected.
   
   ## Verification
   - `mvn test -pl iotdb-core/datanode -am 
-Dsurefire.failIfNoSpecifiedTests=false 
-Dtest=TagFilterParserTest,TabletTagFilterTest,ColumnFilterParserTest,SubscriptionFilterSnapshotTest`:
 28/28 passed (7 + 8 + 9 + 4)
   - `mvn test-compile -pl iotdb-core/datanode -am -DskipTests`: exit 0
   - `mvn test-compile -pl iotdb-core/datanode -am -P with-zh-locale 
-DskipTests`: exit 0
   - `mvn spotless:check -pl iotdb-core/datanode`: exit 0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to