Caideyipi opened a new pull request, #18639:
URL: https://github.com/apache/iotdb/pull/18639
## Description
Table subscriptions can select databases, tables, and output columns, but
they cannot filter individual rows by TAG value. This PR adds the `tag-filter`
topic attribute for table-model subscriptions. Its default value is `true`, so
existing topics keep their current behavior.
### Filter semantics
- Parse a restricted SQL predicate over TAG columns. The supported forms are
`AND`, `OR`, `NOT`, `=`, `!=`, `IN`, `LIKE`, `regexp_like`, and `IS NULL`,
evaluated with SQL three-valued logic.
- Validate expressions on CREATE/ALTER and bind referenced fields against
table schema. Missing schema, stale bindings, unknown fields, and non-TAG
fields fail closed instead of leaking unfiltered rows.
- Filter Tablet rows while preserving row order, data types, timestamps, and
null bitmaps. TAG filtering runs before `column-filter` projection.
### Delivery paths and configuration changes
- Apply the filter to Record/Tablet and TsFile delivery, including normal
subscription queues, consensus subscription queues, and Tree View projection.
- Snapshot filter configuration per event batch so ALTER TOPIC takes effect
on subsequent batches without changing the meaning of an in-flight batch.
- Rebind matchers when table schema versions change, and clean up temporary
TsFile rewrite resources on empty output and failure paths.
### Validation
- DataNode Spotless and Checkstyle: passed.
- `TopicConfigTest`: 10 tests passed.
- `SubscriptionConsumerMultiProviderPollTest`: 3 tests passed.
- Focused DataNode subscription tests: 36 tests passed before the final
review fixes.
- `git diff --check origin/master...HEAD`: passed after rebasing onto
current `origin/master`.
- A fresh DataNode compile reached the recompilation of 2,987 sources but
was stopped after about 20 minutes because of local resource limits; the clean
reactor build and integration tests are left to CI.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] concurrent read
- [x] concurrent write
- [x] concurrent read and write
- [x] added comments explaining the intent where it would not be obvious to
an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths.
- [x] added integration tests.
<hr>
##### Key changed/added classes (or packages if there are too many classes)
in this PR
- `org.apache.iotdb.db.subscription.tagfilter`
- `SubscriptionFilterSnapshot`
- `SubscriptionTreeViewProjector`
- `SubscriptionBrokerAgent`
- `ConsensusLogToTabletConverter`
- `PipeTabletEventTsFileBatch`
- `TopicConfig` and `TopicMeta`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]