Hi all,
Support attribute filters in table deletion by jt2594838 · Pull Request
#18079 · apache/iotdb
This PR enhances table-model DELETE by allowing delete predicates to filter
devices using attribute columns, in addition to existing time and tag
predicates.
Main changes:
1. Support attribute filters in table DELETE
DELETE FROM table WHERE ... can now use attribute columns as device filters.
The implementation first queries devices that match the attribute predicates,
then converts the matched devices into a device-level predicate for deletion.
Supported attribute predicates include:
=, !=, <, <=, >, >=, LIKE, IN, IS NULL, IS NOT NULL
Example:
DELETE FROM t
WHERE time >= 100 AND time <= 200
AND attr1 = 'red'
AND attr2 IN ('small', 'medium');
2. Refactor DELETE predicate parsing
AnalyzeUtils#parsePredicate has been refactored from a long instanceof chain
into a visitor-based parser. Each predicate type now has a dedicated handler,
making the logic easier to read and extend.
3. Generalize device predicates
The previous IDPredicate has been renamed and extended to TagPredicate, so
table deletion can represent richer device-filter semantics, including:
- tag exact match
- tag IS NULL
- tag IS NOT NULL
- combined tag predicates
- attribute filters converted into DeviceIn
4. Update deletion execution paths
The new predicate structure is propagated through the deletion path,
including analysis, planning, storage engine, data region deletion, recovery,
modification files, and compaction-related logic.
5. Add i18n messages
The PR adds or updates English and Chinese error messages for invalid delete
predicates, including unsupported operators, invalid attribute values, invalid
tag values, and invalid time ranges.
6. Add tests
The PR adds unit and integration test coverage for:
- deletion by attribute filters
- attribute + tag + time combined predicates
- LIKE, IN, comparison operators, IS NULL, and IS NOT NULL
- mixed operators in a single DELETE statement
- tables without tag columns
- devices whose tag values are NULL
- protection when too many devices are matched by attribute filters
- serialization/recovery/compaction paths affected by the predicate change
One semantic detail worth highlighting: for tables without tag columns, all
rows are treated as belonging to one logical device. Attributes are therefore
device-level metadata rather than row-level values, and later writes may update
the device attributes. The related test case has been
adjusted to reflect this behavior.
Please pay special attention to:
- whether the limit on devices matched by attribute filters is appropriate
- whether all deletion/recovery/compaction paths correctly handle the new
TagPredicate
- whether the no-tag-table and null-tag-device semantics match expectations
- whether the i18n wording needs refinement
Thanks!
Tian Jiang
