Caideyipi commented on code in PR #18639:
URL: https://github.com/apache/iotdb/pull/18639#discussion_r4281299710
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/payload/evolvable/batch/PipeTabletEventTsFileBatch.java:
##########
@@ -100,25 +126,22 @@ protected boolean constructBatch(final
TabletInsertionEvent event) {
(PipeInsertNodeTabletInsertionEvent) event;
final boolean isTableModel =
insertNodeTabletInsertionEvent.isTableModelEvent();
final List<Tablet> tablets =
insertNodeTabletInsertionEvent.convertToTablets();
- final List<Tablet> retainedTablets = new ArrayList<>(tablets.size());
- final List<Boolean> retainedAlignedFlags = new
ArrayList<>(tablets.size());
+ final List<TabletTransformResult> retainedTablets = new
ArrayList<>(tablets.size());
for (int i = 0; i < tablets.size(); ++i) {
- Tablet tablet = tablets.get(i);
+ final Tablet tablet = tablets.get(i);
if (isTabletEmpty(tablet)) {
continue;
}
- if (isTableModel) {
- tablet =
- pruneTableModelTablet(
- tablet,
insertNodeTabletInsertionEvent.getTableModelDatabaseName());
- if (isTabletEmpty(tablet)) {
- continue;
- }
- }
- retainedTablets.add(tablet);
- if (!isTableModel) {
-
retainedAlignedFlags.add(insertNodeTabletInsertionEvent.isAligned(i));
+ final TabletTransformResult transformedTablet =
+ transformTablet(
+ isTableModel ?
insertNodeTabletInsertionEvent.getTableModelDatabaseName() : null,
+ tablet,
+ isTableModel,
+ !isTableModel && insertNodeTabletInsertionEvent.isAligned(i));
Review Comment:
Table-model data is aligned. Changed both transformation inputs and
projected Tree View table results to report true; tree-model inputs preserve
their original alignment. Added coverage for both models and alignment values.
Commit: 7caec10c275.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/agent/SubscriptionBrokerAgent.java:
##########
@@ -815,6 +821,156 @@ public void dropColumnFilter(final String topicName) {
LOGGER.info(DataNodeMiscMessages.SUBSCRIPTION_DROP_COLUMN_FILTER,
topicName);
}
+ public void refreshTagFilter(final String topicName, final TopicConfig
topicConfig) {
+ final ITableCache tableCache = DataNodeTableCache.getInstance();
+ if (Objects.isNull(topicConfig)
+ || !topicConfig.isTableTopic()
+ || topicConfig.isTagFilterTrivial()) {
+ cacheTagFilterMatcher(
+ topicName, TagFilterMatcher.matchAll(),
tableCache.getInstanceVersion());
+ return;
+ }
+
+ for (int attempt = 0; attempt < TAG_FILTER_SCHEMA_SNAPSHOT_MAX_RETRIES;
attempt++) {
+ final long versionBeforeBinding = tableCache.getInstanceVersion();
+ final TagFilterMatcher matcher;
+ try {
+ final Map<String, Map<String, TsTable>> bindingTables =
+ getTagFilterBindingTables(topicConfig);
+ if (bindingTables == null) {
+ if (versionBeforeBinding != tableCache.getInstanceVersion()) {
+ continue;
+ }
+ cacheTagFilterMatcher(
+ topicName,
+ TagFilterMatcher.failure(
+ new SubscriptionException(
+ DataNodeMiscMessages
+
.EXCEPTION_TABLE_SCHEMA_IS_NOT_AVAILABLE_FOR_TAG_FILTER_993AB728)),
+ versionBeforeBinding);
+ LOGGER.info(
+ DataNodeMiscMessages
+
.LOG_SUBSCRIPTION_TABLE_SCHEMA_IS_NOT_AVAILABLE_FOR_TAG_FILTER_TOPIC_ARG_E864C98F,
+ topicName);
+ return;
+ }
Review Comment:
The failure now identifies the database/table and explains that its schema
is absent from this DataNode's table cache because the table may not exist or
its schema may not have synchronized yet. Repeated concurrent schema changes
and missing topic configuration also have separate diagnostics. Added EN/ZH
messages and a regression test for the missing-schema error. The evaluation
exception now preserves the detailed matcher failure instead of replacing it
with a generic unavailable message.
Commit: 7caec10c275.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/agent/SubscriptionTopicAgent.java:
##########
@@ -152,6 +157,35 @@ static boolean shouldRefreshColumnFilter(final TopicMeta
oldMeta, final TopicMet
newConfig, TopicConstant.TABLE_KEY,
TopicConstant.TABLE_DEFAULT_VALUE)));
}
+ static boolean shouldRefreshTagFilter(final TopicMeta oldMeta, final
TopicMeta newMeta) {
+ if (Objects.isNull(newMeta) || !newMeta.getConfig().isTableTopic()) {
+ return false;
+ }
+ if (Objects.isNull(oldMeta) || !oldMeta.getConfig().isTableTopic()) {
+ return true;
+ }
+
+ final TopicConfig oldConfig = oldMeta.getConfig();
+ final TopicConfig newConfig = newMeta.getConfig();
+ return !Objects.equals(
+ normalizeTagFilterValue(oldConfig.getTagFilter()),
+ normalizeTagFilterValue(newConfig.getTagFilter()))
+ || !Objects.equals(
+ normalizeTagFilterValue(
+ getAttributeIgnoreCase(
+ oldConfig, TopicConstant.DATABASE_KEY,
TopicConstant.DATABASE_DEFAULT_VALUE)),
+ normalizeTagFilterValue(
+ getAttributeIgnoreCase(
+ newConfig, TopicConstant.DATABASE_KEY,
TopicConstant.DATABASE_DEFAULT_VALUE)))
Review Comment:
Renamed it to `trimFilterBindingValue`, since it trims both the predicate
and the database/table binding inputs while preserving case.
Commit: 7caec10c275.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]