lszskye commented on code in PR #485:
URL: https://github.com/apache/paimon-cpp/pull/485#discussion_r4489902404
##########
src/paimon/core/table/source/data_evolution_full_text_scan.cpp:
##########
@@ -156,6 +165,27 @@ bool DataEvolutionFullTextScan::IndexesField(const
GlobalIndexMeta& meta, int32_
meta.extra_field_ids->end();
}
+bool DataEvolutionFullTextScan::IsScalarPreFilterFile(const GlobalIndexMeta&
meta,
+ const std::set<int32_t>&
filter_field_ids,
+ bool supports_full_text)
{
+ bool has_extra_fields = meta.extra_field_ids &&
!meta.extra_field_ids->empty();
+ return IndexesAnyField(meta, filter_field_ids) && (!supports_full_text ||
has_extra_fields);
Review Comment:
Multi-field indexes are collected here and attached to the splits, but
`CreateForScalarFilters()` later skips them unconditionally. Could we exclude
them during planning as well, to avoid unnecessary metadata validation and
split serialization? If retaining them serves another purpose, a comment
explaining it would help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]