Gabriel39 commented on code in PR #66261:
URL: https://github.com/apache/doris/pull/66261#discussion_r3682600497
##########
be/test/format_v2/parquet/parquet_reader_control_test.cpp:
##########
@@ -179,11 +180,45 @@ TEST(SelectionVectorTest,
MaterializedFilterIsReusedUntilSelectionChanges) {
TEST(SelectionVectorTest, IdentitySelectionDoesNotMaterializeFilter) {
SelectionVector selection(4);
+ EXPECT_FALSE(selection.is_set());
const uint8_t* filter = reinterpret_cast<const uint8_t*>(1);
ASSERT_TRUE(selection.materialize_filter(4, 4, &filter).ok());
EXPECT_EQ(filter, nullptr);
}
+TEST(SelectionVectorTest, BulkCompactionSupportsBothFilterCoordinates) {
+ SelectionVector selection(6);
+ const uint8_t row_filter[] = {0, 1, 1, 0, 1, 0};
+ ASSERT_EQ(selection.compact_with_row_filter(row_filter, 6), 3);
+ EXPECT_EQ(selection.get_index(0), 1);
+ EXPECT_EQ(selection.get_index(1), 2);
+ EXPECT_EQ(selection.get_index(2), 4);
+
+ const uint8_t compact_filter[] = {1, 0, 1};
+ ASSERT_EQ(selection.compact_with_selection_filter(compact_filter, 3), 2);
+ EXPECT_EQ(selection.get_index(0), 1);
+ EXPECT_EQ(selection.get_index(1), 4);
+ EXPECT_TRUE(selection.verify(2, 6).ok());
+}
+
+TEST(QueryDictionaryFilterCacheTest, ReusesBitmapWithinMemoryLimit) {
Review Comment:
Added a production-path Parquet scan test with a live QueryContext. It
verifies miss -> hit reuse, predicate digest separation, dictionary fingerprint
separation, cached bitmap correctness, and that TopN dictionary filtering
bypasses this cache.
##########
be/src/format_v2/table_reader.cpp:
##########
@@ -765,6 +765,50 @@ Status TableReader::_build_table_filters_from_conjuncts() {
return Status::OK();
}
+Status TableReader::refresh_conjuncts(VExprContextSPtrs conjuncts) {
+ _conjuncts = std::move(conjuncts);
+ if (_data_reader.reader == nullptr) {
+ // The split is prepared but its physical reader has not opened yet.
open_reader() will use
+ // this newest snapshot directly, so no pending request is needed.
+ return Status::OK();
+ }
+ if (!_data_reader.reader->supports_scan_request_refresh()) {
+ return Status::OK();
+ }
+
+ RETURN_IF_ERROR(_build_table_filters_from_conjuncts());
+ auto refreshed_request = std::make_shared<FileScanRequest>();
+ RETURN_IF_ERROR(_data_reader.column_mapper->create_scan_request(
Review Comment:
Fixed by building the refreshed request with an isolated mapper, leaving the
active mapper/projection snapshot untouched until the physical reader boundary.
The request is queued only when local positions and the recursive nested
projection layout match; incompatible layouts retain scanner-level filtering.
Added a regression test that failed when refresh replaced the active prepared
projection.
##########
be/src/format_v2/table_reader.cpp:
##########
@@ -765,6 +765,50 @@ Status TableReader::_build_table_filters_from_conjuncts() {
return Status::OK();
}
+Status TableReader::refresh_conjuncts(VExprContextSPtrs conjuncts) {
+ _conjuncts = std::move(conjuncts);
+ if (_data_reader.reader == nullptr) {
Review Comment:
Fixed Hudi and Paimon hybrid readers to clone and forward refreshed
conjuncts to the active native/JNI child. Added focused tests for both wrappers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
