Gabriel39 commented on code in PR #67573:
URL: https://github.com/apache/doris/pull/67573#discussion_r3950439592
##########
be/src/format_v2/parquet/reader/native/column_chunk_reader.cpp:
##########
@@ -192,7 +192,12 @@ ParquetReaderCompat parquet_reader_compat(const
std::string& created_by) {
return {.parquet_816_padding =
version.VersionLt(::parquet::ApplicationVersion::PARQUET_816_FIXED_VERSION()),
.data_page_v2_always_compressed = version.VersionLt(
-
::parquet::ApplicationVersion::PARQUET_CPP_10353_FIXED_VERSION())};
+
::parquet::ApplicationVersion::PARQUET_CPP_10353_FIXED_VERSION()),
+ .null_count_trusted =
+ !version.VersionLt(
Review Comment:
Fixed in 3d1569cf71. The compatibility logic now treats the legacy
`parquet-cpp` identity as untrusted and uses `parquet-cpp-arrow 6.0.0` as the
Arrow null-count trust boundary, independently of the min/max statistics
boundary. I added Arrow 3/4/5/6 identity coverage at both row-group and
page-index layers, plus an end-to-end fixture generated by PyArrow 3.0.0 that
reports `null_count=0` while containing five NULL values. The format v2 `IS
NULL` scan now returns all five matching rows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
