This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git
commit 36ad4a25ef58fbd7a686861646a8ab33b814d2a9 Author: yixiutt <[email protected]> AuthorDate: Tue Feb 14 00:06:14 2023 +0800 [bugfix](column_reader) index_page should not be pre-decoded (#16605) In our current logic, index page will be pre-decoded but it will return OK as index page use BinaryPlainPageBuilder and first 4 bytes of the page is a offset so it's high probablility not equal to EncodingTypePB::DICT_ENCODING which is 5. Code in bitshuffle_page_pre_decode.h ``` if constexpr (USED_IN_DICT_ENCODING) { auto type = decode_fixed32_le((const uint8_t*)&data.data[0]); if (static_cast<EncodingTypePB>(type) != EncodingTypePB::DICT_ENCODING) { return Status::OK(); } size_of_dict_header = BINARY_DICT_PAGE_HEADER_SIZE; data.remove_prefix(4); } ``` But if type just equal to EncodingTypePB::DICT_ENCODING and then it will use BitShuffle to decode BinaryPlainPage, which will leads to an fatal error. --- be/src/olap/rowset/segment_v2/column_reader.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/be/src/olap/rowset/segment_v2/column_reader.cpp b/be/src/olap/rowset/segment_v2/column_reader.cpp index 67c37277a2..430251aac2 100644 --- a/be/src/olap/rowset/segment_v2/column_reader.cpp +++ b/be/src/olap/rowset/segment_v2/column_reader.cpp @@ -157,6 +157,10 @@ Status ColumnReader::read_page(const ColumnIteratorOptions& iter_opts, const Pag opts.type = iter_opts.type; opts.encoding_info = _encoding_info; opts.io_ctx = iter_opts.io_ctx; + // index page should not pre decode + if (iter_opts.type == INDEX_PAGE) { + opts.pre_decode = false; + } return PageIO::read_and_decompress_page(opts, handle, page_body, footer); } --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
