Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
kaka11chen commented on PR #44098: URL: https://github.com/apache/doris/pull/44098#issuecomment-2491552595 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
kaka11chen commented on PR #44098: URL: https://github.com/apache/doris/pull/44098#issuecomment-2491427460 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
kaka11chen commented on PR #44098: URL: https://github.com/apache/doris/pull/44098#issuecomment-2491313643 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
github-actions[bot] commented on code in PR #44098:
URL: https://github.com/apache/doris/pull/44098#discussion_r1852103760
##
be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:
##
@@ -476,7 +560,7 @@
}
Status ScalarColumnReader::read_column_data(ColumnPtr& doris_column,
DataTypePtr& type,
Review Comment:
warning: function 'read_column_data' has cognitive complexity of 81
(threshold 50) [readability-function-cognitive-complexity]
```cpp
Status ScalarColumnReader::read_column_data(ColumnPtr& doris_column,
DataTypePtr& type,
^
```
Additional context
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:564:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
if (_converter == nullptr) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:567:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (!_converter->support()) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:576:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
do {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:577:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (_chunk_reader->remaining_num_values() == 0) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:578:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
if (!_chunk_reader->has_next_page()) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:583:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_chunk_reader->next_page());
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:583:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(_chunk_reader->next_page());
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:585:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (_nested_column) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:586:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_chunk_reader->load_page_data_idempotent());
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:586:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(_chunk_reader->load_page_data_idempotent());
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:587:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_read_nested_column(resolved_column,
resolved_type, filter_map,
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:587:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(_read_nested_column(resolved_column,
resolved_type, filter_map,
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:597:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (read_ranges.size() == 0) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:600:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_chunk_reader->skip_page());
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:600:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(_chunk_re
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
kaka11chen commented on PR #44098: URL: https://github.com/apache/doris/pull/44098#issuecomment-2483557268 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
github-actions[bot] commented on code in PR #44098:
URL: https://github.com/apache/doris/pull/44098#discussion_r1846925743
##
be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:
##
@@ -445,7 +437,7 @@ Status RowGroupReader::_read_column_data(Block* block,
const std::vector
Additional context
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:445:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
for (uint32_t i = 0; i < origin_column_num; ++i) {
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:450:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
while (!_state->is_cancelled()) {
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:455:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(_read_column_data(block,
_lazy_read_ctx.predicate_columns.first, batch_size,
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:455:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_read_column_data(block,
_lazy_read_ctx.predicate_columns.first, batch_size,
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:457:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (pre_read_rows == 0) {
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:462:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(_fill_partition_columns(block, pre_read_rows,
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:462:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_fill_partition_columns(block, pre_read_rows,
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:464:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(_fill_missing_columns(block, pre_read_rows,
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:464:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_fill_missing_columns(block, pre_read_rows,
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:467:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(_build_pos_delete_filter(pre_read_rows));
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:467:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_build_pos_delete_filter(pre_read_rows));
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:470:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (_lazy_read_ctx.resize_first_column) {
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:478:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (_position_delete_ctx.has_filter) {
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:489:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(VExprContext::execute_conjuncts(filter_contexts,
&filters, block,
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:489:** +3,
includi
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
github-actions[bot] commented on code in PR #44098:
URL: https://github.com/apache/doris/pull/44098#discussion_r1846701820
##
be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:
##
@@ -476,7 +551,7 @@
}
Status ScalarColumnReader::read_column_data(ColumnPtr& doris_column,
DataTypePtr& type,
Review Comment:
warning: function 'read_column_data' has cognitive complexity of 81
(threshold 50) [readability-function-cognitive-complexity]
```cpp
Status ScalarColumnReader::read_column_data(ColumnPtr& doris_column,
DataTypePtr& type,
^
```
Additional context
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:555:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
if (_converter == nullptr) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:558:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (!_converter->support()) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:567:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
do {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:568:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (_chunk_reader->remaining_num_values() == 0) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:569:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
if (!_chunk_reader->has_next_page()) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:574:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_chunk_reader->next_page());
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:574:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(_chunk_reader->next_page());
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:576:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (_nested_column) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:577:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_chunk_reader->load_page_data_idempotent());
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:577:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(_chunk_reader->load_page_data_idempotent());
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:578:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_read_nested_column(resolved_column,
resolved_type, filter_map,
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:578:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(_read_nested_column(resolved_column,
resolved_type, filter_map,
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:588:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (read_ranges.size() == 0) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:591:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_chunk_reader->skip_page());
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:591:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(_chunk_re
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
kaka11chen commented on PR #44098: URL: https://github.com/apache/doris/pull/44098#issuecomment-2483231643 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
github-actions[bot] commented on code in PR #44098:
URL: https://github.com/apache/doris/pull/44098#discussion_r1845990029
##
be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:
##
@@ -316,9 +319,15 @@ Status ScalarColumnReader::_read_values(size_t num_values,
ColumnPtr& doris_colu
* whether the reader should read the remaining value of the last row in
previous page.
*/
Status ScalarColumnReader::_read_nested_column(ColumnPtr& doris_column,
DataTypePtr& type,
Review Comment:
warning: function '_read_nested_column' has cognitive complexity of 77
(threshold 50) [readability-function-cognitive-complexity]
```cpp
Status ScalarColumnReader::_read_nested_column(ColumnPtr& doris_column,
DataTypePtr& type,
^
```
Additional context
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:331:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
if (align_rows) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:336:** +1,
nesting level increased to 1
```cpp
} else {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:345:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
if (has_rep_level) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:347:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
while (parsed_rows <= batch_size && remaining_values > 0) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:347:** +1
```cpp
while (parsed_rows <= batch_size && remaining_values > 0) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:349:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
if (rep_level == 0) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:350:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
if (parsed_rows == batch_size) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:360:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (filter_map.has_filter()) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:364:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(filter_map.generate_nested_filter_map(
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:364:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(filter_map.generate_nested_filter_map(
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:369:** +1,
nesting level increased to 1
```cpp
} else if (!align_rows) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:379:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
if (has_def_level) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:381:** +1,
nesting level increased to 1
```cpp
} else {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:388:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
if (doris_column->is_nullable()) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:394:** +1,
nesting level increased to 1
```cpp
} else {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:395:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (_field_schema->is_nullable) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:405:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
while (has_read < origin_size + parsed_values) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:408:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
while (has_read < origin_size + parsed_values &&
_def_levels[has_read] == def_level) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:412:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (def_level < _field_schema->repeated_parent_def_level) {
^
```
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
kaka11chen commented on PR #44098: URL: https://github.com/apache/doris/pull/44098#issuecomment-2482116551 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
kaka11chen commented on PR #44098: URL: https://github.com/apache/doris/pull/44098#issuecomment-2481408310 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
github-actions[bot] commented on code in PR #44098:
URL: https://github.com/apache/doris/pull/44098#discussion_r1845553963
##
be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:
##
@@ -476,7 +525,7 @@ Status ScalarColumnReader::_try_load_dict_page(bool*
loaded, bool* has_dict) {
}
Status ScalarColumnReader::read_column_data(ColumnPtr& doris_column,
DataTypePtr& type,
Review Comment:
warning: function 'read_column_data' has cognitive complexity of 81
(threshold 50) [readability-function-cognitive-complexity]
```cpp
Status ScalarColumnReader::read_column_data(ColumnPtr& doris_column,
DataTypePtr& type,
^
```
Additional context
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:529:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
if (_converter == nullptr) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:532:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (!_converter->support()) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:541:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
do {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:542:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (_chunk_reader->remaining_num_values() == 0) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:543:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
if (!_chunk_reader->has_next_page()) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:548:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_chunk_reader->next_page());
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:548:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(_chunk_reader->next_page());
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:550:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (_nested_column) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:551:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_chunk_reader->load_page_data_idempotent());
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:551:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(_chunk_reader->load_page_data_idempotent());
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:552:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_read_nested_column(resolved_column,
resolved_type, filter_map,
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:552:** +4,
including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR(_read_nested_column(resolved_column,
resolved_type, filter_map,
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:562:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (read_ranges.size() == 0) {
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:565:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_chunk_reader->skip_page());
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:565:** +4,
including nesting penalty of 3,
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
doris-robot commented on PR #44098: URL: https://github.com/apache/doris/pull/44098#issuecomment-2481408264 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
kaka11chen commented on PR #44098: URL: https://github.com/apache/doris/pull/44098#issuecomment-2481414630 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](parquet-reader)Implement late materialization of parquet comple… [doris]
github-actions[bot] commented on code in PR #44098:
URL: https://github.com/apache/doris/pull/44098#discussion_r1845550897
##
be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:
##
@@ -445,7 +446,7 @@ Status RowGroupReader::_read_column_data(Block* block,
const std::vector
Additional context
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:454:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
for (uint32_t i = 0; i < origin_column_num; ++i) {
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:459:** +1,
including nesting penalty of 0, nesting level increased to 1
```cpp
while (!_state->is_cancelled()) {
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:464:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(_read_column_data(block,
_lazy_read_ctx.predicate_columns.first, batch_size,
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:464:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_read_column_data(block,
_lazy_read_ctx.predicate_columns.first, batch_size,
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:466:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (pre_read_rows == 0) {
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:471:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(_fill_partition_columns(block, pre_read_rows,
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:471:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_fill_partition_columns(block, pre_read_rows,
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:473:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(_fill_missing_columns(block, pre_read_rows,
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:473:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_fill_missing_columns(block, pre_read_rows,
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:476:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(_build_pos_delete_filter(pre_read_rows));
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:476:** +3,
including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(_build_pos_delete_filter(pre_read_rows));
^
```
**be/src/common/status.h:633:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:479:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (_lazy_read_ctx.resize_first_column) {
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:487:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
if (_position_delete_ctx.has_filter) {
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:498:** +2,
including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(VExprContext::execute_conjuncts(filter_contexts,
&filters, block,
^
```
**be/src/common/status.h:631:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do {\
^
```
**be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:498:** +3,
includi
