jiangshifu commented on issue #50548: URL: https://github.com/apache/arrow/issues/50548#issuecomment-5021562796
> [@jiangshifu](https://github.com/jiangshifu) Can you please make your bug report understandable? You should be writing for other people who will read you. I think parthpathak1201 has understood the problem 1）a parquet file single col(6w rows in a row Group), no compression, no dict 2）when i called record_reader->SkipRecords(59000); #which consume 1.1ms 3）using flame graph，we can see（73% cpu is this call） parquet::internal::standard::FindMinMaxImpl level_comparison_inc.h:51 parquet::internal::FindMinMax level_comparison.cc:77 parquet::LevelDecoder::Decode column_reader.cc:190 parquet::ColumnReaderImplBase::ReadDefinitionLevels column_reader.cc:1004 parquet::TypedColumnReaderImpl::ReadLevels column_reader.cc:1347 parquet::TypedColumnReaderImpl::ReadBatch column_reader.cc:1430 parquet::TypedColumnReaderImpl::Skip column_reader.cc:1553 parquet::internal::TypedRecordReader::SkipRecords column_reader.cc:1946 readCol skip_record.cpp:400 ReadPagesWithPageIndex skip_record.cpp:461 main skip_record.cpp:481 <unknown> 0x0000ffff91486040 __libc_start_main 0x0000ffff91486118 _start 0x00000000004219b0 in such senario, will call FindMinMaxImpl 59000 times -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
