zhuqi-lucas commented on issue #7363: URL: https://github.com/apache/arrow-rs/issues/7363#issuecomment-2816591518
When i increase the data set from 100000 to 1000000, there were regressions about the PointLookup: And i try to fix it with this commit: https://github.com/apache/arrow-rs/commit/0c3aa9b3d4fa913029742f36b650f7b1e1ea2339 branch polish_better_decoder https://github.com/apache/arrow-rs/pull/7428 It seems it will fix the regression about PointLookup with big data set: ```rust critcmp better-decoder polish_better_decoder group better-decoder polish_better_decoder ----- -------------- --------------------- arrow_reader_row_filter/Composite/all_columns/async 1.03 12.9±0.14ms ? ?/sec 1.00 12.6±0.12ms ? ?/sec arrow_reader_row_filter/Composite/all_columns/sync 1.00 14.5±0.10ms ? ?/sec 1.00 14.6±0.31ms ? ?/sec arrow_reader_row_filter/Composite/exclude_filter_column/async 1.03 12.7±0.44ms ? ?/sec 1.00 12.4±0.20ms ? ?/sec arrow_reader_row_filter/Composite/exclude_filter_column/sync 1.02 13.3±0.32ms ? ?/sec 1.00 13.1±0.21ms ? ?/sec arrow_reader_row_filter/ModeratelySelectiveClustered/all_columns/async 1.00 12.3±0.11ms ? ?/sec 1.00 12.3±0.16ms ? ?/sec arrow_reader_row_filter/ModeratelySelectiveClustered/all_columns/sync 1.00 13.1±0.12ms ? ?/sec 1.00 13.2±0.14ms ? ?/sec arrow_reader_row_filter/ModeratelySelectiveClustered/exclude_filter_column/async 1.01 12.3±0.44ms ? ?/sec 1.00 12.1±0.17ms ? ?/sec arrow_reader_row_filter/ModeratelySelectiveClustered/exclude_filter_column/sync 1.00 12.7±0.10ms ? ?/sec 1.01 12.8±0.17ms ? ?/sec arrow_reader_row_filter/ModeratelySelectiveUnclustered/all_columns/async 1.01 41.7±0.59ms ? ?/sec 1.00 41.4±0.51ms ? ?/sec arrow_reader_row_filter/ModeratelySelectiveUnclustered/all_columns/sync 1.00 41.7±0.34ms ? ?/sec 1.00 41.6±0.36ms ? ?/sec arrow_reader_row_filter/ModeratelySelectiveUnclustered/exclude_filter_column/async 1.00 33.9±0.28ms ? ?/sec 1.00 33.8±0.29ms ? ?/sec arrow_reader_row_filter/ModeratelySelectiveUnclustered/exclude_filter_column/sync 1.00 34.3±0.28ms ? ?/sec 1.00 34.2±0.33ms ? ?/sec arrow_reader_row_filter/PointLookup/all_columns/async 3.89 10.2±0.07ms ? ?/sec 1.00 2.6±0.03ms ? ?/sec arrow_reader_row_filter/PointLookup/all_columns/sync 1.02 2.9±0.14ms ? ?/sec 1.00 2.8±0.04ms ? ?/sec arrow_reader_row_filter/PointLookup/exclude_filter_column/async 3.91 10.2±0.08ms ? ?/sec 1.00 2.6±0.04ms ? ?/sec arrow_reader_row_filter/PointLookup/exclude_filter_column/sync 1.00 2.8±0.03ms ? ?/sec 1.00 2.8±0.05ms ? ?/sec arrow_reader_row_filter/SelectiveUnclustered/all_columns/async 1.00 15.1±0.56ms ? ?/sec 1.01 15.2±0.23ms ? ?/sec arrow_reader_row_filter/SelectiveUnclustered/all_columns/sync 1.00 16.3±0.21ms ? ?/sec 1.01 16.5±0.18ms ? ?/sec arrow_reader_row_filter/SelectiveUnclustered/exclude_filter_column/async 1.00 14.0±0.16ms ? ?/sec 1.01 14.1±0.15ms ? ?/sec arrow_reader_row_filter/SelectiveUnclustered/exclude_filter_column/sync 1.01 14.8±0.54ms ? ?/sec 1.00 14.6±0.13ms ? ?/sec arrow_reader_row_filter/UnselectiveClustered/all_columns/async 1.00 20.6±0.14ms ? ?/sec 1.03 21.3±0.25ms ? ?/sec arrow_reader_row_filter/UnselectiveClustered/all_columns/sync 1.00 21.8±0.19ms ? ?/sec 1.03 22.4±0.17ms ? ?/sec arrow_reader_row_filter/UnselectiveClustered/exclude_filter_column/async 1.00 19.8±0.24ms ? ?/sec 1.02 20.2±0.18ms ? ?/sec arrow_reader_row_filter/UnselectiveClustered/exclude_filter_column/sync 1.00 20.5±0.22ms ? ?/sec 1.03 21.1±0.16ms ? ?/sec arrow_reader_row_filter/UnselectiveUnclustered/all_columns/async 1.00 15.1±0.13ms ? ?/sec 1.01 15.2±0.17ms ? ?/sec arrow_reader_row_filter/UnselectiveUnclustered/all_columns/sync 1.00 16.5±0.15ms ? ?/sec 1.00 16.4±0.14ms ? ?/sec arrow_reader_row_filter/UnselectiveUnclustered/exclude_filter_column/async 1.01 14.1±0.13ms ? ?/sec 1.00 14.0±0.18ms ? ?/sec arrow_reader_row_filter/UnselectiveUnclustered/exclude_filter_column/sync 1.00 14.6±0.10ms ? ?/sec 1.00 14.6±0.18ms ? ?/sec arrow_reader_row_filter/Utf8ViewNonEmpty/all_columns/async 1.00 95.9±0.73ms ? ?/sec 1.00 95.8±0.94ms ? ?/sec arrow_reader_row_filter/Utf8ViewNonEmpty/all_columns/sync 1.01 104.4±0.78ms ? ?/sec 1.00 103.1±1.08ms ? ?/sec arrow_reader_row_filter/Utf8ViewNonEmpty/exclude_filter_column/async 1.01 72.7±0.44ms ? ?/sec 1.00 71.8±0.57ms ? ?/sec arrow_reader_row_filter/Utf8ViewNonEmpty/exclude_filter_column/sync 1.02 73.6±0.94ms ? ?/sec 1.00 72.1±0.74ms ? ?/sec ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
