zhuqi-lucas commented on issue #7363: URL: https://github.com/apache/arrow-rs/issues/7363#issuecomment-2816565673
@alamb @XiangpengHao Updated, i try to optimize the code, it is 10X faster for some testing cases and no regression, and i need more testing to verify the result: With this improvement: https://github.com/apache/arrow-rs/commit/0c3aa9b3d4fa913029742f36b650f7b1e1ea2339 Full code: https://github.com/apache/arrow-rs/pull/7428 ```rust critcmp better_decode polish_better_decoder group better_decode polish_better_decoder ----- ------------- --------------------- arrow_reader_row_filter/Composite/all_columns/async 7.27 12.7±0.14ms ? ?/sec 1.00 1749.4±27.27µs ? ?/sec arrow_reader_row_filter/Composite/all_columns/sync 1.05 2.1±0.03ms ? ?/sec 1.00 2.0±0.03ms ? ?/sec arrow_reader_row_filter/Composite/exclude_filter_column/async 7.44 12.4±0.29ms ? ?/sec 1.00 1668.5±25.37µs ? ?/sec arrow_reader_row_filter/Composite/exclude_filter_column/sync 1.05 1871.5±38.32µs ? ?/sec 1.00 1777.2±26.56µs ? ?/sec arrow_reader_row_filter/ModeratelySelectiveClustered/all_columns/async 7.86 12.3±0.25ms ? ?/sec 1.00 1565.1±23.67µs ? ?/sec arrow_reader_row_filter/ModeratelySelectiveClustered/all_columns/sync 1.10 1901.6±57.07µs ? ?/sec 1.00 1734.9±19.45µs ? ?/sec arrow_reader_row_filter/ModeratelySelectiveClustered/exclude_filter_column/async 7.78 12.1±0.15ms ? ?/sec 1.00 1551.8±31.45µs ? ?/sec arrow_reader_row_filter/ModeratelySelectiveClustered/exclude_filter_column/sync 1.09 1803.2±77.68µs ? ?/sec 1.00 1660.0±34.17µs ? ?/sec arrow_reader_row_filter/ModeratelySelectiveUnclustered/all_columns/async 9.25 41.9±0.45ms ? ?/sec 1.00 4.5±0.04ms ? ?/sec arrow_reader_row_filter/ModeratelySelectiveUnclustered/all_columns/sync 1.04 4.8±0.12ms ? ?/sec 1.00 4.6±0.04ms ? ?/sec arrow_reader_row_filter/ModeratelySelectiveUnclustered/exclude_filter_column/async 8.99 34.5±0.31ms ? ?/sec 1.00 3.8±0.04ms ? ?/sec arrow_reader_row_filter/ModeratelySelectiveUnclustered/exclude_filter_column/sync 1.05 4.1±0.05ms ? ?/sec 1.00 3.9±0.04ms ? ?/sec arrow_reader_row_filter/PointLookup/all_columns/async 8.45 10.8±0.39ms ? ?/sec 1.00 1276.3±29.53µs ? ?/sec arrow_reader_row_filter/PointLookup/all_columns/sync 1.13 1553.2±25.69µs ? ?/sec 1.00 1378.0±37.77µs ? ?/sec arrow_reader_row_filter/PointLookup/exclude_filter_column/async 8.26 10.5±0.41ms ? ?/sec 1.00 1266.9±18.21µs ? ?/sec arrow_reader_row_filter/PointLookup/exclude_filter_column/sync 1.14 1549.8±45.82µs ? ?/sec 1.00 1355.6±22.66µs ? ?/sec arrow_reader_row_filter/SelectiveUnclustered/all_columns/async 7.73 15.2±0.36ms ? ?/sec 1.00 1961.3±28.47µs ? ?/sec arrow_reader_row_filter/SelectiveUnclustered/all_columns/sync 1.12 2.4±0.09ms ? ?/sec 1.00 2.2±0.03ms ? ?/sec arrow_reader_row_filter/SelectiveUnclustered/exclude_filter_column/async 8.09 14.6±0.53ms ? ?/sec 1.00 1808.5±30.74µs ? ?/sec arrow_reader_row_filter/SelectiveUnclustered/exclude_filter_column/sync 1.10 2.1±0.06ms ? ?/sec 1.00 1926.8±24.00µs ? ?/sec arrow_reader_row_filter/UnselectiveClustered/all_columns/async 9.82 20.6±0.54ms ? ?/sec 1.00 2.1±0.03ms ? ?/sec arrow_reader_row_filter/UnselectiveClustered/all_columns/sync 1.05 2.4±0.04ms ? ?/sec 1.00 2.3±0.02ms ? ?/sec arrow_reader_row_filter/UnselectiveClustered/exclude_filter_column/async 9.84 19.6±0.14ms ? ?/sec 1.00 1994.2±21.19µs ? ?/sec arrow_reader_row_filter/UnselectiveClustered/exclude_filter_column/sync 1.04 2.2±0.05ms ? ?/sec 1.00 2.1±0.04ms ? ?/sec arrow_reader_row_filter/UnselectiveUnclustered/all_columns/async 7.63 15.1±0.16ms ? ?/sec 1.00 1981.6±29.41µs ? ?/sec arrow_reader_row_filter/UnselectiveUnclustered/all_columns/sync 1.08 2.4±0.09ms ? ?/sec 1.00 2.2±0.04ms ? ?/sec arrow_reader_row_filter/UnselectiveUnclustered/exclude_filter_column/async 7.73 14.1±0.47ms ? ?/sec 1.00 1827.8±21.50µs ? ?/sec arrow_reader_row_filter/UnselectiveUnclustered/exclude_filter_column/sync 1.08 2.1±0.05ms ? ?/sec 1.00 1928.2±25.36µs ? ?/sec arrow_reader_row_filter/Utf8ViewNonEmpty/all_columns/async 9.78 97.1±1.43ms ? ?/sec 1.00 9.9±0.07ms ? ?/sec arrow_reader_row_filter/Utf8ViewNonEmpty/all_columns/sync 1.04 11.0±0.30ms ? ?/sec 1.00 10.6±0.10ms ? ?/sec arrow_reader_row_filter/Utf8ViewNonEmpty/exclude_filter_column/async 9.91 73.6±1.52ms ? ?/sec 1.00 7.4±0.06ms ? ?/sec arrow_reader_row_filter/Utf8ViewNonEmpty/exclude_filter_column/sync 1.07 8.0±0.29ms ? ?/sec 1.00 7.5±0.07ms ? ?/sec ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
