[
https://issues.apache.org/jira/browse/ARROW-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922758#comment-16922758
]
Wes McKinney commented on ARROW-6417:
-------------------------------------
So on closer inspection, in v0.11.1 we weren't yet handling chunked binary
reads at all, so the comparison is not really apples to oranges. v0.12.x was
the first release series to include chunking support, so could be the more
appropriate comparison.
This performance issue is really vexing. We also changed jemalloc versions
between 0.12.x and 0.15.x so I wonder if the allocator version could be
impacting performance
> [C++][Parquet] Non-dictionary BinaryArray reads from Parquet format have
> slowed down since 0.11.x
> -------------------------------------------------------------------------------------------------
>
> Key: ARROW-6417
> URL: https://issues.apache.org/jira/browse/ARROW-6417
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, Python
> Reporter: Wes McKinney
> Priority: Major
> Labels: pull-request-available
> Attachments: 20190903_parquet_benchmark.py,
> 20190903_parquet_read_perf.png
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In doing some benchmarking, I have found that binary reads seem to be slower
> from Arrow 0.11.1 to master branch. It would be a good idea to do some basic
> profiling to see where we might improve our memory allocation strategy (or
> whatever the bottleneck turns out to be)
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
