[
https://issues.apache.org/jira/browse/ARROW-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921164#comment-16921164
]
Wes McKinney commented on ARROW-6417:
-------------------------------------
The dreaded {{\_\_memmove_avx_unaligned_erms}} has showed up again. I'll have a
poke at this to see what could be done
{code:java}
+ 60.85% 0.01% python libparquet.so.15.0.0
[.]
parquet::internal::TypedRecordReader<parquet::PhysicalType<(parquet::Type::ty▒
+ 58.16% 0.00% python libparquet.so.15.0.0
[.] parquet::internal::ByteArrayChunkedRecordReader::ReadValuesSpaced
▒
+ 57.54% 0.00% python libparquet.so.15.0.0
[.] parquet::PlainByteArrayDecoder::DecodeArrow
▒
+ 57.52% 5.57% python libparquet.so.15.0.0
[.]
parquet::PlainByteArrayDecoder::DecodeArrow<arrow::internal::ChunkedBinaryBui▒
+ 49.99% 4.80% python libparquet.so.15.0.0
[.] arrow::internal::ChunkedBinaryBuilder::Append
▒
+ 44.66% 9.74% python libarrow.so.15.0.0
[.] arrow::BaseBinaryBuilder<arrow::BinaryType>::Append
▒
+ 30.62% 9.64% python libarrow.so.15.0.0
[.] arrow::BufferBuilder::Append
▒
+ 23.51% 10.91% python libc-2.27.so
[.] __memmove_avx_unaligned_erms
▒
+ 21.23% 0.00% python [unknown]
[.] 0xffffffffffffffff
▒
+ 18.58% 0.01% python libparquet.so.15.0.0
[.]
parquet::ColumnReaderImplBase<parquet::PhysicalType<(parquet::Type::type)6> >▒
+ 18.45% 14.80% python libsnappy.so.1.1.7
[.] snappy::RawUncompress
▒
+ 18.42% 0.02% python libparquet.so.15.0.0
[.] parquet::SerializedPageReader::NextPage
▒
+ 18.27% 0.00% python libarrow.so.15.0.0
[.] arrow::util::SnappyCodec::Decompress
▒
+ 18.27% 0.00% python libarrow.so.15.0.0
[.] arrow::util::SnappyCodec::Decompress
▒
+ 18.27% 0.00% python libsnappy.so.1.1.7
[.] snappy::RawUncompress
▒
+ 14.99% 0.00% python libarrow.so.15.0.0
[.] arrow::PoolBuffer::Resize
▒
+ 14.99% 0.00% python libarrow.so.15.0.0
[.] arrow::PoolBuffer::Reserve
▒
+ 14.99% 0.00% python libarrow.so.15.0.0
[.] arrow::DefaultMemoryPool::Reallocate
▒
+ 14.98% 0.01% python libarrow.so.15.0.0
[.] je_arrow_rallocx
▒
+ 14.97% 0.00% python libarrow.so.15.0.0
[.] je_arrow_private_je_arena_ralloc
▒
+ 14.96% 0.00% python libarrow.so.15.0.0
[.] je_arrow_private_je_large_ralloc
▒
+ 14.64% 0.00% python libarrow.so.15.0.0
[.] arrow::BufferBuilder::Resize
▒
+ 12.82% 12.82% python [unknown]
[k] 0xffffffff98e00a67
▒
+ 11.74% 3.73% python libarrow.so.15.0.0
[.] arrow::BaseBinaryBuilder<arrow::BinaryType>::AppendNextOffset {code}
> [C++][Parquet] Non-dictionary BinaryArray reads from Parquet format have
> slowed down since 0.11.x
> -------------------------------------------------------------------------------------------------
>
> Key: ARROW-6417
> URL: https://issues.apache.org/jira/browse/ARROW-6417
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, Python
> Reporter: Wes McKinney
> Priority: Major
> Attachments: 20190903_parquet_benchmark.py,
> 20190903_parquet_read_perf.png
>
>
> In doing some benchmarking, I have found that binary reads seem to be slower
> from Arrow 0.11.1 to master branch. It would be a good idea to do some basic
> profiling to see where we might improve our memory allocation strategy (or
> whatever the bottleneck turns out to be)
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
