Juliusz Sompolski created SPARK-23366:
-----------------------------------------
Summary: Improve hot reading path in ReadAheadInputStream
Key: SPARK-23366
URL: https://issues.apache.org/jira/browse/SPARK-23366
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.3.0
Reporter: Juliusz Sompolski
ReadAheadInputStream was introduced in
[apache/spark#18317|https://github.com/apache/spark/pull/18317] to optimize
reading spill files from disk.
However, investigating flamegraphs of profiles from investigating some
regressed workloads after switch to Spark 2.3, it seems that the hot path of
reading small amounts of data (like readInt) is inefficient - it involves
taking locks, and multiple checks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
