[
https://issues.apache.org/jira/browse/SPARK-37710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eren Avsarogullari updated SPARK-37710:
---------------------------------------
Summary: Add clear error message for java.io.IOException: Input/output
error (was: Log clear message for Input/output error)
> Add clear error message for java.io.IOException: Input/output error
> -------------------------------------------------------------------
>
> Key: SPARK-37710
> URL: https://issues.apache.org/jira/browse/SPARK-37710
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.1.1
> Reporter: Eren Avsarogullari
> Priority: Major
>
> *Input/output error* usually points environmental issues such as disk
> read/write failures due to disk corruption, network access failures etc. This
> PR aims to be added clear message to catch this kind of environmental cases
> occurring on BlockManager and logs with {*}BlockManager hostname, blockId and
> blockPath{*}.
> Following stack-trace occurred on disk corruption:
> {code:java}
> com.esotericsoftware.kryo.KryoException: java.io.IOException: Input/output
> error
> Serialization trace:
> buffers (org.apache.spark.sql.execution.columnar.DefaultCachedBatch)
> at com.esotericsoftware.kryo.io.Input.fill(Input.java:166)
> at com.esotericsoftware.kryo.io.Input.require(Input.java:196)
> at com.esotericsoftware.kryo.io.Input.readBytes(Input.java:346)
> at com.esotericsoftware.kryo.io.Input.readBytes(Input.java:326)
> at
> com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ByteArraySerializer.read(DefaultArraySerializers.java:55)
> at
> com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ByteArraySerializer.read(DefaultArraySerializers.java:38)
> at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:789)
> at
> com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:381)
> at
> com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:302)
> at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:789)
> at
> com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:816)
> at
> org.apache.spark.serializer.KryoDeserializationStream.readObject(KryoSerializer.scala:296)
> at
> org.apache.spark.serializer.DeserializationStream$$anon$1.getNext(Serializer.scala:168)
> at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:73)
> at
> org.apache.spark.storage.memory.MemoryStore.putIterator(MemoryStore.scala:221)
> at
> org.apache.spark.storage.memory.MemoryStore.putIteratorAsValues(MemoryStore.scala:299)
> at
> org.apache.spark.storage.BlockManager.maybeCacheDiskValuesInMemory(BlockManager.scala:1569)
> at
> org.apache.spark.storage.BlockManager.getLocalValues(BlockManager.scala:877)
> at org.apache.spark.storage.BlockManager.get(BlockManager.scala:1163)
> ...
> Caused by: java.io.IOException: Input/output error
> at java.io.FileInputStream.readBytes(Native Method)
> at java.io.FileInputStream.read(FileInputStream.java:255)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> net.jpountz.lz4.LZ4BlockInputStream.tryReadFully(LZ4BlockInputStream.java:269)
> at
> net.jpountz.lz4.LZ4BlockInputStream.readFully(LZ4BlockInputStream.java:280)
> at
> net.jpountz.lz4.LZ4BlockInputStream.refill(LZ4BlockInputStream.java:243)
> at net.jpountz.lz4.LZ4BlockInputStream.read(LZ4BlockInputStream.java:157)
> at com.esotericsoftware.kryo.io.Input.fill(Input.java:164)
> ... 87 more {code}
> *Sample Failure Message:*
> {code:java}
> java.io.IOException: Input/output error usually occurs due to environmental
> problems (e.g: disk corruption, network failure etc) so please check env
> status if healthy. BlockManagerId(driver, localhost, 54937, None) -
> blockName: test_my-block-id - blockDiskPath:
> /private/var/folders/kj/mccyycwn6mjdwnglw9g3k6pm0000gq/T/blockmgr-e86d8f67-a993-407f-ad3b-3cfb667b4ad4/11/test_my-block-id
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
