[
https://issues.apache.org/jira/browse/ORC-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oscar Torreno updated ORC-1787:
-------------------------------
Description:
We have an application that does RecordReader building & row reading
concurrently. Our code worked fine on ORC v1.9.1 but it started sporadically
failing with ORC 2.0.2. We failed to find any claims about thread-safety
guaranties for record reader creation. Are we supposed to construct
RecordReaders in parallel for a single file and it's an implementation bug? or
it's not supported and RecordReader instances creation should be synchronized?
when we synchronize the creation the problem goes away
The stacktrace:
{code:java}
java.nio.channels.AsynchronousCloseException
at
sun.nio.ch.SimpleAsynchronousFileChannelImpl$2.run(SimpleAsynchronousFileChannelImpl.java:326)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:840)
[wrapped] java.util.concurrent.ExecutionException:
java.nio.channels.AsynchronousCloseException
at
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
at
org.apache.orc.impl.RecordReaderUtils.readDiskRangesVectored(RecordReaderUtils.java:595)
[wrapped] java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.nio.channels.AsynchronousCloseException
at
org.apache.orc.impl.RecordReaderUtils.readDiskRangesVectored(RecordReaderUtils.java:597)
at
org.apache.orc.impl.RecordReaderUtils$DefaultDataReader.readFileData(RecordReaderUtils.java:114)
at org.apache.orc.impl.reader.StripePlanner.readData(StripePlanner.java:178)
at
org.apache.orc.impl.RecordReaderImpl.readStripe(RecordReaderImpl.java:1314)
at
org.apache.orc.impl.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:1354)
at
org.apache.orc.impl.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1397)
at
org.apache.orc.impl.RecordReaderImpl.<init>(RecordReaderImpl.java:367){code}
was:
We have an application that does RecordReader building & row reading
concurrently. Our code worked fine on ORC v1.9.1 but it started sporadically
failing with ORC 2.0.2. We failed to find any claims about thread-safety
guaranties for record reader creation. Are we supposed to construct
RecordReaders in parallel for a single file and it's an implementation bug? or
it's not supported and RecordReader instances creation should be synchronized?
when we synchronize the creation the problem goes away
The stacktrace:
{code:java}
java.nio.channels.AsynchronousCloseException at
sun.nio.ch.SimpleAsynchronousFileChannelImpl$2.run(SimpleAsynchronousFileChannelImpl.java:326)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:840) [wrapped]
java.util.concurrent.ExecutionException:
java.nio.channels.AsynchronousCloseException at
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) at
org.apache.orc.impl.RecordReaderUtils.readDiskRangesVectored(RecordReaderUtils.java:595)
[wrapped] java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.nio.channels.AsynchronousCloseException at
org.apache.orc.impl.RecordReaderUtils.readDiskRangesVectored(RecordReaderUtils.java:597)
at
org.apache.orc.impl.RecordReaderUtils$DefaultDataReader.readFileData(RecordReaderUtils.java:114)
at org.apache.orc.impl.reader.StripePlanner.readData(StripePlanner.java:178)
at org.apache.orc.impl.RecordReaderImpl.readStripe(RecordReaderImpl.java:1314)
at
org.apache.orc.impl.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:1354)
at
org.apache.orc.impl.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1397)
at org.apache.orc.impl.RecordReaderImpl.<init>(RecordReaderImpl.java:367)
{code}
> Failure on readDiskRangesVectored with concurrent record readers
> ----------------------------------------------------------------
>
> Key: ORC-1787
> URL: https://issues.apache.org/jira/browse/ORC-1787
> Project: ORC
> Issue Type: Bug
> Affects Versions: 2.0.2
> Reporter: Oscar Torreno
> Priority: Major
>
> We have an application that does RecordReader building & row reading
> concurrently. Our code worked fine on ORC v1.9.1 but it started sporadically
> failing with ORC 2.0.2. We failed to find any claims about thread-safety
> guaranties for record reader creation. Are we supposed to construct
> RecordReaders in parallel for a single file and it's an implementation bug?
> or it's not supported and RecordReader instances creation should be
> synchronized? when we synchronize the creation the problem goes away
>
> The stacktrace:
> {code:java}
> java.nio.channels.AsynchronousCloseException
> at
> sun.nio.ch.SimpleAsynchronousFileChannelImpl$2.run(SimpleAsynchronousFileChannelImpl.java:326)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.lang.Thread.run(Thread.java:840)
> [wrapped] java.util.concurrent.ExecutionException:
> java.nio.channels.AsynchronousCloseException
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
> at
> org.apache.orc.impl.RecordReaderUtils.readDiskRangesVectored(RecordReaderUtils.java:595)
> [wrapped] java.lang.RuntimeException:
> java.util.concurrent.ExecutionException:
> java.nio.channels.AsynchronousCloseException
> at
> org.apache.orc.impl.RecordReaderUtils.readDiskRangesVectored(RecordReaderUtils.java:597)
> at
> org.apache.orc.impl.RecordReaderUtils$DefaultDataReader.readFileData(RecordReaderUtils.java:114)
> at
> org.apache.orc.impl.reader.StripePlanner.readData(StripePlanner.java:178)
> at
> org.apache.orc.impl.RecordReaderImpl.readStripe(RecordReaderImpl.java:1314)
> at
> org.apache.orc.impl.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:1354)
> at
> org.apache.orc.impl.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1397)
> at
> org.apache.orc.impl.RecordReaderImpl.<init>(RecordReaderImpl.java:367){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
