[
https://issues.apache.org/jira/browse/YARN-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17143689#comment-17143689
]
bianqi commented on YARN-4322:
------------------------------
{quote}EBADF: Bad file descriptor at
org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method) at
org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
at
org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
at
org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 2020-06-24 16:39:04,960 WARN
[Readahead Thread #3] org.apache.hadoop.io.ReadaheadPool: Failed readahead on
ifile EBADF: Bad file descriptor at
org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method) at
org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
at
org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
at
org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{quote}
I have encountered this problem too.
> local framwork + io.ReadaheadPool is throwing Failed readahead on ifile
> EBADF: Bad file descriptor
> --------------------------------------------------------------------------------------------------
>
> Key: YARN-4322
> URL: https://issues.apache.org/jira/browse/YARN-4322
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.7.1
> Environment: Linux 2.6.32.12-0.7-default x86_64
> Reporter: Mohammad Shahid Khan
> Priority: Major
>
> run the pi job with 100 map and 2 reduce
> io.ReadaheadPool is throwing Failed readahead on ifile EBADF: Bad file
> descriptor
> stacktrace
> {code}
> 15/10/30 16:47:23 INFO reduce.MergeManagerImpl: closeInMemoryFile ->
> map-output of size: 24, inMemoryMapOutputs.size() -> 89, commitMemory ->
> 2112, usedMemory ->2136
> 15/10/30 16:47:23 WARN io.ReadaheadPool: Failed readahead on ifile
> EBADF: Bad file descriptor
> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native
> Method)
> at
> org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
> at
> org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
> at
> org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 15/10/30 16:47:23 WARN io.ReadaheadPool: Failed readahead on ifile
> EBADF: Bad file descriptor
> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native
> Method)
> at
> org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
> at
> org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
> at
> org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 15/10/30 16:47:23 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle
> output of map attempt_local108678459_0001_m_000086_0 decomp: 24 len: 28 to
> MEMORY
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
