[
https://issues.apache.org/jira/browse/HDFS-119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249107#comment-13249107
]
Konstantin Shvachko commented on HDFS-119:
------------------------------------------
This has never been ported to 0.20. And I have seen NN stuck in wait() on all
threads, when one of the drives got full.
{code}
2012-04-05 05:25:52,817 ERROR
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Unable to sync edit log.
Fatal Error.
2012-04-05 05:25:52,818 INFO org.apache.hadoop.hdfs.server.common.Storage:
removing /name/3
2012-04-05 05:25:53,016 INFO org.apache.hadoop.ipc.Server: IPC Server handler
108 on 8020, call create(/myDir/myFile, rwxr-xr-x,
DFSClient_attempt_201204032140_3909_r_000891_0, true, 3, 134217728) from
aaa.bbb.ccc.ddd:12345: error: java.io.IOException:
java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
java.io.IOException: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLog.logSync(FSEditLog.java:961)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1123)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:551)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:524)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1386)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1382)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1380)
{code}
Looks like a critical bug to me.
> logSync() may block NameNode forever.
> -------------------------------------
>
> Key: HDFS-119
> URL: https://issues.apache.org/jira/browse/HDFS-119
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Konstantin Shvachko
> Assignee: Suresh Srinivas
> Fix For: 0.21.0
>
> Attachments: HDFS-119.patch, HDFS-119.patch
>
>
> # {{FSEditLog.logSync()}} first waits until {{isSyncRunning}} is false and
> then performs syncing to file streams by calling
> {{EditLogOutputStream.flush()}}.
> If an exception is thrown after {{isSyncRunning}} is set to {{true}} all
> threads will always wait on this condition.
> An {{IOException}} may be thrown by {{EditLogOutputStream.setReadyToFlush()}}
> or a {{RuntimeException}} may be thrown by {{EditLogOutputStream.flush()}} or
> by {{processIOError()}}.
> # The loop that calls {{eStream.flush()}} for multiple
> {{EditLogOutputStream}}-s is not synchronized, which means that another
> thread may encounter an error and modify {{editStreams}} by say calling
> {{processIOError()}}. Then the iterating process in {{logSync()}} will break
> with {{IndexOutOfBoundException}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
