[ http://issues.apache.org/jira/browse/HADOOP-286?page=all ]
Konstantin Shvachko updated HADOOP-286:
---------------------------------------
Attachment: Lease.patch
This is a very simple patch that renews leases only when pendingCreates is not
empty.
This prevents the client from sending lease renewal messages when the client
is not writing into dfs, say just reading or doing local stuff.
This should make the name node less busy.
I tried to change the ipc.client.timeout from 60 secs to 20 secs.
On my 3 node cluster everything worked fine.
On a large cluster the timeout was changed only for the DFSClient.
The LeaseExpiredException does not appear anymore.
But we need more statistics on that, especially with slower networks.
The ipc timeout is global for all ipc connections, so if we make it
smaller there is a risk that long lasting operations like block transfers
will start to timeout. I haven't seen it.
If anybody is willing to try 20 sec ipc timeout please post the results.
Failing early, and retrying might make things faster in general.
> copyFromLocal throws LeaseExpiredException
> ------------------------------------------
>
> Key: HADOOP-286
> URL: http://issues.apache.org/jira/browse/HADOOP-286
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.3.0
> Environment: redhar linux
> Reporter: Runping Qi
> Attachments: Lease.patch
>
>
> Loading local files to dfs through hadoop dfs -copyFromLocal failed due to
> the following exception:
> copyFromLocal: org.apache.hadoop.dfs.LeaseExpiredException: No lease on
> output_crawled.1.txt
> at
> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:414)
> at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:190)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:243)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:231)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
