CME iterating return from ServerManager.getMarkedToClose
--------------------------------------------------------
Key: HBASE-599
URL: https://issues.apache.org/jira/browse/HBASE-599
Project: Hadoop HBase
Issue Type: Bug
Reporter: stack
Below is a note from list from David Alves
{code}
I'm seeing some CMEs in the logs they occurred while I still had bad
dfs.replication settings between hadoop and hbase but still thought you
should know.
Trace:
2008-04-21 13:20:46,443 WARN
org.apache.hadoop.hbase.regionserver.HRegionServer: Processing message
(Retry: 0)
java.io.IOException: java.io.IOException:
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$ValueIterator.next(HashMap.java:822)
at
org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.java:350)
at
org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell(ServerManager.java:299)
at
org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:217)
at
org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:560)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
at
org.apache.hadoop.hbase.RemoteExceptionHandler.checkIOException(RemoteExceptionHandler.java:48)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:388)
at java.lang.Thread.run(Thread.java:619)
{code}
Looking at it, the fix would seem to be cloning the Map we give out but the
clone needs to be done inside a synchronize of the Map being cloned so some
more extensive synchronizing needs to be added to the ServerManager class.
Looking too, it seems like regionsToClose is not protected everywhere (maybe
I'm not reading it right).
Bryan, you'd probably be best handlng this issue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
