[
https://issues.apache.org/jira/browse/HADOOP-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560615#action_12560615
]
Billy Pearson commented on HADOOP-2662:
---------------------------------------
10.0.0.4 was the server I was getting the above log from.
{code}
2008-01-18 17:52:41,721 INFO org.apache.hadoop.hbase.HMaster: Cluster shutdown
requested. Starting to quiesce servers
2008-01-18 17:52:46,588 INFO org.apache.hadoop.hbase.HMaster: Region server
10.0.0.3:60020 quiesced
2008-01-18 17:52:46,837 INFO org.apache.hadoop.hbase.HMaster: Region server
10.0.0.2:60020 quiesced
2008-01-18 17:52:47,193 DEBUG org.apache.hadoop.hbase.HMaster: Region server
10.0.0.4:60020: MSG_REPORT_EXITING -- cancelling lease
2008-01-18 17:52:47,193 INFO org.apache.hadoop.hbase.HMaster: Cancelling lease
for 10.0.0.4:60020
2008-01-18 17:52:47,194 INFO org.apache.hadoop.hbase.HMaster: Region server
10.0.0.4:60020: MSG_REPORT_EXITING -- lease cancelled
2008-01-18 17:52:47,263 DEBUG org.apache.hadoop.hbase.HMaster: Region server
10.0.0.1:60020: MSG_REPORT_EXITING -- cancelling lease
2008-01-18 17:52:47,263 INFO org.apache.hadoop.hbase.HMaster: Cancelling lease
for 10.0.0.1:60020
2008-01-18 17:52:47,263 INFO org.apache.hadoop.hbase.HMaster: Region server
10.0.0.1:60020: MSG_REPORT_EXITING -- lease cancelled
2008-01-18 17:52:47,651 DEBUG org.apache.hadoop.hbase.HMaster: Region server
10.0.0.4:60020: MSG_REPORT_EXITING -- cancelling lease
2008-01-18 17:52:47,655 DEBUG org.apache.hadoop.hbase.HMaster: Region server
10.0.0.1:60020: MSG_REPORT_EXITING -- cancelling lease
2008-01-18 17:52:49,592 INFO org.apache.hadoop.hbase.HMaster: All user tables
quiesced. Proceeding with shutdown
2008-01-18 17:52:49,593 DEBUG org.apache.hadoop.hbase.HMaster: telling root
scanner to stop
2008-01-18 17:52:49,593 DEBUG org.apache.hadoop.hbase.HMaster: telling meta
scanner to stop
2008-01-18 17:52:49,593 DEBUG org.apache.hadoop.hbase.HMaster: meta and root
scanners notified
2008-01-18 17:52:49,593 INFO org.apache.hadoop.hbase.HMaster$RootScanner:
HMaster.rootScanner exiting
2008-01-18 17:52:49,593 INFO org.apache.hadoop.hbase.HMaster$MetaScanner:
HMaster.metaScanner exiting
2008-01-18 17:52:49,841 INFO org.apache.hadoop.hbase.HMaster: All user tables
quiesced. Proceeding with shutdown
2008-01-18 17:52:49,841 DEBUG org.apache.hadoop.hbase.HMaster: telling root
scanner to stop
2008-01-18 17:52:49,841 DEBUG org.apache.hadoop.hbase.HMaster: telling meta
scanner to stop
2008-01-18 17:52:49,841 DEBUG org.apache.hadoop.hbase.HMaster: meta and root
scanners notified
2008-01-18 17:52:50,081 DEBUG org.apache.hadoop.hbase.HMaster: Region server
10.0.0.3:60020: MSG_REPORT_EXITING -- cancelling lease
2008-01-18 17:52:50,081 INFO org.apache.hadoop.hbase.HMaster: Cancelling lease
for 10.0.0.3:60020
2008-01-18 17:52:50,081 INFO org.apache.hadoop.hbase.HMaster: Region server
10.0.0.3:60020: MSG_REPORT_EXITING -- lease cancelled
2008-01-18 17:52:50,307 DEBUG org.apache.hadoop.hbase.HMaster: Region server
10.0.0.2:60020: MSG_REPORT_EXITING -- cancelling lease
2008-01-18 17:52:50,307 INFO org.apache.hadoop.hbase.HMaster: Cancelling lease
for 10.0.0.2:60020
2008-01-18 17:52:50,307 INFO org.apache.hadoop.hbase.HMaster: Region server
10.0.0.2:60020: MSG_REPORT_EXITING -- lease cancelled
2008-01-18 17:52:57,568 DEBUG org.apache.hadoop.hbase.HMaster: telling root
scanner to stop
2008-01-18 17:52:57,568 DEBUG org.apache.hadoop.hbase.HMaster: telling meta
scanner to stop
2008-01-18 17:52:57,568 DEBUG org.apache.hadoop.hbase.HMaster: meta and root
scanners notified
2008-01-18 17:52:57,568 INFO org.apache.hadoop.hbase.HMaster: Stopping
infoServer
2008-01-18 17:52:57,568 INFO org.mortbay.util.ThreadedServer: Stopping Acceptor
ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=60010]
2008-01-18 17:52:57,570 INFO org.mortbay.http.SocketListener: Stopped
SocketListener on 0.0.0.0:60010
{code}
> master stopping before regions finish stopping
> ----------------------------------------------
>
> Key: HADOOP-2662
> URL: https://issues.apache.org/jira/browse/HADOOP-2662
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/hbase
> Reporter: Billy Pearson
> Priority: Minor
> Fix For: 0.16.1
>
>
> I get this on my region servers logs sometimes when I shutdown the cluster it
> repeats several times sometimes trying to find the master
> I thank we need to look at the master and make sure we do not stop the master
> on exit before all region servers report down.
> I am not sure if there could be data loss or not but we should not leave
> region servers looking for the master unless it has failed on it own.
> {code}
> 2008-01-18 17:40:42,009 WARN org.apache.hadoop.hbase.HRegionServer: Failed to
> send exiting message to master:
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
> at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:519)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:159)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:575)
> at org.apache.hadoop.ipc.Client.call(Client.java:498)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java:210)
> at $Proxy0.regionServerReport(Unknown Source)
> at org.apache.hadoop.hbase.HRegionServer.run(HRegionServer.java:898)
> at java.lang.Thread.run(Thread.java:595)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
