[
https://issues.apache.org/jira/browse/HBASE-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-1878:
-------------------------
Attachment: basescanner.patch
Here's a patch.
I've been testing it up on cluster doing restarts of individual regionserver.
The below happened at least once:
{code}
2009-09-30 19:54:40,784 [IPC Server handler 1 on 60001] INFO
org.apache.hadoop.hbase.master.ServerManager: Processing MSG_REPORT_OPEN:
TestTable,0195089359,12
54340471703 from XX.XX.XX.XX,60020,1254339974414; 1 of 1
2009-09-30 19:54:40,784 [HMaster] DEBUG org.apache.hadoop.hbase.master.HMaster:
Processing todo: PendingOpenOperation from XX.XX.XX.XX,60020,125
4339974414
2009-09-30 19:54:40,784 [HMaster] INFO
org.apache.hadoop.hbase.master.RegionServerOperation:
TestTable,0195089359,1254340471703 open on XX.XX.XX.XX:60020
2009-09-30 19:54:40,840 [HMaster] INFO
org.apache.hadoop.hbase.master.RegionServerOperation: Updated row
TestTable,0195089359,1254340471703 in region .META.,,
1 with startcode=1254339974414, server= XX.XX.XX.XX:60020
2009-09-30 19:54:41,389 [main-SendThread] DEBUG
org.apache.zookeeper.ClientCnxn: Got ping response for
sessionid:0x1240c6fb09c0000 after 1ms
2009-09-30 19:54:42,043 [RegionManager.metaScanner] DEBUG
org.apache.hadoop.hbase.master.BaseScanner: GET got different address than
SCAN: sa= XX.XX.XX.XX:60
020, serverAddress= XX.XX.XX.XX:60020
2009-09-30 19:54:42,043 [RegionManager.metaScanner] DEBUG
org.apache.hadoop.hbase.master.BaseScanner: GET got different startcode than
SCAN: sc=1254339974414,
serverAddress=1254339099457
{code}
Above is saying that the GET got different result than scan... You can see the
row being updated seconds beforehand.
I'm going to commit this change. It means effectively that each scan now is
heavier in that its a scan and then a get on each row.
> BaseScanner results can't be trusted at all (Related to hbase-1784)
> -------------------------------------------------------------------
>
> Key: HBASE-1878
> URL: https://issues.apache.org/jira/browse/HBASE-1878
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Attachments: basescanner.patch
>
>
> Gario this morning got a few double-assignments doing rolling restart.
> {code}
> 1.
> Sep 29 15:18:24 hnn0.int.meetup.com [hbase] 2009-09-29 15:18:24,010
> [IPC Server handler 177 on 60000] INFO
> org.apache.hadoop.hbase.master.ServerManager - Processing MSG_REPORT_CLOSE:
> MemberFeedIndex,8828808,1252839447949 from
> hdn8.int.meetup.com,60020,1253572119095
> 2.
> Sep 29 15:18:39 hnn0.int.meetup.com [hbase] 2009-09-29 15:18:39,369
> [IPC Server handler 4 on 60000] INFO
> org.apache.hadoop.hbase.master.RegionManager - Assigning region
> MemberFeedIndex,8828808,1252839447949 to
> hdn7.int.meetup.com,60020,1254251818253
> 3.
> Sep 29 15:18:42 hnn0.int.meetup.com [hbase] 2009-09-29 15:18:42,399
> [IPC Server handler 32 on 60000] INFO
> org.apache.hadoop.hbase.master.ServerManager - Processing
> MSG_REPORT_PROCESS_OPEN: MemberFeedIndex,8828808,1252839447949 from
> hdn7.int.meetup.com,60020,1254251818253; 3 of 19
> 4.
> Sep 29 15:18:42 hnn0.int.meetup.com [hbase] 2009-09-29 15:18:42,400
> [IPC Server handler 32 on 60000] INFO
> org.apache.hadoop.hbase.master.ServerManager - Processing MSG_REPORT_OPEN:
> MemberFeedIndex,8828808,1252839447949 from
> hdn7.int.meetup.com,60020,1254251818253; 13 of 19
> 5.
> Sep 29 15:19:01 hnn0.int.meetup.com [hbase] 2009-09-29 15:19:01,195
> [HMaster] INFO org.apache.hadoop.hbase.master.RegionServerOperation -
> MemberFeedIndex,8828808,1252839447949 open on 192.168.60.213:60020
> 6.
> Sep 29 15:19:01 hnn0.int.meetup.com [hbase] 2009-09-29 15:19:01,196
> [HMaster] INFO org.apache.hadoop.hbase.master.RegionServerOperation -
> Updated row MemberFeedIndex,8828808,1252839447949 in region .META.,,1 with
> startcode=1254251818253, server=192.168.60.213:60020
> 7.
> Sep 29 15:19:01 hnn0.int.meetup.com [hbase] 2009-09-29 15:19:01,230
> [RegionManager.metaScanner] DEBUG org.apache.hadoop.hbase.master.BaseScanner
> - Current assignment of MemberFeedIndex,8828808,1252839447949 is not valid;
> serverAddress=192.168.60.214:60020, startCode=1253572119095 unknown.
> 8.
> Sep 29 15:19:03 hnn0.int.meetup.com [hbase] 2009-09-29 15:19:03,059
> [IPC Server handler 88 on 60000] INFO
> org.apache.hadoop.hbase.master.RegionManager - Assigning region
> MemberFeedIndex,8828808,1252839447949 to
> hdn8.int.meetup.com,60020,1254251940990
> 9.
> Sep 29 15:19:06 hnn0.int.meetup.com [hbase] 2009-09-29 15:19:06,089
> [IPC Server handler 97 on 60000] INFO
> org.apache.hadoop.hbase.master.ServerManager - Processing MSG_REPORT_OPEN:
> MemberFeedIndex,8828808,1252839447949 from
> hdn8.int.meetup.com,60020,1254251940990; 10 of 19
> 10.
> Sep 29 15:19:06 hnn0.int.meetup.com [hbase] 2009-09-29 15:19:06,091
> [HMaster] INFO org.apache.hadoop.hbase.master.RegionServerOperation -
> MemberFeedIndex,8828808,1252839447949 open on 192.168.60.214:60020
> 11.
> Sep 29 15:19:06 hnn0.int.meetup.com [hbase] 2009-09-29 15:19:06,092
> [HMaster] INFO org.apache.hadoop.hbase.master.RegionServerOperation -
> Updated row MemberFeedIndex,8828808,1252839447949 in region .META.,,1 with
> startcode=1254251940990, server=192.168.60.214:60020
> {code}
> There's 35ms between update of row and ruling of region being unassigned.
> HBASE-1784 added a GET on .META. row before ruling a region unassigned IFF
> the server cell was empty.
> Need to do this GET every time. BaseScanner can't be trusted. Its info is
> stale. Need to hurry up with 0.21 rewrite.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
