[
https://issues.apache.org/jira/browse/HBASE-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277236#comment-13277236
]
[email protected] commented on HBASE-5986:
------------------------------------------------------
bq. On 2012-05-16 21:32:56, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/client/HTable.java, line 473
bq. > <https://reviews.apache.org/r/5133/diff/1/?file=109116#file109116line473>
bq. >
bq. > Why you remove this? We don't return these any more? Offline I
think is 'dead', unused now. Split not.
If we discover that a region has been split in META, than, it is past
point-of-no-return and the region cannot be seen un-split anymore, even though
concurrent rs failures. for getStartEndKeys() we are returning whatever the
getRegionLocation() provides. getRegionLocations() ignores offline regions, but
returns daughter regions for split-parents (which are offline as well).
bq. On 2012-05-16 21:32:56, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java, line 351
bq. > <https://reviews.apache.org/r/5133/diff/1/?file=109117#file109117line351>
bq. >
bq. > Should this be public? Should it remain internal to the HTable
hidden?
MetaScanner is confusing in its visibility. It's javadoc states "Although
public visibility, this is not a public-facing API and may evolve in minor
releases.", but it is annotated with @InterfaceAudience.Public
@InterfaceStability.Evolving. I think for BlockingMetaScannerVisitor, we should
set the visibility the same as MetaScannerVisitor. I think, MetaScannerVisitor
itself is of little use without the BlockingMetaScannerVisitor functionality
due to this issue.
Shall we change MetaScanner, MSV and BMSV to be @InterfaceAudience.Private,
wdyt?
bq. On 2012-05-16 21:32:56, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java, line 310
bq. > <https://reviews.apache.org/r/5133/diff/1/?file=109115#file109115line310>
bq. >
bq. > Bit of javadoc to say this is best-effort.
bq. >
bq. > Also, does this belong in MetaReader (Won't hold you to it... these
two classes, a MetaReader vs MetaEditor are kinda silly... this whole catalog
package needs killing).
agreed
bq. On 2012-05-16 21:32:56, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java, line 395
bq. > <https://reviews.apache.org/r/5133/diff/1/?file=109117#file109117line395>
bq. >
bq. > Yeah, so, what happens if daughter has split by the time I get here?
The scanner provides the regions results in sorted order, and since the
daughters are sorted after the parent, we always process parent first. When we
process the daughter regions manually (processRow(resultA)), we also add them
to the daughterRegions set. If the scanner also sees them, they are just
skipped (line 373)
bq. On 2012-05-16 21:32:56, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java, line 434
bq. > <https://reviews.apache.org/r/5133/diff/1/?file=109117#file109117line434>
bq. >
bq. > So if interrupted or we don't find it by the time the blocking time
has passed, we just return null? What you reckon? We should at least
complain?
yeah, I think we can throw RegionOfflineException upon timeout and interrupt.
The operation can be retried by the client upon timeout.
bq. On 2012-05-16 21:32:56, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java, line 465
bq. > <https://reviews.apache.org/r/5133/diff/1/?file=109117#file109117line465>
bq. >
bq. > Does the scan of meta start at first table region?
MetaScanner.metaScan() ensures that the scan starts at the first table region.
bq. On 2012-05-16 21:32:56, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java, line 390
bq. > <https://reviews.apache.org/r/5133/diff/1/?file=109117#file109117line390>
bq. >
bq. > Yeah, what Ted says... Can you close when done? See
MetaEditor/MetaReader. They do this a bunch. Closing means for sure the zk
and connection resources will be cleaned up afterward. Its reference counting
so keepign around an HTable could mess it up.
We are already closing the HTable's. But let me see how we can best reuse
Htable instances.
- enis
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5133/#review7939
-----------------------------------------------------------
On 2012-05-16 01:53:09, enis wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/5133/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-16 01:53:09)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. We found this issue when running large scale ingestion tests for
HBASE-5754. The problem is that the .META. table updates are not atomic while
splitting a region. In SplitTransaction, there is a time lap between the
marking the parent offline, and adding of daughters to the META table. This can
result in clients using MetaScanner, of HTable.getStartEndKeys (used by the
TableInputFormat) missing regions which are made just offline, but the
daughters are not added yet.
bq.
bq. This patch is the approach 2 mentioned in the issue comments, mainly
during META scan, if we detect that the region is split, we block until the
information for the child regions are available in META and manually feed those
rows to the MetaScanner. Although approach 3 (using local region transactions)
seems cleaner, they are not available under branch 0.92, which I think should
also incorporate this fix. I'll provide ports once we are clear for trunk.
bq.
bq. Also this patch does not fix MetaReader (see
https://issues.apache.org/jira/browse/HBASE-3475).
bq.
bq.
bq. This addresses bug HBASE-5986.
bq. https://issues.apache.org/jira/browse/HBASE-5986
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java 8873512
bq. src/main/java/org/apache/hadoop/hbase/client/HTable.java b8290e4
bq. src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java f404999
bq.
src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
a8091e6
bq.
bq. Diff: https://reviews.apache.org/r/5133/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. added extensive tests under TestEndToEndSplitTranscation, and ran existing
unit tests.
bq.
bq.
bq. Thanks,
bq.
bq. enis
bq.
bq.
> Clients can see holes in the META table when regions are being split
> --------------------------------------------------------------------
>
> Key: HBASE-5986
> URL: https://issues.apache.org/jira/browse/HBASE-5986
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.1, 0.96.0, 0.94.1
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Attachments: HBASE-5986-test_v1.patch
>
>
> We found this issue when running large scale ingestion tests for HBASE-5754.
> The problem is that the .META. table updates are not atomic while splitting a
> region. In SplitTransaction, there is a time lap between the marking the
> parent offline, and adding of daughters to the META table. This can result in
> clients using MetaScanner, of HTable.getStartEndKeys (used by the
> TableInputFormat) missing regions which are made just offline, but the
> daughters are not added yet.
> This is also related to HBASE-4335.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
