[
https://issues.apache.org/jira/browse/HBASE-29696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HBASE-29696:
-----------------------------------
Labels: pull-request-available (was: )
> TableInputFormatBase with NUM_MAPPERS_PER_REGION produces incorrect last
> InputSplit
> -----------------------------------------------------------------------------------
>
> Key: HBASE-29696
> URL: https://issues.apache.org/jira/browse/HBASE-29696
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 3.0.0, 2.6.3
> Reporter: Jasper van Maanen
> Assignee: mazhengxuan
> Priority: Major
> Labels: pull-request-available
>
> *How to reproduce*
> Create a table with a single region and add the row key
> {code:java}
> new byte[] {-1, -1}{code}
> Perform a MapReduce job with at least this setting:
> {code:java}
> "hbase.mapreduce.tableinput.mappers.per.region" = 2 {code}
> The row key is missing in the scan result.
>
> *Analysis*
> In TableInputF{color:#172b4d}ormatBase#createNInputSplitsUniform there is
> this code:
> {color}
> {code:java}
> // For special case: startRow or endRow is empty
> if (startRow.length == 0 && endRow.length == 0) {
> startRow = new byte[1];
> endRow = new byte[1];
> startRow[0] = 0;
> endRow[0] = -1;
> }
> if (startRow.length == 0 && endRow.length != 0) {
> startRow = new byte[1];
> startRow[0] = 0;
> }
> if (startRow.length != 0 && endRow.length == 0) {
> endRow = new byte[startRow.length];
> for (int k = 0; k < startRow.length; k++) {
> endRow[k] = -1;
> }
> } {code}
> Unfortunately, in the first 'if', the endRow is set to
> {code:java}
> new byte[] {-1} {code}
> But what if there is a row key with
> {code:java}
> new byte[] {-1, -1}{code}
> This row key is after the endRow and will be ignored by the scan.
> This is also an issue in the third 'if'. Since a row key can be of
> potentially of an infinite length, setting the end row in the third 'if' also
> prevents longer row keys (compared to the start row) to be ignored.
> Therefore, in both situations, the end row should stay empty. I can imagine
> the endRow is set for the next step:
> {code:java}
> // Split Region into n chunks evenly
> byte[][] splitKeys = Bytes.split(startRow, endRow, true, n - 1); {code}
> In that case, some compensation should be done later to return the correct
> input splits.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
