[jira] [Updated] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

2019-03-04 Thread Guanghao Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-20769:
---
Fix Version/s: (was: 2.2.0)

> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0, 1.4.0, 2.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.3.3, 1.4.6, 2.0.2
>
> Attachments: HBASE-20769.branch-1.001.patch, 
> HBASE-20769.branch-1.3.001.patch, HBASE-20769.master.001.patch, 
> HBASE-20769.master.002.patch, HBASE-20769.master.003.patch, 
> HBASE-20769.master.004.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller 
> than user specified scan's start row or stop row larger than user specified 
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, 
> true);
> for (int i = 0; i < sp.length - 1; i++) {
>   if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), 
> scan.getStopRow(), sp[i],
>   sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir, 
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan, 
> restoreDir));
>   }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow() 
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range 
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
>  Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
>  boundedScan.setStopRow(
>  Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i 
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

2019-03-04 Thread Guanghao Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-20769:
---
Fix Version/s: 2.2.0

> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0, 1.4.0, 2.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.3.3, 1.4.6, 2.2.0, 2.0.2
>
> Attachments: HBASE-20769.branch-1.001.patch, 
> HBASE-20769.branch-1.3.001.patch, HBASE-20769.master.001.patch, 
> HBASE-20769.master.002.patch, HBASE-20769.master.003.patch, 
> HBASE-20769.master.004.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller 
> than user specified scan's start row or stop row larger than user specified 
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, 
> true);
> for (int i = 0; i < sp.length - 1; i++) {
>   if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), 
> scan.getStopRow(), sp[i],
>   sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir, 
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan, 
> restoreDir));
>   }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow() 
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range 
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
>  Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
>  boundedScan.setStopRow(
>  Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i 
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

2019-02-01 Thread Andrew Purtell (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-20769:
---
Fix Version/s: (was: 1.5.0)

> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0, 1.4.0, 2.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.3.3, 1.4.6, 2.0.2
>
> Attachments: HBASE-20769.branch-1.001.patch, 
> HBASE-20769.branch-1.3.001.patch, HBASE-20769.master.001.patch, 
> HBASE-20769.master.002.patch, HBASE-20769.master.003.patch, 
> HBASE-20769.master.004.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller 
> than user specified scan's start row or stop row larger than user specified 
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, 
> true);
> for (int i = 0; i < sp.length - 1; i++) {
>   if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), 
> scan.getStopRow(), sp[i],
>   sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir, 
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan, 
> restoreDir));
>   }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow() 
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range 
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
>  Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
>  boundedScan.setStopRow(
>  Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i 
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

2018-06-29 Thread Andrew Purtell (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-20769:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: (was: 2.0.0)
   2.0.2
   1.4.6
   1.3.3
   1.5.0
   2.1.0
   3.0.0
   Status: Resolved  (was: Patch Available)

> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0, 1.4.0, 2.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.4.6, 2.0.2
>
> Attachments: HBASE-20769.branch-1.001.patch, 
> HBASE-20769.branch-1.3.001.patch, HBASE-20769.master.001.patch, 
> HBASE-20769.master.002.patch, HBASE-20769.master.003.patch, 
> HBASE-20769.master.004.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller 
> than user specified scan's start row or stop row larger than user specified 
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, 
> true);
> for (int i = 0; i < sp.length - 1; i++) {
>   if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), 
> scan.getStopRow(), sp[i],
>   sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir, 
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan, 
> restoreDir));
>   }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow() 
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range 
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
>  Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
>  boundedScan.setStopRow(
>  Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i 
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

2018-06-29 Thread Jingyun Tian (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingyun Tian updated HBASE-20769:
-
Attachment: HBASE-20769.branch-1.3.001.patch

> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0, 1.4.0, 2.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20769.branch-1.001.patch, 
> HBASE-20769.branch-1.3.001.patch, HBASE-20769.master.001.patch, 
> HBASE-20769.master.002.patch, HBASE-20769.master.003.patch, 
> HBASE-20769.master.004.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller 
> than user specified scan's start row or stop row larger than user specified 
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, 
> true);
> for (int i = 0; i < sp.length - 1; i++) {
>   if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), 
> scan.getStopRow(), sp[i],
>   sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir, 
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan, 
> restoreDir));
>   }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow() 
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range 
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
>  Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
>  boundedScan.setStopRow(
>  Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i 
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

2018-06-28 Thread Jingyun Tian (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingyun Tian updated HBASE-20769:
-
Attachment: HBASE-20769.branch-1.001.patch

> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0, 1.4.0, 2.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20769.branch-1.001.patch, 
> HBASE-20769.master.001.patch, HBASE-20769.master.002.patch, 
> HBASE-20769.master.003.patch, HBASE-20769.master.004.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller 
> than user specified scan's start row or stop row larger than user specified 
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, 
> true);
> for (int i = 0; i < sp.length - 1; i++) {
>   if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), 
> scan.getStopRow(), sp[i],
>   sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir, 
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan, 
> restoreDir));
>   }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow() 
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range 
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
>  Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
>  boundedScan.setStopRow(
>  Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i 
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

2018-06-25 Thread Jingyun Tian (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingyun Tian updated HBASE-20769:
-
Attachment: HBASE-20769.master.004.patch

> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0, 1.4.0, 2.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20769.master.001.patch, 
> HBASE-20769.master.002.patch, HBASE-20769.master.003.patch, 
> HBASE-20769.master.004.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller 
> than user specified scan's start row or stop row larger than user specified 
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, 
> true);
> for (int i = 0; i < sp.length - 1; i++) {
>   if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), 
> scan.getStopRow(), sp[i],
>   sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir, 
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan, 
> restoreDir));
>   }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow() 
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range 
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
>  Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
>  boundedScan.setStopRow(
>  Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i 
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

2018-06-24 Thread Jingyun Tian (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingyun Tian updated HBASE-20769:
-
Attachment: HBASE-20769.master.003.patch

> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0, 1.4.0, 2.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20769.master.001.patch, 
> HBASE-20769.master.002.patch, HBASE-20769.master.003.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller 
> than user specified scan's start row or stop row larger than user specified 
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, 
> true);
> for (int i = 0; i < sp.length - 1; i++) {
>   if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), 
> scan.getStopRow(), sp[i],
>   sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir, 
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan, 
> restoreDir));
>   }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow() 
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range 
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
>  Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
>  boundedScan.setStopRow(
>  Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i 
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

2018-06-24 Thread Jingyun Tian (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingyun Tian updated HBASE-20769:
-
Status: Patch Available  (was: Open)

> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0, 1.3.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20769.master.001.patch, 
> HBASE-20769.master.002.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller 
> than user specified scan's start row or stop row larger than user specified 
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, 
> true);
> for (int i = 0; i < sp.length - 1; i++) {
>   if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), 
> scan.getStopRow(), sp[i],
>   sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir, 
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan, 
> restoreDir));
>   }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow() 
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range 
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
>  Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
>  boundedScan.setStopRow(
>  Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i 
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

2018-06-24 Thread Jingyun Tian (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingyun Tian updated HBASE-20769:
-
Attachment: HBASE-20769.master.002.patch

> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0, 1.4.0, 2.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20769.master.001.patch, 
> HBASE-20769.master.002.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller 
> than user specified scan's start row or stop row larger than user specified 
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, 
> true);
> for (int i = 0; i < sp.length - 1; i++) {
>   if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), 
> scan.getStopRow(), sp[i],
>   sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir, 
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan, 
> restoreDir));
>   }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow() 
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range 
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
>  Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
>  boundedScan.setStopRow(
>  Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i 
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

2018-06-21 Thread Jingyun Tian (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingyun Tian updated HBASE-20769:
-
Attachment: HBASE-20769.master.001.patch

> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.0, 1.4.0, 2.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20769.master.001.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller 
> than user specified scan's start row or stop row larger than user specified 
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, 
> true);
> for (int i = 0; i < sp.length - 1; i++) {
>   if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), 
> scan.getStopRow(), sp[i],
>   sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir, 
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan, 
> restoreDir));
>   }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow() 
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range 
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
>  Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
>  boundedScan.setStopRow(
>  Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i 
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)