[jira] [Created] (HBASE-24398) [hbase-operator-tools] RegionsMerger ConcurrentModificationException
ChenKai created HBASE-24398: --- Summary: [hbase-operator-tools] RegionsMerger ConcurrentModificationException Key: HBASE-24398 URL: https://issues.apache.org/jira/browse/HBASE-24398 Project: HBase Issue Type: Bug Components: hbase-operator-tools Reporter: ChenKai -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-11768) Register region server in zookeeper by ip address
[ https://issues.apache.org/jira/browse/HBASE-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805195#comment-16805195 ] ChenKai commented on HBASE-11768: - [~stack] Now we use k8s's $POD_NAME + SERVICE_NAME to config *hbase.master(regionserver).hostname*, and it seems to work well. I will run some spark jobs to do more production tests, and give feedback when I encounter problems. I think this feature of registering (pod) ip to zk is very good. I will try later and give you more error messages, thanks. > Register region server in zookeeper by ip address > - > > Key: HBASE-11768 > URL: https://issues.apache.org/jira/browse/HBASE-11768 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Cheney Sun >Priority: Major > Labels: patch > Attachments: HBASE-11768.master.001.patch, HBASE_11768.patch > > > HBase cluster isn't always setup along with a DNS server. But regionservers > now register their hostnames in zookeeper, which bring some inconvenience > when regionserver isn't in one DNS server. In such situation, clients have to > maintain the ip/hostname mapping in their /etc/hosts files in order to > resolve the hostname returned from zookeeper to the right address. > However, this causes a lot of pain for clients to maintain the mapping, > especially when adding new machines to the cluster, or some machines' address > changed due to some reason. All clients need to update their host mapping > files. > The issue is to address this problem above, and try to add an option to let > each regionserver record themself by ip address, instead of hostname only. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-11768) Register region server in zookeeper by ip address
[
https://issues.apache.org/jira/browse/HBASE-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805107#comment-16805107
]
ChenKai commented on HBASE-11768:
-
[~stack]
We run HBase cluster on k8s, and it seems to get the wrong ip(hostname) back
with this patch when invoking *regionServerStartup* and
*handleReportForDutyResponse* method.
The error is {color:red}"Master passed us a different hostname to use;
x"{color}.
I am not sure if there's something wrong with k8s DNS, so we returned and use
*hbase.master(regionserver).hostname*.
> Register region server in zookeeper by ip address
> -
>
> Key: HBASE-11768
> URL: https://issues.apache.org/jira/browse/HBASE-11768
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 2.0.0
>Reporter: Cheney Sun
>Priority: Major
> Labels: patch
> Attachments: HBASE-11768.master.001.patch, HBASE_11768.patch
>
>
> HBase cluster isn't always setup along with a DNS server. But regionservers
> now register their hostnames in zookeeper, which bring some inconvenience
> when regionserver isn't in one DNS server. In such situation, clients have to
> maintain the ip/hostname mapping in their /etc/hosts files in order to
> resolve the hostname returned from zookeeper to the right address.
> However, this causes a lot of pain for clients to maintain the mapping,
> especially when adding new machines to the cluster, or some machines' address
> changed due to some reason. All clients need to update their host mapping
> files.
> The issue is to address this problem above, and try to add an option to let
> each regionserver record themself by ip address, instead of hostname only.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-11768) Register region server in zookeeper by ip address
[ https://issues.apache.org/jira/browse/HBASE-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804554#comment-16804554 ] ChenKai commented on HBASE-11768: - +1 > Register region server in zookeeper by ip address > - > > Key: HBASE-11768 > URL: https://issues.apache.org/jira/browse/HBASE-11768 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Cheney Sun >Priority: Major > Labels: patch > Attachments: HBASE-11768.master.001.patch, HBASE_11768.patch > > > HBase cluster isn't always setup along with a DNS server. But regionservers > now register their hostnames in zookeeper, which bring some inconvenience > when regionserver isn't in one DNS server. In such situation, clients have to > maintain the ip/hostname mapping in their /etc/hosts files in order to > resolve the hostname returned from zookeeper to the right address. > However, this causes a lot of pain for clients to maintain the mapping, > especially when adding new machines to the cluster, or some machines' address > changed due to some reason. All clients need to update their host mapping > files. > The issue is to address this problem above, and try to add an option to let > each regionserver record themself by ip address, instead of hostname only. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724122#comment-16724122
]
ChenKai commented on HBASE-21594:
-
I tried both 0.98 and branch-1.2 just now, unlike you error. Here is my
screenshot:
!image-2018-12-18-22-40-20-930.png|width=280,height=100!
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Assignee: Zheng Hu
>Priority: Major
> Attachments: 2e4db6a7f8d14f2bb846781b07750dc1_SeqId_84_.bak,
> image-2018-12-13-20-11-00-818.png, image-2018-12-18-22-40-20-930.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChenKai updated HBASE-21594:
Attachment: image-2018-12-18-22-40-20-930.png
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Assignee: Zheng Hu
>Priority: Major
> Attachments: 2e4db6a7f8d14f2bb846781b07750dc1_SeqId_84_.bak,
> image-2018-12-13-20-11-00-818.png, image-2018-12-18-22-40-20-930.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724028#comment-16724028
]
ChenKai commented on HBASE-21594:
-
Strangely.What's you hbase version? My version is 0.98.x or 1.2.x.
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Assignee: Zheng Hu
>Priority: Major
> Attachments: 2e4db6a7f8d14f2bb846781b07750dc1_SeqId_84_.bak,
> image-2018-12-13-20-11-00-818.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724006#comment-16724006
]
ChenKai commented on HBASE-21594:
-
What puzzles me is the phenomenon is a very, very accidental in our production
environment. And i test my bulk load code locally before, it's very ok.
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Assignee: Zheng Hu
>Priority: Major
> Attachments: 2e4db6a7f8d14f2bb846781b07750dc1_SeqId_84_.bak,
> image-2018-12-13-20-11-00-818.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723986#comment-16723986
]
ChenKai commented on HBASE-21594:
-
Is there any suggestion for me to make a patch for our environment? Thanks.
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Assignee: Zheng Hu
>Priority: Major
> Attachments: 2e4db6a7f8d14f2bb846781b07750dc1_SeqId_84_.bak,
> image-2018-12-13-20-11-00-818.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720927#comment-16720927
]
ChenKai commented on HBASE-21594:
-
You can read the hfile from the
attachments(2e4db6a7f8d14f2bb846781b07750dc1_SeqId_84_.bak). And the test code:
{code:java}
public static void main(String[] args) throws IOException,
ClassNotFoundException {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
HFile.Reader reader = HFile.createReader(fs,
new
Path("file:///xxx/xxx/2e4db6a7f8d14f2bb846781b07750dc1_SeqId_84_.bak"), new
CacheConfig(conf), conf);
// 1. Trailer
FixedFileTrailer trailer = reader.getTrailer();
System.out.println(trailer.getMajorVersion());
System.out.println(trailer.getFirstDataBlockOffset());
System.out.println(trailer.getLastDataBlockOffset());
System.out.println(trailer.getLoadOnOpenDataOffset());
// Load up the index.
reader.loadFileInfo();
// Get a scanner that caches and that does not use pread.
HFileScanner scanner = reader.getScanner(false, false);
// Align scanner at start of the file.
scanner.seekTo();
// read data
while (scanner.next()) {
Cell cell = scanner.getKeyValue();
String row = Bytes.toString(CellUtil.cloneRow(cell));
String cf = Bytes.toString(CellUtil.cloneFamily(cell));
String col = Bytes.toString(CellUtil.cloneQualifier(cell));
// String value = Bytes.toString(CellUtil.cloneValue(cell));
// System.out.println(row + " => " + cf + ":" + col + " = " + value);
}{code}
Thanks a lot.
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Priority: Major
> Attachments: 2e4db6a7f8d14f2bb846781b07750dc1_SeqId_84_.bak,
> image-2018-12-13-20-11-00-818.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChenKai updated HBASE-21594:
Attachment: 2e4db6a7f8d14f2bb846781b07750dc1_SeqId_84_.bak
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Priority: Major
> Attachments: 2e4db6a7f8d14f2bb846781b07750dc1_SeqId_84_.bak,
> image-2018-12-13-20-11-00-818.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720906#comment-16720906
]
ChenKai commented on HBASE-21594:
-
[~openinx] I just use version 1.2 to test reading some of these hfiles, our
production is using hbase-client (0.98.13-hadoop2) to bulk load into
hbase-server (0.98.13-hadoop2).
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Priority: Major
> Attachments: image-2018-12-13-20-11-00-818.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720090#comment-16720090
]
ChenKai edited comment on HBASE-21594 at 12/14/18 2:14 AM:
---
!image-2018-12-13-20-11-00-818.png|width=280,height=100!
here shoud be this? Because i see next block is ROOT_INDEX (no meta blocks).
{code:java}
if (dataBlockOffset < 0 || dataBlockOffset > trailerOffset) {
}{code}
was (Author: 514793...@qq.com):
!image-2018-12-13-20-11-00-818.png|width=280,height=100!
here shoud be this? Because i see next block is ROOT_INDEX.
{code:java}
if (dataBlockOffset < 0 || dataBlockOffset > trailerOffset) {
}{code}
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Priority: Major
> Attachments: image-2018-12-13-20-11-00-818.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720090#comment-16720090
]
ChenKai edited comment on HBASE-21594 at 12/13/18 12:17 PM:
!image-2018-12-13-20-11-00-818.png|width=280,height=100!
here shoud be this? Because i see next block is ROOT_INDEX.
{code:java}
if (dataBlockOffset < 0 || dataBlockOffset > trailerOffset) {
}{code}
was (Author: 514793...@qq.com):
!image-2018-12-13-20-11-00-818.png|height=100,width=280!
here shoud be this?
{code:java}
if (dataBlockOffset < 0 || dataBlockOffset > trailerOffset) {
}{code}
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Priority: Major
> Attachments: image-2018-12-13-20-11-00-818.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720090#comment-16720090
]
ChenKai edited comment on HBASE-21594 at 12/13/18 12:15 PM:
!image-2018-12-13-20-11-00-818.png|height=100,width=280!
here shoud be this?
{code:java}
if (dataBlockOffset < 0 || dataBlockOffset > trailerOffset) {
}{code}
was (Author: 514793...@qq.com):
!image-2018-12-13-20-11-00-818.png!
here shoud be this?
{code:java}
if (dataBlockOffset < 0 || dataBlockOffset > trailerOffset) {
}{code}
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Priority: Major
> Attachments: image-2018-12-13-20-11-00-818.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720090#comment-16720090
]
ChenKai commented on HBASE-21594:
-
!image-2018-12-13-20-11-00-818.png!
here shoud be this?
{code:java}
if (dataBlockOffset < 0 || dataBlockOffset > trailerOffset) {
}{code}
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Priority: Major
> Attachments: image-2018-12-13-20-11-00-818.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChenKai updated HBASE-21594:
Attachment: image-2018-12-13-20-11-00-818.png
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Priority: Major
> Attachments: image-2018-12-13-20-11-00-818.png
>
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719937#comment-16719937
]
ChenKai edited comment on HBASE-21594 at 12/13/18 9:04 AM:
---
[~openinx] HBase version is 0.98.13-hadoop2, Phoenix version is
4.7.0-HBase-0.98. I write a unit test with hbase(branch-1.2), same problems,
i'm not sure if hfile is incorrect? Thanks.
was (Author: 514793...@qq.com):
[~openinx] HBase version is 0.98.13-hadoop2, Phoenix version is
4.7.0-HBase-0.98. I write a unit test with hbase(branch-1.2), same problems.
Thanks.
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Priority: Major
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21594) Requested block is out of range when reading hfile
[
https://issues.apache.org/jira/browse/HBASE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719937#comment-16719937
]
ChenKai commented on HBASE-21594:
-
[~openinx] HBase version is 0.98.13-hadoop2, Phoenix version is
4.7.0-HBase-0.98. I write a unit test with hbase(branch-1.2), same problems.
Thanks.
> Requested block is out of range when reading hfile
> --
>
> Key: HBASE-21594
> URL: https://issues.apache.org/jira/browse/HBASE-21594
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.98.10
>Reporter: ChenKai
>Priority: Major
>
> My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
> them(or hbase do compact), i encounter the following exceptions.
>
> {code:java}
> Exception in thread "main" java.io.IOException: Requested block is out of
> range: 77329641, lastDataBlockOffset: 77329641,
> trailer.getLoadOnOpenDataOffset: 77329641
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
> at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
> {code}
> Looks like `lastDataBlockOffset` is equals to
> `trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21594) Requested block is out of range when reading hfile
ChenKai created HBASE-21594:
---
Summary: Requested block is out of range when reading hfile
Key: HBASE-21594
URL: https://issues.apache.org/jira/browse/HBASE-21594
Project: HBase
Issue Type: Bug
Components: HFile
Affects Versions: 0.98.10
Reporter: ChenKai
My HFiles are generated by Spark HBaseBulkLoad. And then when i read a few of
them(or hbase do compact), i encounter the following exceptions.
{code:java}
Exception in thread "main" java.io.IOException: Requested block is out of
range: 77329641, lastDataBlockOffset: 77329641,
trailer.getLoadOnOpenDataOffset: 77329641
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:396)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:734)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.isNextBlock(HFileReaderV2.java:859)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.positionForNextBlock(HFileReaderV2.java:854)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2._next(HFileReaderV2.java:871)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:891)
at io.patamon.hbase.test.read.TestHFileRead.main(TestHFileRead.java:49)
{code}
Looks like `lastDataBlockOffset` is equals to
`trailer.getLoadOnOpenDataOffset`. Could anyone help me? Thanks very much.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
