[jira] [Commented] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[
https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575780#comment-16575780
]
Kuan-Po Tseng commented on HBASE-21012:
---
{quote}+1 on patch. +1 for branch-2.0. Copy the doc changes and paste as RN for
this issue. Thanks.
{quote}
Done. Thanks for your comment.
> Revert the change of serializing TimeRangeTracker
> -
>
> Key: HBASE-21012
> URL: https://issues.apache.org/jira/browse/HBASE-21012
> Project: HBase
> Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Critical
> Fix For: 3.0.0, 2.0.2, 2.1.1
>
> Attachments: HBASE-21012.master.001.patch,
> HBASE-21012.master.002.patch, HBASE-21012.master.003.patch,
> HBASE-21012.master.003.patch, HBASE-21012.master.004.patch,
> HBASE-21012.master.005.patch
>
>
> HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to
> protobuf. However, the change breaks the backward compatibility of hfile. We
> should revert the change ASAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-21012: -- Release Note: HFiles generated by 2.0.0, 2.0.1, 2.1.0 are not forward compatible to 1.4.6-, 1.3.2.1-, 1.2.6.1-, and other inactive releases. Why HFile lose compatability is hbase in new versions (2.0.0, 2.0.1, 2.1.0) use protobuf to serialize/deserialize TimeRangeTracker (TRT) while old versions use DataInput/DataOutput. To solve this, We have to put HBASE-21012 to 2.x and put HBASE-21013 in 1.x. For more information, please check HBASE-21008. > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch, > HBASE-21012.master.002.patch, HBASE-21012.master.003.patch, > HBASE-21012.master.003.patch, HBASE-21012.master.004.patch, > HBASE-21012.master.005.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575728#comment-16575728 ] Kuan-Po Tseng commented on HBASE-18201: --- [~reidchan] OK, I will find a day to backport this to branch-1, thanks everyone. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 2.2.0, 2.1.1 > > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.005.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.006.patch, > HBASE-18201.master.006.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-21012: -- Attachment: HBASE-21012.master.005.patch > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch, > HBASE-21012.master.002.patch, HBASE-21012.master.003.patch, > HBASE-21012.master.003.patch, HBASE-21012.master.004.patch, > HBASE-21012.master.005.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574234#comment-16574234 ] Kuan-Po Tseng commented on HBASE-18201: --- [~reidchan] No problem. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.005.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.006.patch, > HBASE-18201.master.006.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574229#comment-16574229 ] Kuan-Po Tseng commented on HBASE-21012: --- [~chia7712] Sorry, I didn't notice the wrong term either. The next patch will correct that. > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch, > HBASE-21012.master.002.patch, HBASE-21012.master.003.patch, > HBASE-21012.master.003.patch, HBASE-21012.master.004.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[
https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574192#comment-16574192
]
Kuan-Po Tseng commented on HBASE-21012:
---
Josh Elser Thanks for your comments.
{quote}I'd suggest just including an HFile with the old serialization and
including it in src/test/resources. Then, you just write a test to make sure it
can read that old HFile version.
{quote}
HBase in 2.x are capable of reading HFile from old version, the problem is old
HBase version can't read HFile generated by new HBase version. So we have to
revert the way of serializing TRT. (This situation might not be called as
"backward compatability", sorry about that.)
> Revert the change of serializing TimeRangeTracker
> -
>
> Key: HBASE-21012
> URL: https://issues.apache.org/jira/browse/HBASE-21012
> Project: HBase
> Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Critical
> Fix For: 3.0.0, 2.0.2, 2.1.1
>
> Attachments: HBASE-21012.master.001.patch,
> HBASE-21012.master.002.patch, HBASE-21012.master.003.patch,
> HBASE-21012.master.003.patch, HBASE-21012.master.004.patch
>
>
> HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to
> protobuf. However, the change breaks the backward compatibility of hfile. We
> should revert the change ASAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-21012: -- Attachment: HBASE-21012.master.004.patch > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch, > HBASE-21012.master.002.patch, HBASE-21012.master.003.patch, > HBASE-21012.master.003.patch, HBASE-21012.master.004.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573391#comment-16573391 ] Kuan-Po Tseng commented on HBASE-21012: --- [~chia7712] I have already test the patch locally, but I have no idea about how to write the test since it cross two different hbase branches... > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch, > HBASE-21012.master.002.patch, HBASE-21012.master.003.patch, > HBASE-21012.master.003.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-21012: -- Attachment: HBASE-21012.master.003.patch Status: Patch Available (was: Open) resubmit patch. > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch, > HBASE-21012.master.002.patch, HBASE-21012.master.003.patch, > HBASE-21012.master.003.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572872#comment-16572872 ] Kuan-Po Tseng commented on HBASE-21012: --- All done. > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch, > HBASE-21012.master.002.patch, HBASE-21012.master.003.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-21012: -- Attachment: HBASE-21012.master.003.patch > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch, > HBASE-21012.master.002.patch, HBASE-21012.master.003.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-21012: -- Attachment: HBASE-21012.master.002.patch > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch, > HBASE-21012.master.002.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-21012: -- Attachment: (was: image-2018-08-08-13-41-05-900.png) > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-21012: -- Attachment: image-2018-08-08-13-41-05-900.png > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch, > image-2018-08-08-13-41-05-900.png > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571802#comment-16571802 ] Kuan-Po Tseng commented on HBASE-21012: --- No problem, I will add docs in next patch. Thanks. > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng reassigned HBASE-21012: - Assignee: Kuan-Po Tseng > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-21012: -- Attachment: HBASE-21012.master.001.patch > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > Attachments: HBASE-21012.master.001.patch > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21012) Revert the change of serializing TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571546#comment-16571546 ] Kuan-Po Tseng commented on HBASE-21012: --- [~chia7712] May I take over this issue ? > Revert the change of serializing TimeRangeTracker > - > > Key: HBASE-21012 > URL: https://issues.apache.org/jira/browse/HBASE-21012 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0, 2.0.2, 2.1.1 > > > HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to > protobuf. However, the change breaks the backward compatibility of hfile. We > should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-18201: -- Attachment: HBASE-18201.master.006.patch > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.005.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.006.patch, > HBASE-18201.master.006.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-18201: -- Attachment: HBASE-18201.master.006.patch > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.005.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.006.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[
https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570122#comment-16570122
]
Kuan-Po Tseng commented on HBASE-18201:
---
[~reidchan] Thanks for your review.
{quote} Is {{meta.getDataBlockEncoding()}} equals {{this.encoding}} (it looks
so to me). If so, {{this.encoding = encoding}} can be removed IMO.
{quote}
Yes. meta.getDataBlockEncoding == encodingCtx.getDataBlockEncoding() ==
encoding.
But
{code:java}
meta.getDataBlockEncoding().equals(DataBlockEncoding.ROW_INDEX_V1){code}
is needed.
We can change the code to
{code:java}
encoding.equals(DataBlockEncoding.ROW_INDEX_V1)
{code}
It seems more clear.
> add UT and docs for DataBlockEncodingTool
> -
>
> Key: HBASE-18201
> URL: https://issues.apache.org/jira/browse/HBASE-18201
> Project: HBase
> Issue Type: Sub-task
> Components: tooling
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-18201.master.001.patch,
> HBASE-18201.master.002.patch, HBASE-18201.master.002.patch,
> HBASE-18201.master.003.patch, HBASE-18201.master.004.patch,
> HBASE-18201.master.005.patch, HBASE-18201.master.005.patch,
> HBASE-18201.master.005.patch
>
>
> There is no example, documents, or tests for DataBlockEncodingTool. We should
> have it friendly if any use case exists. Otherwise, we should just get rid of
> it because DataBlockEncodingTool presumes that the implementation of cell
> returned from DataBlockEncoder is KeyValue. The presume may obstruct the
> cleanup of KeyValue references in the code base of read/write path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569728#comment-16569728 ] Kuan-Po Tseng commented on HBASE-18201: --- [~reidchan] Could you take a look ? > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.005.patch, > HBASE-18201.master.005.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-18201: -- Attachment: HBASE-18201.master.005.patch > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.005.patch, > HBASE-18201.master.005.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545111#comment-16545111 ] Kuan-Po Tseng commented on HBASE-18201: --- It seems fail test isn't relate to this patch, resummit patch. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.005.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-18201: -- Attachment: HBASE-18201.master.005.patch > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch, > HBASE-18201.master.005.patch, HBASE-18201.master.005.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544881#comment-16544881 ] Kuan-Po Tseng commented on HBASE-18201: --- patch 005 - delete useless code. - fix checkstyle issues. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch, > HBASE-18201.master.005.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-18201: -- Attachment: HBASE-18201.master.005.patch > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch, > HBASE-18201.master.005.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544557#comment-16544557 ] Kuan-Po Tseng commented on HBASE-18201: --- Patch 004 Bugs * When HFile includeTags is true, it does'nt mean all Cells in HFile contains tags, those don't contains tags will have a tagsLen((short) 0) part after the KeyValue structure. During the decoding process, decoder ROW_INDEX_V1 will not put tagsLen part back in cell if it is zero. Tests * Add DataBlockEncodingTool testing for HFile with cells contasin tags or not. Others * Fix some coding style problems. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-18201: -- Attachment: HBASE-18201.master.004.patch > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch, HBASE-18201.master.004.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529364#comment-16529364 ] Kuan-Po Tseng commented on HBASE-18201: --- No problem, I will add comments for every bug fixes, thanks for your patience. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[
https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529342#comment-16529342
]
Kuan-Po Tseng commented on HBASE-18201:
---
Sorry, I gave the wrong explain. The probolm is in
{code:java}
baosBytes = baos.toByteArray();
{code}
Encoder ROW_INDEX_V1 write integer in outputStream while the others write
integer in byte array(baosBytes), we should change code like this
{code:java}
baos.flush();
baosBytes = baos.toByteArray();
this.dataBlockEncoder.endBlockEncoding(encodingCtx, out, baosBytes);
if(this.encodingCtx.getDataBlockEncoding().equals(DataBlockEncoding.ROW_INDEX_V1))
{
baosBytes = baos.toByteArray();
}{code}
Add baos.toByteArray() after endBlockEncoding again to make sure the integer
writes in outputstream with Encoder ROW_INDEX_V1 dump to byte array (baosBytes).
The if branch is neccessary because Encoders excepts ROW_INDEX_V1 write integer
in baosBytes directly, without if branch and do toByteArray() again, baosBytes
won't contains the integer wrotten in endBlockEncoding.
> add UT and docs for DataBlockEncodingTool
> -
>
> Key: HBASE-18201
> URL: https://issues.apache.org/jira/browse/HBASE-18201
> Project: HBase
> Issue Type: Sub-task
> Components: tooling
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-18201.master.001.patch,
> HBASE-18201.master.002.patch, HBASE-18201.master.002.patch,
> HBASE-18201.master.003.patch
>
>
> There is no example, documents, or tests for DataBlockEncodingTool. We should
> have it friendly if any use case exists. Otherwise, we should just get rid of
> it because DataBlockEncodingTool presumes that the implementation of cell
> returned from DataBlockEncoder is KeyValue. The presume may obstruct the
> cleanup of KeyValue references in the code base of read/write path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[
https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527455#comment-16527455
]
Kuan-Po Tseng commented on HBASE-18201:
---
{quote}BufferedDataBlockEncoder should work as well? Those encoders don't write
onDiskDataSize.{quote}
BufferedDataBlockEncoer#endBlockEncoding not writes onDiskDataSize but still
writes integer called unencodedDataSizeWritten
{code:java}
@Override
public void endBlockEncoding(HFileBlockEncodingContext encodingCtx,
DataOutputStream out,
byte[] uncompressedBytesWithHeader) throws IOException {
BufferedDataBlockEncodingState state = (BufferedDataBlockEncodingState)
encodingCtx
.getEncodingState();
// Write the unencodedDataSizeWritten (with header size)
Bytes.putInt(uncompressedBytesWithHeader,
HConstants.HFILEBLOCK_HEADER_SIZE + DataBlockEncoding.ID_SIZE,
state.unencodedDataSizeWritten
);
postEncoding(encodingCtx);
}
{code}
{quote}That's what i meant, but i'm not sure if it should.{quote}
Yeah, DataBlockEncodingTool#checkStatistics iteratively called
KeyValue#getBuffer and write it to unCompressedOutputStream. KeyValue#getBuffer
return the backing array which should contains all elements in KV.
{code:java}
uncompressedOutputStream.write(currentKV.getBuffer(),
currentKV.getOffset(), currentKV.getLength());
{code}
> add UT and docs for DataBlockEncodingTool
> -
>
> Key: HBASE-18201
> URL: https://issues.apache.org/jira/browse/HBASE-18201
> Project: HBase
> Issue Type: Sub-task
> Components: tooling
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-18201.master.001.patch,
> HBASE-18201.master.002.patch, HBASE-18201.master.002.patch,
> HBASE-18201.master.003.patch
>
>
> There is no example, documents, or tests for DataBlockEncodingTool. We should
> have it friendly if any use case exists. Otherwise, we should just get rid of
> it because DataBlockEncodingTool presumes that the implementation of cell
> returned from DataBlockEncoder is KeyValue. The presume may obstruct the
> cleanup of KeyValue references in the code base of read/write path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[
https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527252#comment-16527252
]
Kuan-Po Tseng commented on HBASE-18201:
---
Thanks for reviewing, Reid Chan.
{quote}I think adjust the call order like following should works. No need to
add another if branch, kind of confusing.
{code:java}
this.dataBlockEncoder.endBlockEncoding(encodingCtx, out, baosBytes);
baos.flush();
baosBytes = baos.toByteArray();
{code}
{quote}
The problem is ROW_INDEX_V1 extends different class, its #endBlockEncoding
write (int) onDiskDataSize in OutputStream while other encoders write in byte
array which is under
OutputStream. If we do #endBlockEncoding first and #flush, Encoder
ROW_INDEX_V1 runs well, while byte array with other encoders is
{(int)onDiskDataSize, byte, byte,byte} since they write (int)onDiskDataSize
in byte array first and flush all data, but the right order is {byte, byte,
, (int) onDiskDataSize}. (int)onDiskDataSize should be the last.
Could we add useTag = currentKV.getTagsLength() > 0 in while loop above? Once
it is set true, the rest no needs to check.
{quote}
{code:java}
HStoreFile hsf = new HStoreFile(fs, path, conf, cacheConf, BloomType.NONE,
true);
StoreFileReader reader = hsf.getReader();
boolean useTag = reader.getHFileReader().getFileContext().isIncludesTags();
{code}
Kinds of heavy to create a HStoreFile instance just to use its isIncludesTags
method.
{quote}
Sorry, I didn't explain carefully. HStoreFile instance is already created in
#testCodecs which happened before #checkStatistics , we could check if useTag
is true in #testCodecs instead of creating a new one.
{quote}
{code:java}
DataBlockEncodingTool#checkStatistics
rawKVs = uncompressedOutputStream.toByteArray();
{code}
I doubt it a real rawKVs, since i see no about writing tags (if a kv has).
{quote}
Pardon me. Do you mean rawKVs may not be a real rawKVs because #checkStatistics
doesn't write tags to rawKVs ?
> add UT and docs for DataBlockEncodingTool
> -
>
> Key: HBASE-18201
> URL: https://issues.apache.org/jira/browse/HBASE-18201
> Project: HBase
> Issue Type: Sub-task
> Components: tooling
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-18201.master.001.patch,
> HBASE-18201.master.002.patch, HBASE-18201.master.002.patch,
> HBASE-18201.master.003.patch
>
>
> There is no example, documents, or tests for DataBlockEncodingTool. We should
> have it friendly if any use case exists. Otherwise, we should just get rid of
> it because DataBlockEncodingTool presumes that the implementation of cell
> returned from DataBlockEncoder is KeyValue. The presume may obstruct the
> cleanup of KeyValue references in the code base of read/write path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527115#comment-16527115 ] Kuan-Po Tseng commented on HBASE-18201: --- [~chia7712] OK, will submit next patch soon, thanks for your patience. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527099#comment-16527099 ] Kuan-Po Tseng commented on HBASE-18201: --- Yes, I think the way to check if hfile use tag by getting the HFileContext#isIncludesTags() is a more simple way. But I am not sure if there would be any bad consequence by doing this. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[
https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526485#comment-16526485
]
Kuan-Po Tseng commented on HBASE-18201:
---
Could we replace the way checking *_useTag_* with this code ?
{code:java}
HStoreFile hsf = new HStoreFile(fs, path, conf, cacheConf, BloomType.NONE,
true);
StoreFileReader reader = hsf.getReader();
boolean useTag = reader.getHFileReader().getFileContext().isIncludesTags();
{code}
> add UT and docs for DataBlockEncodingTool
> -
>
> Key: HBASE-18201
> URL: https://issues.apache.org/jira/browse/HBASE-18201
> Project: HBase
> Issue Type: Sub-task
> Components: tooling
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-18201.master.001.patch,
> HBASE-18201.master.002.patch, HBASE-18201.master.002.patch,
> HBASE-18201.master.003.patch
>
>
> There is no example, documents, or tests for DataBlockEncodingTool. We should
> have it friendly if any use case exists. Otherwise, we should just get rid of
> it because DataBlockEncodingTool presumes that the implementation of cell
> returned from DataBlockEncoder is KeyValue. The presume may obstruct the
> cleanup of KeyValue references in the code base of read/write path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526030#comment-16526030 ] Kuan-Po Tseng edited comment on HBASE-18201 at 6/28/18 7:40 AM: -Yeah, that would be wrong, > 0 should be >=0. Will fix in next patch.- We need to go through all KVs and check if there is one or more KVs that tagLength > 0，then useTag is true. was (Author: brandboat): Yeah, that would be wrong, > 0 should be >=0. Will fix in next patch. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526030#comment-16526030 ] Kuan-Po Tseng commented on HBASE-18201: --- Yeah, that would be wrong, > 0 should be >=0. Will fix in next patch. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[
https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526008#comment-16526008
]
Kuan-Po Tseng commented on HBASE-18201:
---
[~chia7712] The reason useTag only checks the last KV is because this tool load
data from HFile directly, tags would be all or none.
{code:java}
boolean useTag = (prevKV.getTagsLength() > 0);
{code}
> add UT and docs for DataBlockEncodingTool
> -
>
> Key: HBASE-18201
> URL: https://issues.apache.org/jira/browse/HBASE-18201
> Project: HBase
> Issue Type: Task
> Components: tooling
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-18201.master.001.patch,
> HBASE-18201.master.002.patch, HBASE-18201.master.002.patch,
> HBASE-18201.master.003.patch
>
>
> There is no example, documents, or tests for DataBlockEncodingTool. We should
> have it friendly if any use case exists. Otherwise, we should just get rid of
> it because DataBlockEncodingTool presumes that the implementation of cell
> returned from DataBlockEncoder is KeyValue. The presume may obstruct the
> cleanup of KeyValue references in the code base of read/write path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[
https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524687#comment-16524687
]
Kuan-Po Tseng commented on HBASE-18201:
---
[~chia7712] patch003 is as follows,
Bugs(4)
- Encoder ROW_INDEX_V1 throw error, things go wrong in class EncodedDataBlock
{code:java}
this.dataBlockEncoder.endBlockEncoding(encodingCtx, out, baosBytes);
{code}
the problem is ROW_INDEX_V1 write _onDiskDataSize_ in _out(DataOutputStream)_,
the others write _onDisDataSize_ in _baosBytes(byte array)_ directly,
since _onDiskDataSize_ is neccessary in the next steps, we need to flush _out_
again after _endBlockEncoding_ to write _onDiskDataSize_.
- DataBlockEncodingTool _checkStatistics_ would let currentKV be null, fixed.
- DataBlockEncodingTool _checkStatistics_ missing MemstoreTS.
- _compressedStream.reset()_ should happen before
_compressingStream.resetState()_ since in GZ
_resetStatue()_ will write header in outputstream. If we let
_compressedStream.reset()_ under _compressingStream.resetState()_, the header
is gone.
Tests(1)
- Going through all the DataBlockEncodingTool with GZ compression algorithm to
make sure it will run, not just compile correct.
Docs(1)
- Write down how to use DataBlockEncodingTool, which options is neccessary and
all the options in detail, and the result after using this tool.
Others(1)
- Change options name OPT_ENCODING_ALGORITHM to OPT_COMPRESSION_ALGORITHM.
> add UT and docs for DataBlockEncodingTool
> -
>
> Key: HBASE-18201
> URL: https://issues.apache.org/jira/browse/HBASE-18201
> Project: HBase
> Issue Type: Task
> Components: tooling
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-18201.master.001.patch,
> HBASE-18201.master.002.patch, HBASE-18201.master.002.patch,
> HBASE-18201.master.003.patch
>
>
> There is no example, documents, or tests for DataBlockEncodingTool. We should
> have it friendly if any use case exists. Otherwise, we should just get rid of
> it because DataBlockEncodingTool presumes that the implementation of cell
> returned from DataBlockEncoder is KeyValue. The presume may obstruct the
> cleanup of KeyValue references in the code base of read/write path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524677#comment-16524677 ] Kuan-Po Tseng commented on HBASE-18201: --- [~chia7712] Me too, I will correct it in the next patch. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524566#comment-16524566 ] Kuan-Po Tseng commented on HBASE-18201: --- [~chia7712] I wasn't change the logic of the code, what I modifed is to make sure the last kv isn't null. So yes, this only check the last kv, and I am also confusing about this segment... > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-18201: -- Attachment: HBASE-18201.master.003.patch > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch, HBASE-18201.master.002.patch, > HBASE-18201.master.003.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516567#comment-16516567 ] Kuan-Po Tseng commented on HBASE-18201: --- reattach patch to trigger testing. > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-18201: -- Attachment: HBASE-18201.master.002.patch > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch, > HBASE-18201.master.002.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513480#comment-16513480 ] Kuan-Po Tseng commented on HBASE-18201: --- Hi [~chia7712] ! I am not sure if I meet all the needs. Could you take a took ? > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-18201: -- Attachment: HBASE-18201.master.001.patch > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Task > Components: tooling >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-18201.master.001.patch > > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-8518) Get rid of hbase.hstore.compaction.complete setting
[ https://issues.apache.org/jira/browse/HBASE-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-8518: - Attachment: HBASE-8518.master.v5.patch Fixed that. Thanks for your advice. > Get rid of hbase.hstore.compaction.complete setting > --- > > Key: HBASE-8518 > URL: https://issues.apache.org/jira/browse/HBASE-8518 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-8518-1.patch, HBASE-8518.master.v0.patch, > HBASE-8518.master.v1.patch, HBASE-8518.master.v2.patch, > HBASE-8518.master.v3.patch, HBASE-8518.master.v4.patch, > HBASE-8518.master.v5.patch, HBASE-8518.wip.patch > > > hbase.hstore.compaction.complete is a strange setting that causes the > finished compaction to not complete (files are just left in tmp) in HStore. > It's used by one test. > The setting with the same name is also used by CompactionTool, but that usage > is semi-unrelated and could probably be removed easily. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-8518) Get rid of hbase.hstore.compaction.complete setting
[
https://issues.apache.org/jira/browse/HBASE-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuan-Po Tseng updated HBASE-8518:
-
Attachment: HBASE-8518.master.v4.patch
Modifying comments as below.
{code:java}
/**
* Create HStore instance.
* @return If Mob is enabled, return HMobStore, otherwise return
* HStoreForTesting, a HStore for testing.
*/
{code}
Thanks for your review.
> Get rid of hbase.hstore.compaction.complete setting
> ---
>
> Key: HBASE-8518
> URL: https://issues.apache.org/jira/browse/HBASE-8518
> Project: HBase
> Issue Type: Improvement
>Reporter: Sergey Shelukhin
>Assignee: Kuan-Po Tseng
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-8518-1.patch, HBASE-8518.master.v0.patch,
> HBASE-8518.master.v1.patch, HBASE-8518.master.v2.patch,
> HBASE-8518.master.v3.patch, HBASE-8518.master.v4.patch, HBASE-8518.wip.patch
>
>
> hbase.hstore.compaction.complete is a strange setting that causes the
> finished compaction to not complete (files are just left in tmp) in HStore.
> It's used by one test.
> The setting with the same name is also used by CompactionTool, but that usage
> is semi-unrelated and could probably be removed easily.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-8518) Get rid of hbase.hstore.compaction.complete setting
[ https://issues.apache.org/jira/browse/HBASE-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-8518: - Attachment: HBASE-8518.master.v3.patch All done. Thanks for your comments. > Get rid of hbase.hstore.compaction.complete setting > --- > > Key: HBASE-8518 > URL: https://issues.apache.org/jira/browse/HBASE-8518 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-8518-1.patch, HBASE-8518.master.v0.patch, > HBASE-8518.master.v1.patch, HBASE-8518.master.v2.patch, > HBASE-8518.master.v3.patch, HBASE-8518.wip.patch > > > hbase.hstore.compaction.complete is a strange setting that causes the > finished compaction to not complete (files are just left in tmp) in HStore. > It's used by one test. > The setting with the same name is also used by CompactionTool, but that usage > is semi-unrelated and could probably be removed easily. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-8518) Get rid of hbase.hstore.compaction.complete setting
[ https://issues.apache.org/jira/browse/HBASE-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-8518: - Attachment: HBASE-8518.master.v2.patch Fix import errors > Get rid of hbase.hstore.compaction.complete setting > --- > > Key: HBASE-8518 > URL: https://issues.apache.org/jira/browse/HBASE-8518 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-8518-1.patch, HBASE-8518.master.v0.patch, > HBASE-8518.master.v1.patch, HBASE-8518.master.v2.patch, HBASE-8518.wip.patch > > > hbase.hstore.compaction.complete is a strange setting that causes the > finished compaction to not complete (files are just left in tmp) in HStore. > It's used by one test. > The setting with the same name is also used by CompactionTool, but that usage > is semi-unrelated and could probably be removed easily. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-8518) Get rid of hbase.hstore.compaction.complete setting
[ https://issues.apache.org/jira/browse/HBASE-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-8518: - Attachment: HBASE-8518.master.v1.patch [~chia7712] Already done what you said, can you take a look ? > Get rid of hbase.hstore.compaction.complete setting > --- > > Key: HBASE-8518 > URL: https://issues.apache.org/jira/browse/HBASE-8518 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-8518-1.patch, HBASE-8518.master.v0.patch, > HBASE-8518.master.v1.patch, HBASE-8518.wip.patch > > > hbase.hstore.compaction.complete is a strange setting that causes the > finished compaction to not complete (files are just left in tmp) in HStore. > It's used by one test. > The setting with the same name is also used by CompactionTool, but that usage > is semi-unrelated and could probably be removed easily. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-8518) Get rid of hbase.hstore.compaction.complete setting
[ https://issues.apache.org/jira/browse/HBASE-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-8518: - Attachment: HBASE-8518.master.v0.patch * delete "hbase.hstore.compaction.complete" settings in CompactionTool.java and HStore.java * create another HStore (HStoreForTesting.java) which keep "hbase.hstore.compaction.complete" for Testing. * HStoreForTesting.java contains the former compact method and its related method. * HRegionForTesting.java is for creating different Stores : HStoreForTesting. > Get rid of hbase.hstore.compaction.complete setting > --- > > Key: HBASE-8518 > URL: https://issues.apache.org/jira/browse/HBASE-8518 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-8518-1.patch, HBASE-8518.master.v0.patch > > > hbase.hstore.compaction.complete is a strange setting that causes the > finished compaction to not complete (files are just left in tmp) in HStore. > It's used by one test. > The setting with the same name is also used by CompactionTool, but that usage > is semi-unrelated and could probably be removed easily. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-8518) Get rid of hbase.hstore.compaction.complete setting
[ https://issues.apache.org/jira/browse/HBASE-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304120#comment-16304120 ] Kuan-Po Tseng commented on HBASE-8518: -- [~chia7712] Thanks for your comment. I will give it a try. > Get rid of hbase.hstore.compaction.complete setting > --- > > Key: HBASE-8518 > URL: https://issues.apache.org/jira/browse/HBASE-8518 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Kuan-Po Tseng >Priority: Minor > Labels: beginner > Attachments: HBASE-8518-1.patch > > > hbase.hstore.compaction.complete is a strange setting that causes the > finished compaction to not complete (files are just left in tmp) in HStore. > It's used by one test. > The setting with the same name is also used by CompactionTool, but that usage > is semi-unrelated and could probably be removed easily. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18201) add UT and docs for DataBlockEncodingTool
[ https://issues.apache.org/jira/browse/HBASE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165905#comment-16165905 ] Kuan-Po Tseng commented on HBASE-18201: --- [~chia7712] May I take over this issue ? > add UT and docs for DataBlockEncodingTool > - > > Key: HBASE-18201 > URL: https://issues.apache.org/jira/browse/HBASE-18201 > Project: HBase > Issue Type: Task > Components: tooling >Reporter: Chia-Ping Tsai >Priority: Minor > Labels: beginner > > There is no example, documents, or tests for DataBlockEncodingTool. We should > have it friendly if any use case exists. Otherwise, we should just get rid of > it because DataBlockEncodingTool presumes that the implementation of cell > returned from DataBlockEncoder is KeyValue. The presume may obstruct the > cleanup of KeyValue references in the code base of read/write path. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-17980: -- Attachment: HBASE-17980.master.v6.patch > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-17980.master.001.patch, > HBASE-17980.master.002.patch, HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch, > HBASE-17980-master.v2.patch, HBASE-17980.master.v3.patch, > HBASE-17980.master.v4.patch, HBASE-17980.master.v5.patch, > HBASE-17980.master.v6.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-17980: -- Attachment: HBASE-17980.master.v5.patch [~chia7712] I try my best to replace almost static method impl inside HRI. And Fix all javadoc warnings. May u take a look ? > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-17980.master.001.patch, > HBASE-17980.master.002.patch, HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch, > HBASE-17980-master.v2.patch, HBASE-17980.master.v3.patch, > HBASE-17980.master.v4.patch, HBASE-17980.master.v5.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[
https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164837#comment-16164837
]
Kuan-Po Tseng commented on HBASE-17980:
---
[~stack] Almost done. My last question is could RegionInfoDisplay.java be
placed in hbase-client module (in org.apache.hadoop.hbase.util) ? So we could
replace all these deprecated methods below in HRegionInfo.
{code:java}
public static String getDescriptiveNameFromRegionStateForDisplay(RegionState
state, Configuration conf)
public static byte[] getEndKeyForDisplay(RegionInfo ri, Configuration conf)
public static byte[] getStartKeyForDisplay(RegionInfo ri, Configuration conf)
public static String getRegionNameAsStringForDisplay(RegionInfo ri,
Configuration conf)
public static byte[] getRegionNameForDisplay(RegionInfo ri, Configuration
conf)
{code}
> Any HRegionInfo we give out should be immutable
> ---
>
> Key: HBASE-17980
> URL: https://issues.apache.org/jira/browse/HBASE-17980
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
> Labels: beginner
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-17980.master.001.patch,
> HBASE-17980.master.002.patch, HBASE-17980.master.v0.patch,
> HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch,
> HBASE-17980-master.v2.patch, HBASE-17980.master.v3.patch,
> HBASE-17980.master.v4.patch
>
>
> This is similar to HBASE-15583.
> # Introduce RegionInfo class. HRegionInfo will extend RegionInfo.
> # Deprecate HRegionInfo to be removed in 3.0
> # RegionInfo contain all of the read-only methods of HRegionInfo
> # Add "RegionInfo Builder"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164773#comment-16164773 ] Kuan-Po Tseng commented on HBASE-17980: --- [~stack] I am now working on it. Could you give me some time ? I will finish it as soon as possibile. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-17980.master.001.patch, > HBASE-17980.master.002.patch, HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch, > HBASE-17980-master.v2.patch, HBASE-17980.master.v3.patch, > HBASE-17980.master.v4.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-17980: -- Attachment: HBASE-17980.master.v4.patch remove accidentally uploaded netbeans settings. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-17980.master.001.patch, > HBASE-17980.master.002.patch, HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch, > HBASE-17980-master.v2.patch, HBASE-17980.master.v3.patch, > HBASE-17980.master.v4.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-17980: -- Attachment: HBASE-17980.master.v3.patch [~chia7712]] I already make those static IA.private and put them in RegionInfoUtil.java. And put pb <-> RegionInfo to ProtobufUtil.java > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-17980.master.001.patch, > HBASE-17980.master.002.patch, HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch, > HBASE-17980-master.v2.patch, HBASE-17980.master.v3.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164247#comment-16164247 ] Kuan-Po Tseng edited comment on HBASE-17980 at 9/13/17 7:44 AM: [~stack] Yeah, I have time to take it on. -But the file you recently attach is the same as I attached. Is it something wrong ?- My mistake was (Author: brandboat): -[~stack] Yeah, I have time to take it on. But the file you recently attach is the same as I attached. Is it something wrong ?- My mistake > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.001.patch, > HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch, > HBASE-17980-master.v2.patch, HBASE-17980-master.v2.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164247#comment-16164247 ] Kuan-Po Tseng edited comment on HBASE-17980 at 9/13/17 7:33 AM: -[~stack] Yeah, I have time to take it on. But the file you recently attach is the same as I attached. Is it something wrong ?- My mistake was (Author: brandboat): [~stack] Yeah, I have time to take it on. But the file you recently attach is the same as I attached. Is it something wrong ? > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.001.patch, > HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch, > HBASE-17980-master.v2.patch, HBASE-17980-master.v2.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164247#comment-16164247 ] Kuan-Po Tseng commented on HBASE-17980: --- [~stack] Yeah, I have time to take it on. But the file you recently attach is the same as I attached. Is it something wrong ? > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.001.patch, > HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch, > HBASE-17980-master.v2.patch, HBASE-17980-master.v2.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164177#comment-16164177 ] Kuan-Po Tseng commented on HBASE-17980: --- [~stack] Are you alreday doing this stuff ? There are just a few private static methods in RegionInfoBuilder. I think I can fix that today. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch, > HBASE-17980-master.v2.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164154#comment-16164154 ] Kuan-Po Tseng commented on HBASE-17980: --- [~chia7712] [~stack] Here is the review board [link|http://example.com]. Yeah, those static methods makes RegionInfoBuilder looks too fat. I will go check what problem cause these test stuck... > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch, > HBASE-17980-master.v2.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163312#comment-16163312 ] Kuan-Po Tseng edited comment on HBASE-17980 at 9/12/17 5:30 PM: v2 - Rename UnmodifyableHRegionInfo.java to ImmutableHRegionInfo.java - make ImmutableHRegionInfo IA.private - make MutableRegionInfo private - move all static methods inside MutableRI to RegionInfoBuilder - move pb <-> RegionInfo to shaded ProtoBufUtil - deprecated all static methods and toByteArray() and toDelimitedByteArray() in HRegionInfo, deprecated HRegionInfo. - Deprecated Admin getOnlineRegions(ServerName) and getTableRegions(TableName), add getRegions(ServerName) getRegions(TableName) - Add TestRegionInfoBuilder which copy all tests from HRegionInfo. - purge useless method MetaMockingUtil#getMetaTableRowResultAsSplitRegion was (Author: brandboat): v2 - Rename UnmodifyableHRegionInfo.java to ImmutableHRegionInfo.java - make ImmutableHRegionInfo IA.private - make MutableRegionInfo private - move all static methods inside MutableRI to RegionInfoBuilder - move pb <-> RegionInfo to shaded ProtoBufUtil - deprecated all static methods and toByteArray() and toDelimitedByteArray() in HRegionInfo, deprecated HRegionInfo. - Deprecated Admin getOnlineRegions(ServerName) and getTableRegions(TableName), add getRegions(ServerName) getRegions(TableName) - Add TestRegionInfoBuilder which copy all tests from HRegionInfo. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163312#comment-16163312 ] Kuan-Po Tseng edited comment on HBASE-17980 at 9/12/17 5:29 PM: v2 - Rename UnmodifyableHRegionInfo.java to ImmutableHRegionInfo.java - make ImmutableHRegionInfo IA.private - make MutableRegionInfo private - move all static methods inside MutableRI to RegionInfoBuilder - move pb <-> RegionInfo to shaded ProtoBufUtil - deprecated all static methods and toByteArray() and toDelimitedByteArray() in HRegionInfo, deprecated HRegionInfo. - Deprecated Admin getOnlineRegions(ServerName) and getTableRegions(TableName), add getRegions(ServerName) getRegions(TableName) - Add TestRegionInfoBuilder which copy all tests from HRegionInfo. was (Author: brandboat): v2 - Rename UnmodifyableHRegionInfo.java to ImmutableHRegionInfo.java - make ImmutableHRegionInfo IA.private - make MutableRegionInfo private - move all static methods inside MutableRI to RegionInfoBuilder - deprecated all static methods and toByteArray() and toDelimitedByteArray() in HRegionInfo, deprecated HRegionInfo. - Deprecated Admin getOnlineRegions(ServerName) and getTableRegions(TableName), add getRegions(ServerName) getRegions(TableName) - Add TestRegionInfoBuilder which copy all tests from HRegionInfo. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163312#comment-16163312 ] Kuan-Po Tseng edited comment on HBASE-17980 at 9/12/17 5:23 PM: v2 - Rename UnmodifyableHRegionInfo.java to ImmutableHRegionInfo.java - make ImmutableHRegionInfo IA.private - make MutableRegionInfo private - move all static methods inside MutableRI to RegionInfoBuilder - deprecated all static methods and toByteArray() and toDelimitedByteArray() in HRegionInfo, deprecated HRegionInfo. - Deprecated Admin getOnlineRegions(ServerName) and getTableRegions(TableName), add getRegions(ServerName) getRegions(TableName) - Add TestRegionInfoBuilder which copy all tests from HRegionInfo. was (Author: brandboat): v2 - Rename UnmodifyableHRegionInfo.java to ImmutableHRegionInfo.java - make ImmutableHRegionInfo IA.private - make MutableRegionInfo private - move all static methods inside MutableRI under RegionInfoBuilder - deprecated all static methods and toByteArray() and toDelimitedByteArray() in HRegionInfo, deprecated HRegionInfo. - Deprecated Admin getOnlineRegions(ServerName) and getTableRegions(TableName), add getRegions(ServerName) getRegions(TableName) - Add TestRegionInfoBuilder which copy all tests from HRegionInfo. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163312#comment-16163312 ] Kuan-Po Tseng commented on HBASE-17980: --- v2 - Rename UnmodifyableHRegionInfo.java to ImmutableHRegionInfo.java - make ImmutableHRegionInfo IA.private - make MutableRegionInfo private - move all static methods inside MutableRI under RegionInfoBuilder - deprecated all static methods and toByteArray() and toDelimitedByteArray() in HRegionInfo, deprecated HRegionInfo. - Deprecated Admin getOnlineRegions(ServerName) and getTableRegions(TableName), add getRegions(ServerName) getRegions(TableName) - Add TestRegionInfoBuilder which copy all tests from HRegionInfo. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-17980: -- Attachment: HBASE-17980-master.v2.patch > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, > HBASE-17980.master.v1.patch, HBASE-17980-master.v2.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163066#comment-16163066 ] Kuan-Po Tseng commented on HBASE-17980: --- [~stack] OK, I will leave closeRegion alone. Then change both getOnlineRegions and getTableRegions to getRegions, thanks. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[
https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163038#comment-16163038
]
Kuan-Po Tseng commented on HBASE-17980:
---
Hello [~chia7712],
{code:java}
void closeRegion(final ServerName sn, final HRegionInfo ri) throws IOException;
List getOnlineRegions(final ServerName sn) throws IOException;
List getTableRegions(final TableName tableName) throws IOException;
{code}
These are the three method need to be corrected, my question is if
getOnlineRegions and getTableRegions is deprecated,
what name should we use to replace these new method ?
> Any HRegionInfo we give out should be immutable
> ---
>
> Key: HBASE-17980
> URL: https://issues.apache.org/jira/browse/HBASE-17980
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch
>
>
> This is similar to HBASE-15583.
> # Introduce RegionInfo class. HRegionInfo will extend RegionInfo.
> # Deprecate HRegionInfo to be removed in 3.0
> # RegionInfo contain all of the read-only methods of HRegionInfo
> # Add "RegionInfo Builder"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162458#comment-16162458 ] Kuan-Po Tseng commented on HBASE-17980: --- Sorry, I will submit another patch today or tomorrow. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159082#comment-16159082 ] Kuan-Po Tseng commented on HBASE-17980: --- Copy that. Thanks for your advice~ > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159008#comment-16159008 ] Kuan-Po Tseng commented on HBASE-17980: --- Hello stack, 1. Yes, MutableRI is set to IA.Private. 2. I will correct the ImmutableRI to be IA.Private. Sorry about this mistake. 3. MutableRI is public because HRI delegate all things to MutableRI in this patch, if not so, we have to let MutableRI be private. Do you think let MutabableRI be private is better ??? > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158302#comment-16158302 ] Kuan-Po Tseng commented on HBASE-17980: --- Hello stack, Now I make MutableRegionInfo public is because I make nearly all methods inside HRegionInfo delegate to MutableRegionInfo, so MutableRegion must be public in this patch. My thought is let MutableRegionInfo be a private class only for RegionInfoBuilder to use after HRegionInfo is totally deleted. Or make MutableRegionInfo private right now and let HRegionInfo remain it used to be ? > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-17980) Any HRegionInfo we give out should be immutable
[
https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154878#comment-16154878
]
Kuan-Po Tseng edited comment on HBASE-17980 at 9/6/17 6:29 AM:
---
Hello stack,
Now we have a Mutable(inside RegionInfoBuilder) and Immutable one, all
HRegionInfo could be replaced with ImmutableHRegionInfo instead of
{code:java}
public static Result getMetaTableRowResultAsSplitRegion(
final HRegionInfo hri, final ServerName sn) throws IOException {
hri.setOffline(true);
hri.setSplit(true);
return getMetaTableRowResult(hri, sn);
}
{code} in MetaMockingUtil.java. But I don't find any class is using this
mehod...
I stiil keep MutableRegionInfo if anyone still need it can be modified.
And RegionInfoBuilder will build RegionInfo which isn't mutable.
I didn't change Admin Interface API in this patch, should I replace all
HRegionInfo to RegionInfo in this patch ???
I will modify getRegionInfoForReplica from
{code:java}
public static MutableRegionInfo getRegionInfoForReplica(MutableRegionInfo
regionInfo, int replicaId)
{code}
into this :
{code:java}
public static RegionInfo getRegionInfoForReplica(RegionInfo regionInfo, int
replicaId)
{code}
Thanks for your review.
was (Author: brandboat):
Hello stack,
Now we have a Mutable(inside RegionInfoBuilder) and Immutable one, all
HRegionInfo could be replaced with ImmutableHRegionInfo instead of
{code:java}
public static Result getMetaTableRowResultAsSplitRegion(
final HRegionInfo hri, final ServerName sn) throws IOException {
hri.setOffline(true);
hri.setSplit(true);
return getMetaTableRowResult(hri, sn);
}
{code} in MetaMockingUtil.java. But I don't find any class is using this
mehod...
I stiil keep MutableRegionInfo if anyone still need it can be modified.
And RegionInfoBuilder will build RegionInfo which isn't a mutable.
I didn't change Admin Interface API in this patch, should I replace all
HRegionInfo to RegionInfo in this patch ???
I will modify getRegionInfoForReplica from
{code:java}
public static MutableRegionInfo getRegionInfoForReplica(MutableRegionInfo
regionInfo, int replicaId)
{code}
into this :
{code:java}
public static RegionInfo getRegionInfoForReplica(RegionInfo regionInfo, int
replicaId)
{code}
Thanks for your review.
> Any HRegionInfo we give out should be immutable
> ---
>
> Key: HBASE-17980
> URL: https://issues.apache.org/jira/browse/HBASE-17980
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch
>
>
> This is similar to HBASE-15583.
> # Introduce RegionInfo class. HRegionInfo will extend RegionInfo.
> # Deprecate HRegionInfo to be removed in 3.0
> # RegionInfo contain all of the read-only methods of HRegionInfo
> # Add "RegionInfo Builder"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[
https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154878#comment-16154878
]
Kuan-Po Tseng commented on HBASE-17980:
---
Hello stack,
Now we have a Mutable(inside RegionInfoBuilder) and Immutable one, all
HRegionInfo could be replaced with ImmutableHRegionInfo instead of
{code:java}
public static Result getMetaTableRowResultAsSplitRegion(
final HRegionInfo hri, final ServerName sn) throws IOException {
hri.setOffline(true);
hri.setSplit(true);
return getMetaTableRowResult(hri, sn);
}
{code} in MetaMockingUtil.java. But I don't find any class is using this
mehod...
I stiil keep MutableRegionInfo if anyone still need it can be modified.
And RegionInfoBuilder will build RegionInfo which isn't a mutable.
I didn't change Admin Interface API in this patch, should I replace all
HRegionInfo to RegionInfo in this patch ???
I will modify getRegionInfoForReplica from
{code:java}
public static MutableRegionInfo getRegionInfoForReplica(MutableRegionInfo
regionInfo, int replicaId)
{code}
into this :
{code:java}
public static RegionInfo getRegionInfoForReplica(RegionInfo regionInfo, int
replicaId)
{code}
Thanks for your review.
> Any HRegionInfo we give out should be immutable
> ---
>
> Key: HBASE-17980
> URL: https://issues.apache.org/jira/browse/HBASE-17980
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Kuan-Po Tseng
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch
>
>
> This is similar to HBASE-15583.
> # Introduce RegionInfo class. HRegionInfo will extend RegionInfo.
> # Deprecate HRegionInfo to be removed in 3.0
> # RegionInfo contain all of the read-only methods of HRegionInfo
> # Add "RegionInfo Builder"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-17980: -- Attachment: HBASE-17980.master.v1.patch Add missing license and remove trailing space. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-17980: -- Status: Patch Available (was: Open) > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan-Po Tseng updated HBASE-17980: -- Attachment: HBASE-17980.master.v0.patch 1. Introduce RegionInfo class. HRegionInfo implements RegionInfo. 2. Deprecate HRegionInfo to be removed in 3.0 3. Interface RegionInfo contain all of the read-only methods of HRegionInfo 4. Add "RegionInfo Builder" 5. Rename UnmodifyableHRegionInfo.java to ImmutableHRegionInfo.java 6. Replace HRegionInfo in #getOnlineRegions and #getOfflineRegions in HBaseAdmin.java with ImmutableHRegionInfo > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151339#comment-16151339 ] Kuan-Po Tseng commented on HBASE-17980: --- Almost done. Now I am adding some tests. But I have a question, I am not quite sure which place should I replace HRegionInfo with ImmutableHRegionInfo. What I pretty sure is that in HBaseAdmin List getOnlineRegions(ServerName sn) this methods should return ImmutableHRegionInfo. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16014) Get and Put constructor argument lists are divergent
[
https://issues.apache.org/jira/browse/HBASE-16014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369058#comment-15369058
]
Kuan-Po Tseng commented on HBASE-16014:
---
[~ndimiduk] Are you going to take this issue ? I would like to solve this. May
I ?
> Get and Put constructor argument lists are divergent
>
>
> Key: HBASE-16014
> URL: https://issues.apache.org/jira/browse/HBASE-16014
> Project: HBase
> Issue Type: Bug
>Reporter: Nick Dimiduk
>
> API for construing Get and Put objects for a specific rowkey is quite
> different.
> [Put|http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Put.html#constructor_summary]
> supports many more variations for specifying the target rowkey and timestamp
> compared to
> [Get|http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Get.html#constructor_summary].
> Notably lacking are {{Get(byte[], int, int)}} and {{Get(ByteBuffer)}}
> variations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
