[jira] [Commented] (HBASE-17291) Remove ImmutableSegment#getKeyValueScanner
[ https://issues.apache.org/jira/browse/HBASE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15777830#comment-15777830 ] ramkrishna.s.vasudevan commented on HBASE-17291: Thanks for the review [~anastas]. bq. I do not understand where the close() of the SnapshotScanner should happen in the common path. The close of the SnapshotScanner happens when the Flush scanner is closed. The flush path creates a StoreScanner which has the memstore scanner - which is made up of this snapshotScanner. When the flush store scanner is closed this will get automatically closed. In tests - since in the old way there was no segment ref counting - we were just directly clearing the snapshot and it was able to return the segments back when the Segment close was called. bq.I assume you will need to wait till HBASE-17081 is finally committed (after it was reverted) till you can commit this one, right? If you need more time to get HBASE-17081 to check in after all the changes are corrected I can commit this but you need to rebase HBASE-17081 on top of this. BTW what I think since HBASE-17333 solves the test case issue it is better we commit HBASE-17081 now? > Remove ImmutableSegment#getKeyValueScanner > -- > > Key: HBASE-17291 > URL: https://issues.apache.org/jira/browse/HBASE-17291 > Project: HBase > Issue Type: Improvement > Components: Scanners >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-17291.patch, HBASE-17291_1.patch, > HBASE-17291_2.patch > > > This is based on a discussion over [~anastas]'s patch. The MemstoreSnapshot > uses a KeyValueScanner which actually seems redundant considering we already > have a SegmentScanner. The idea is that the snapshot scanner should be a > simple iterator type of scanner but it lacks the capability to do the > reference counting on that segment that is now used in snapshot. With > snapshot having mulitple segments in the latest impl it is better we hold on > to the segment by doing ref counting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17336) get/update replication peer config requests should be routed through master
[ https://issues.apache.org/jira/browse/HBASE-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17336: --- Attachment: HBASE-17336-v2.patch Fix failed ut. > get/update replication peer config requests should be routed through master > --- > > Key: HBASE-17336 > URL: https://issues.apache.org/jira/browse/HBASE-17336 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17336-v1.patch, HBASE-17336-v2.patch > > > As HBASE-11392 description says, we should move replication operations to be > routed through master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17320) Add inclusive/exclusive support for startRow and endRow of scan
[ https://issues.apache.org/jira/browse/HBASE-17320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17320: -- Attachment: HBASE-17320-v1.patch Fix failing tests. > Add inclusive/exclusive support for startRow and endRow of scan > --- > > Key: HBASE-17320 > URL: https://issues.apache.org/jira/browse/HBASE-17320 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17320-v1.patch, HBASE-17320.patch > > > This is especially useful when doing reverse scan. HBASE-17168 maybe a more > powerful solution but we need to be careful about the atomicity, and I do not > think we will provide the feature to end user. But I think it is OK to > provide inclusive/exclusive option to end user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16981) Expand Mob Compaction Partition policy from daily to weekly, monthly and beyond
[ https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15776708#comment-15776708 ] huaxiang sun commented on HBASE-16981: -- Hi [~anoop.hbase], the design doc is at https://docs.google.com/document/d/1y-jCl1TgMecK7j62MHRtholzyoQaXB3bq0jIKQszE64/edit#heading=h.u1drxejvn2lt . It is linked in the jira as well. Thanks. > Expand Mob Compaction Partition policy from daily to weekly, monthly and > beyond > --- > > Key: HBASE-16981 > URL: https://issues.apache.org/jira/browse/HBASE-16981 > Project: HBase > Issue Type: New Feature > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-16981.master.001.patch, > HBASE-16981.master.002.patch, HBASE-16981.master.003.patch, > HBASE-16981.master.004.patch, > Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf > > > Today the mob region holds all mob files for all regions. With daily > partition mob compaction policy, after major mob compaction, there is still > one file per region daily. Given there is 365 days in one year, at least 365 > files per region. Since HDFS has limitation for number of files under one > folder, this is not going to scale if there are lots of regions. To reduce > mob file number, we want to introduce other partition policies such as > weekly, monthly to compact mob files within one week or month into one file. > This jira is create to track this effort. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17320) Add inclusive/exclusive support for startRow and endRow of scan
[
https://issues.apache.org/jira/browse/HBASE-17320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15776678#comment-15776678
]
Hadoop QA commented on HBASE-17320:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 51s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 56s
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 33s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 41s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 17s
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 27s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 16s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 76m 1s {color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestStoreFile |
| | hadoop.hbase.filter.TestFilter |
| | hadoop.hbase.protobuf.TestProtobufUtil |
| | hadoop.hbase.regionserver.TestScanWithBloomError |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA
[jira] [Commented] (HBASE-17345) Implement batch
[ https://issues.apache.org/jira/browse/HBASE-17345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15776639#comment-15776639 ] Hudson commented on HBASE-17345: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2196 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2196/]) HBASE-17345 Implement batch (zhangduo: rev 8fa5b0b946c01516076fa944a310b33224ff21a9) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCallerFactory.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/AbstractTestAsyncTableScan.java * (add) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncBatchRpcRetryingCaller.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableGetMultiThreaded.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncTableBase.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableBatch.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncTableImpl.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncGetMultiThread.java * (delete) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncMultiGetRpcRetryingCaller.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncSingleRequestRpcRetryingCaller.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncScanSingleRegionRpcRetryingCaller.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableMultiGet.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncTableImpl.java > Implement batch > --- > > Key: HBASE-17345 > URL: https://issues.apache.org/jira/browse/HBASE-17345 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17345-v1.patch, HBASE-17345-v2.patch, > HBASE-17345-v3.patch, HBASE-17345-v4.patch, HBASE-17345.patch > > > Add the support for general batch based on the code introduced in HBASE-17142. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17320) Add inclusive/exclusive support for startRow and endRow of scan
[ https://issues.apache.org/jira/browse/HBASE-17320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17320: -- Status: Patch Available (was: Open) > Add inclusive/exclusive support for startRow and endRow of scan > --- > > Key: HBASE-17320 > URL: https://issues.apache.org/jira/browse/HBASE-17320 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17320.patch > > > This is especially useful when doing reverse scan. HBASE-17168 maybe a more > powerful solution but we need to be careful about the atomicity, and I do not > think we will provide the feature to end user. But I think it is OK to > provide inclusive/exclusive option to end user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17320) Add inclusive/exclusive support for startRow and endRow of scan
[ https://issues.apache.org/jira/browse/HBASE-17320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17320: -- Attachment: HBASE-17320.patch An initial version. Have passed all the scan tests of async table. The patch contains server side changes and some semantics change of the Scan object, for example, isGetScan, so I'm not sure if it will breaks something. Let's see the pre commit result. > Add inclusive/exclusive support for startRow and endRow of scan > --- > > Key: HBASE-17320 > URL: https://issues.apache.org/jira/browse/HBASE-17320 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17320.patch > > > This is especially useful when doing reverse scan. HBASE-17168 maybe a more > powerful solution but we need to be careful about the atomicity, and I do not > think we will provide the feature to end user. But I think it is OK to > provide inclusive/exclusive option to end user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17320) Add inclusive/exclusive support for startRow and endRow of scan
[ https://issues.apache.org/jira/browse/HBASE-17320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17320: -- Affects Version/s: (was: 1.4.0) Fix Version/s: (was: 1.4.0) > Add inclusive/exclusive support for startRow and endRow of scan > --- > > Key: HBASE-17320 > URL: https://issues.apache.org/jira/browse/HBASE-17320 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17320.patch > > > This is especially useful when doing reverse scan. HBASE-17168 maybe a more > powerful solution but we need to be careful about the atomicity, and I do not > think we will provide the feature to end user. But I think it is OK to > provide inclusive/exclusive option to end user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16981) Expand Mob Compaction Partition policy from daily to weekly, monthly and beyond
[ https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15776501#comment-15776501 ] Anoop Sam John commented on HBASE-16981: bq.Hi Anoop Sam John, would you mind putting some comments in the doc? Thanks. Whether there is a new design doc in place? Will be better if we have one and share it as a google doc. Easy to review and add comments. > Expand Mob Compaction Partition policy from daily to weekly, monthly and > beyond > --- > > Key: HBASE-16981 > URL: https://issues.apache.org/jira/browse/HBASE-16981 > Project: HBase > Issue Type: New Feature > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-16981.master.001.patch, > HBASE-16981.master.002.patch, HBASE-16981.master.003.patch, > HBASE-16981.master.004.patch, > Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf > > > Today the mob region holds all mob files for all regions. With daily > partition mob compaction policy, after major mob compaction, there is still > one file per region daily. Given there is 365 days in one year, at least 365 > files per region. Since HDFS has limitation for number of files under one > folder, this is not going to scale if there are lots of regions. To reduce > mob file number, we want to introduce other partition policies such as > weekly, monthly to compact mob files within one week or month into one file. > This jira is create to track this effort. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17345) Implement batch
[ https://issues.apache.org/jira/browse/HBASE-17345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17345: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to master. Thanks all for reviewing. > Implement batch > --- > > Key: HBASE-17345 > URL: https://issues.apache.org/jira/browse/HBASE-17345 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17345-v1.patch, HBASE-17345-v2.patch, > HBASE-17345-v3.patch, HBASE-17345-v4.patch, HBASE-17345.patch > > > Add the support for general batch based on the code introduced in HBASE-17142. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17345) Implement batch
[ https://issues.apache.org/jira/browse/HBASE-17345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15776415#comment-15776415 ] Duo Zhang commented on HBASE-17345: --- Thanks [~carp84]. Let me commit. > Implement batch > --- > > Key: HBASE-17345 > URL: https://issues.apache.org/jira/browse/HBASE-17345 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17345-v1.patch, HBASE-17345-v2.patch, > HBASE-17345-v3.patch, HBASE-17345-v4.patch, HBASE-17345.patch > > > Add the support for general batch based on the code introduced in HBASE-17142. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17345) Implement batch
[ https://issues.apache.org/jira/browse/HBASE-17345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15776169#comment-15776169 ] Yu Li commented on HBASE-17345: --- Just noticed... +1 on patch v4, thanks. > Implement batch > --- > > Key: HBASE-17345 > URL: https://issues.apache.org/jira/browse/HBASE-17345 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17345-v1.patch, HBASE-17345-v2.patch, > HBASE-17345-v3.patch, HBASE-17345-v4.patch, HBASE-17345.patch > > > Add the support for general batch based on the code introduced in HBASE-17142. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
