[jira] [Commented] (HBASE-18830) TestCanaryTool does not check Canary monitor's error code
[ https://issues.apache.org/jira/browse/HBASE-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180299#comment-16180299 ] Hudson commented on HBASE-18830: FAILURE: Integrated in Jenkins build HBase-1.4 #932 (See [https://builds.apache.org/job/HBase-1.4/932/]) HBASE-18830 TestCanaryTool does not check Canary monitor's error code (apurtell: rev decf9d6235e23a1e0b2bb4ea315079c513c8e059) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java > TestCanaryTool does not check Canary monitor's error code > - > > Key: HBASE-18830 > URL: https://issues.apache.org/jira/browse/HBASE-18830 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18830.001.patch > > > None of the tests inside TestCanaryTool check Canary monitor's error code. > Thus, it is possible that the monitor has registered an error and yet the > tests pass. We should check the value returned by the _ToolRunner.run()_ > method inside each unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18762) Canary sink type cast error
[ https://issues.apache.org/jira/browse/HBASE-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180298#comment-16180298 ] Hudson commented on HBASE-18762: FAILURE: Integrated in Jenkins build HBase-1.4 #932 (See [https://builds.apache.org/job/HBase-1.4/932/]) HBASE-18762 Canary sink type cast error (apurtell: rev eace8b14b3443b61562c434d84ded51bb9843a56) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java > Canary sink type cast error > --- > > Key: HBASE-18762 > URL: https://issues.apache.org/jira/browse/HBASE-18762 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18762.001.patch > > > When running the main method of Canary.java, we see the following error: > Exception in thread "main" java.lang.ClassCastException: > org.apache.hadoop.hbase.tool.Canary$RegionServerStdOutSink cannot be cast to > org.apache.hadoop.hbase.tool.Canary$RegionStdOutSink > at org.apache.hadoop.hbase.tool.Canary.newMonitor(Canary.java:911) > at org.apache.hadoop.hbase.tool.Canary.run(Canary.java:796) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.hbase.tool.Canary.main(Canary.java:1571) > This happens because we typecast the sink depending on the mode (zookeeper > mode/region server mode) that Canary is configured in. In case no mode is > specified, we typecast the sink into _RegionStdOutSink_. In general, it is > possible to provide inconsistent mode and sink types while running Canary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18875) Thrift server supports read-only mode
[ https://issues.apache.org/jira/browse/HBASE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180297#comment-16180297 ] Hudson commented on HBASE-18875: FAILURE: Integrated in Jenkins build HBase-1.4 #932 (See [https://builds.apache.org/job/HBase-1.4/932/]) HBASE-18875 Thrift server supports read-only mode (tedyu: rev 9a940be1339bfed35721daaccaa65d15f00c75d9) * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java * (add) hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandlerWithReadOnly.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftHBaseServiceHandler.java HBASE-18875 Thrift server supports read-only mode - addendum fixes (tedyu: rev 1af5bf110544e369c8da50404862a6e527008e62) * (edit) hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandlerWithReadOnly.java > Thrift server supports read-only mode > - > > Key: HBASE-18875 > URL: https://issues.apache.org/jira/browse/HBASE-18875 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-4 > > Attachments: 18875.add, HBASE-18875.branch-1.v1.patch, > HBASE-18875.patch, HBASE-18875.v1.patch > > > Provide option for thrift server to support read-only mode. > To start the thrift server, use the -ro option or set hbase.thrift.readonly > to true. > false: Both read and write request are permitted.(all methods) > true : Only the read request is permitted. (only get/scan method) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18298) RegionServerServices Interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180293#comment-16180293 ] Anoop Sam John commented on HBASE-18298: Latest patch with rebase and fixing last set of comments from Stack.. Will commit once QA is ok. Thanks all for the reviews. > RegionServerServices Interface cleanup for CP expose > > > Key: HBASE-18298 > URL: https://issues.apache.org/jira/browse/HBASE-18298 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18298.patch, HBASE-18298_V2.patch, > HBASE-18298_V3.patch, HBASE-18298_V4.patch, HBASE-18298_V5.patch, > HBASE-18298_V6.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18298) RegionServerServices Interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-18298: --- Attachment: HBASE-18298_V6.patch > RegionServerServices Interface cleanup for CP expose > > > Key: HBASE-18298 > URL: https://issues.apache.org/jira/browse/HBASE-18298 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18298.patch, HBASE-18298_V2.patch, > HBASE-18298_V3.patch, HBASE-18298_V4.patch, HBASE-18298_V5.patch, > HBASE-18298_V6.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18490) Modifying a table descriptor to enable replicas does not create replica regions
[ https://issues.apache.org/jira/browse/HBASE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180283#comment-16180283 ] huaxiang sun commented on HBASE-18490: -- Thanks Ram. I will look at the patch tomorrow. > Modifying a table descriptor to enable replicas does not create replica > regions > --- > > Key: HBASE-18490 > URL: https://issues.apache.org/jira/browse/HBASE-18490 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0-alpha-1 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-18490_master_1.patch, HBASE-18490_master.patch, > TestRegionReplicasWithRestartScenarios.java > > > After creating a table, if we try to modify the table to enable region > replication, the new Htable Descriptor is not taken into account and the > table is enabled again with default single region. > Ping [~enis], [~tedyu], [~devaraj]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17732:
-
Attachment: HBASE-17732.master.011.patch
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch, HBASE-17732.master.011.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18649) Deprecate KV Usage in MR to move to Cells in 3.0
[ https://issues.apache.org/jira/browse/HBASE-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180269#comment-16180269 ] ramkrishna.s.vasudevan commented on HBASE-18649: bq.So, a MR job that is from branch-1 going against a branch-2 cluster will fail? I think that violates our attempt at making it so a branch-1 client can work against a branch-2 cluster? Am still not sure what will be the client code looking like in this case. For eg. if there is a custom MR job written that tries to read a set of Hfiles and then dump them to another table. Ideally the custom MR would have used the KeyValueMapper/Reducer and in the client code they would have set KeyValue as the output type. Then it should work. No problem. (because we are still having the KV based mapper and reducer in the code base). But if a client (custom MR job) is using HBase's Import tool and the output of the reducer it expects KeyValue then it will fail. But one arguement is that KeyValue is actually @Private (but the counter arg is that these KVMappers and Reducers) should not have been exposed with KeyValue then :). Are you good with the above plan of including all the KV based mappers/reducers in the branch-2 code with @deprecated tags over them? > Deprecate KV Usage in MR to move to Cells in 3.0 > > > Key: HBASE-18649 > URL: https://issues.apache.org/jira/browse/HBASE-18649 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 2.0.0-alpha-2 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 3.0.0, 2.1.0, 2.0.0-alpha-4 > > Attachments: HBASE-18649_branch-2.patch, HBASE-18649_master_2.patch, > HBASE-18649_master_3.patch, HBASE-18649_master_5.patch, > HBASE-18649_master_6.patch, HBASE-18649_master.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18874) HMaster abort message will be skipped if Throwable is passed null
[
https://issues.apache.org/jira/browse/HBASE-18874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-18874:
-
Environment: (was: In HMaster class, we are logging abort message only
in case when Throwable is not null,
{noformat}
if (t != null) LOG.fatal(msg, t);
{noformat}
We will miss the abort message in this case.)
> HMaster abort message will be skipped if Throwable is passed null
> -
>
> Key: HBASE-18874
> URL: https://issues.apache.org/jira/browse/HBASE-18874
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 3.0.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18874) HMaster abort message will be skipped if Throwable is passed null
[
https://issues.apache.org/jira/browse/HBASE-18874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-18874:
-
Description:
In HMaster class, we are logging abort message only in case when Throwable is
not null,
{noformat}
if (t != null) LOG.fatal(msg, t);
{noformat}
We will miss the abort message in this case.
> HMaster abort message will be skipped if Throwable is passed null
> -
>
> Key: HBASE-18874
> URL: https://issues.apache.org/jira/browse/HBASE-18874
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 3.0.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
>
> In HMaster class, we are logging abort message only in case when Throwable is
> not null,
> {noformat}
> if (t != null) LOG.fatal(msg, t);
> {noformat}
> We will miss the abort message in this case.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-16769: --- Release Note: Signature of below methods in MasterObserver changed and instead of org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotDescription param, we will be passing org.apache.hadoop.hbase.client.SnapshotDescription preListSnapshot postListSnapshot preSnapshot postSnapshot preCloneSnapshot postCloneSnapshot preRestoreSnapshot postRestoreSnapshot preDeleteSnapshot postDeleteSnapshot Also changed signature of RegionServerObserver#preReplicateLogEntries and preReplicateLogEntries by removing params List, org.apache.hadoop.hbase.CellScanner > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch, > HBASE-16769_V4.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-16769: --- Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed Status: Resolved (was: Patch Available) > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch, > HBASE-16769_V4.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18762) Canary sink type cast error
[ https://issues.apache.org/jira/browse/HBASE-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180258#comment-16180258 ] Hudson commented on HBASE-18762: FAILURE: Integrated in Jenkins build HBase-1.5 #76 (See [https://builds.apache.org/job/HBase-1.5/76/]) HBASE-18762 Canary sink type cast error (apurtell: rev 414e0ca2927c83cb810fbbb89ea06741d5c79a6a) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java > Canary sink type cast error > --- > > Key: HBASE-18762 > URL: https://issues.apache.org/jira/browse/HBASE-18762 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18762.001.patch > > > When running the main method of Canary.java, we see the following error: > Exception in thread "main" java.lang.ClassCastException: > org.apache.hadoop.hbase.tool.Canary$RegionServerStdOutSink cannot be cast to > org.apache.hadoop.hbase.tool.Canary$RegionStdOutSink > at org.apache.hadoop.hbase.tool.Canary.newMonitor(Canary.java:911) > at org.apache.hadoop.hbase.tool.Canary.run(Canary.java:796) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.hbase.tool.Canary.main(Canary.java:1571) > This happens because we typecast the sink depending on the mode (zookeeper > mode/region server mode) that Canary is configured in. In case no mode is > specified, we typecast the sink into _RegionStdOutSink_. In general, it is > possible to provide inconsistent mode and sink types while running Canary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18830) TestCanaryTool does not check Canary monitor's error code
[ https://issues.apache.org/jira/browse/HBASE-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180259#comment-16180259 ] Hudson commented on HBASE-18830: FAILURE: Integrated in Jenkins build HBase-1.5 #76 (See [https://builds.apache.org/job/HBase-1.5/76/]) HBASE-18830 TestCanaryTool does not check Canary monitor's error code (apurtell: rev 9d68535e8ecf58c879f880c227233e408c0fd32f) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java > TestCanaryTool does not check Canary monitor's error code > - > > Key: HBASE-18830 > URL: https://issues.apache.org/jira/browse/HBASE-18830 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18830.001.patch > > > None of the tests inside TestCanaryTool check Canary monitor's error code. > Thus, it is possible that the monitor has registered an error and yet the > tests pass. We should check the value returned by the _ToolRunner.run()_ > method inside each unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18875) Thrift server supports read-only mode
[ https://issues.apache.org/jira/browse/HBASE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180257#comment-16180257 ] Hudson commented on HBASE-18875: FAILURE: Integrated in Jenkins build HBase-1.5 #76 (See [https://builds.apache.org/job/HBase-1.5/76/]) HBASE-18875 Thrift server supports read-only mode (tedyu: rev 9485835e555d4d8e317e10af780a3cd520931053) * (add) hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandlerWithReadOnly.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftHBaseServiceHandler.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java HBASE-18875 Thrift server supports read-only mode - addendum fixes (tedyu: rev d6d62a546724672acaff71b5acc4c687a4df119e) * (edit) hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandlerWithReadOnly.java > Thrift server supports read-only mode > - > > Key: HBASE-18875 > URL: https://issues.apache.org/jira/browse/HBASE-18875 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-4 > > Attachments: 18875.add, HBASE-18875.branch-1.v1.patch, > HBASE-18875.patch, HBASE-18875.v1.patch > > > Provide option for thrift server to support read-only mode. > To start the thrift server, use the -ro option or set hbase.thrift.readonly > to true. > false: Both read and write request are permitted.(all methods) > true : Only the read request is permitted. (only get/scan method) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18490) Modifying a table descriptor to enable replicas does not create replica regions
[ https://issues.apache.org/jira/browse/HBASE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180255#comment-16180255 ] ramkrishna.s.vasudevan commented on HBASE-18490: [~enis], [~huaxiang] Got a green QA run. Want to have a look at the latest patch - the one in RB. > Modifying a table descriptor to enable replicas does not create replica > regions > --- > > Key: HBASE-18490 > URL: https://issues.apache.org/jira/browse/HBASE-18490 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0-alpha-1 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-18490_master_1.patch, HBASE-18490_master.patch, > TestRegionReplicasWithRestartScenarios.java > > > After creating a table, if we try to modify the table to enable region > replication, the new Htable Descriptor is not taken into account and the > table is enabled again with default single region. > Ping [~enis], [~tedyu], [~devaraj]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18298) RegionServerServices Interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180254#comment-16180254 ] ramkrishna.s.vasudevan commented on HBASE-18298: Had a quick glance at the patch. LGTM. If Stack's and Duo's concerns are addressed +1 from me. (if already addressed fine then). > RegionServerServices Interface cleanup for CP expose > > > Key: HBASE-18298 > URL: https://issues.apache.org/jira/browse/HBASE-18298 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18298.patch, HBASE-18298_V2.patch, > HBASE-18298_V3.patch, HBASE-18298_V4.patch, HBASE-18298_V5.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17732:
--
Fix Version/s: 2.0.0-alpha-4
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17732:
--
Priority: Critical (was: Major)
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18839) Region#getRegionInfo should return RegionInfo instead of HRegionInfo
[ https://issues.apache.org/jira/browse/HBASE-18839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180250#comment-16180250 ] Chia-Ping Tsai commented on HBASE-18839: Will upload it later...Busy these day.:( > Region#getRegionInfo should return RegionInfo instead of HRegionInfo > > > Key: HBASE-18839 > URL: https://issues.apache.org/jira/browse/HBASE-18839 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0-alpha-4 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-15867) Move HBase replication tracking from ZooKeeper to HBase
[ https://issues.apache.org/jira/browse/HBASE-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180239#comment-16180239 ] stack edited comment on HBASE-15867 at 9/26/17 4:42 AM: Be our guest [~openinx] Shout if we can help (Hopefully this ok w/ you [~ashu210890]) was (Author: stack): Be our guest [~openinx] Shout if we can help. > Move HBase replication tracking from ZooKeeper to HBase > --- > > Key: HBASE-15867 > URL: https://issues.apache.org/jira/browse/HBASE-15867 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Joseph >Assignee: Zheng Hu > > Move the WAL file and offset tracking out of ZooKeeper and into an HBase > table called hbase:replication. > The largest three new changes will be two classes ReplicationTableBase, > TableBasedReplicationQueues, and TableBasedReplicationQueuesClient. As of now > ReplicationPeers and HFileRef's tracking will not be implemented. Subtasks > have been filed for these two jobs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15867) Move HBase replication tracking from ZooKeeper to HBase
[ https://issues.apache.org/jira/browse/HBASE-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180239#comment-16180239 ] stack commented on HBASE-15867: --- Be our guest [~openinx] Shout if we can help. > Move HBase replication tracking from ZooKeeper to HBase > --- > > Key: HBASE-15867 > URL: https://issues.apache.org/jira/browse/HBASE-15867 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Joseph > > Move the WAL file and offset tracking out of ZooKeeper and into an HBase > table called hbase:replication. > The largest three new changes will be two classes ReplicationTableBase, > TableBasedReplicationQueues, and TableBasedReplicationQueuesClient. As of now > ReplicationPeers and HFileRef's tracking will not be implemented. Subtasks > have been filed for these two jobs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-15867) Move HBase replication tracking from ZooKeeper to HBase
[ https://issues.apache.org/jira/browse/HBASE-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reassigned HBASE-15867: - Assignee: Zheng Hu > Move HBase replication tracking from ZooKeeper to HBase > --- > > Key: HBASE-15867 > URL: https://issues.apache.org/jira/browse/HBASE-15867 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Joseph >Assignee: Zheng Hu > > Move the WAL file and offset tracking out of ZooKeeper and into an HBase > table called hbase:replication. > The largest three new changes will be two classes ReplicationTableBase, > TableBasedReplicationQueues, and TableBasedReplicationQueuesClient. As of now > ReplicationPeers and HFileRef's tracking will not be implemented. Subtasks > have been filed for these two jobs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18826) Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface
[ https://issues.apache.org/jira/browse/HBASE-18826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180235#comment-16180235 ] stack commented on HBASE-18826: --- Patch looks good to me. +1 That refactor with getStoreFileAgeStream looks good but we sure it returns the right answer? Is there test coverage for it? The purge of deprecated stuff is great. +1 > Use HStore instead of Store in our own code base and remove unnecessary > methods in Store interface > -- > > Key: HBASE-18826 > URL: https://issues.apache.org/jira/browse/HBASE-18826 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18826.patch, HBASE-18826-v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16290) Dump summary of callQueue content; can help debugging
[
https://issues.apache.org/jira/browse/HBASE-16290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180231#comment-16180231
]
Hadoop QA commented on HBASE-16290:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
22s{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 2m
32s{color} | {color:red} patch has 10 errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 22s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 49s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
10s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 74m 18s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.ipc.TestFifoRpcScheduler |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-16290 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888993/HBASE-16290.master.003.patch
|
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7d8be7b6c92c 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 4b208eb |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/8784/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/8784/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180229#comment-16180229 ] Hudson commented on HBASE-18731: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #959 (See [https://builds.apache.org/job/HBase-1.2-IT/959/]) HBASE-18731 [compat 1-2] Mark protected methods of QuotaSettings that (stack: rev fa49bb4db9431879ec71e39ddc099b7bf123fb4e) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaSettings.java > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180222#comment-16180222 ] Hudson commented on HBASE-18731: FAILURE: Integrated in Jenkins build HBase-1.2-JDK7 #226 (See [https://builds.apache.org/job/HBase-1.2-JDK7/226/]) HBASE-18731 [compat 1-2] Mark protected methods of QuotaSettings that (stack: rev fa49bb4db9431879ec71e39ddc099b7bf123fb4e) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaSettings.java > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18859) Purge PB from BulkLoadObserver
[ https://issues.apache.org/jira/browse/HBASE-18859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18859: -- Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed Release Note: No longer pass the protobuf request to prePrepareBulkLoad and preCleanupBulkLoad in BulkLoadObserver as part of our effort to purge protobuf from our Coprocessor API Interface (if you need to read the Table and RegionInfo, pull it from the passed in RegionCoprocessorEnvironment ObserverContext). Status: Resolved (was: Patch Available) Thanks for the reviews up on rb lads (nice suggestion [~ram_krish] on getting region and table from env). > Purge PB from BulkLoadObserver > -- > > Key: HBASE-18859 > URL: https://issues.apache.org/jira/browse/HBASE-18859 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18859.master.001.patch, > HBASE-18859.master.002.patch, HBASE-18859.master.002.patch > > > Noticed by [~appy] We expose PBs in this CP. Pass POJOs. This is like > [~anoop.hbase] and [~elserj] pb purging that is going on contemporaneously. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180221#comment-16180221 ] Hudson commented on HBASE-18731: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #215 (See [https://builds.apache.org/job/HBase-1.3-IT/215/]) HBASE-18731 [compat 1-2] Mark protected methods of QuotaSettings that (stack: rev b41dbd37e1360e2dfaa9ff1768a5fa87aea82558) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaSettings.java > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18870) Hbase Backup should set the details to MR jobs
[ https://issues.apache.org/jira/browse/HBASE-18870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180212#comment-16180212 ] Vishal Khandelwal edited comment on HBASE-18870 at 9/26/17 4:15 AM: I can provide the fix. I will assign to myself was (Author: vishk): I can provide the fix will assign to myself > Hbase Backup should set the details to MR jobs > -- > > Key: HBASE-18870 > URL: https://issues.apache.org/jira/browse/HBASE-18870 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Minor > Labels: backup > > For Incremental and Full Backups job names should be set correctly. It would > make the debug and monitoring better. > Current : > incremental -> WALPlayer_1506332227619 > ExportSnapshot-snapshot_1506323760362_default_t1 > Proposed : > incremental -> Backup_Incremental__ > Full -> Backups_Full__ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180218#comment-16180218 ] Hudson commented on HBASE-18731: FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #286 (See [https://builds.apache.org/job/HBase-1.3-JDK7/286/]) HBASE-18731 [compat 1-2] Mark protected methods of QuotaSettings that (stack: rev b41dbd37e1360e2dfaa9ff1768a5fa87aea82558) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaSettings.java > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18871) Counters for Backup Job
[ https://issues.apache.org/jira/browse/HBASE-18871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180215#comment-16180215 ] Vishal Khandelwal commented on HBASE-18871: --- [~vladmir] : How do we check the counter in backinfo ? Also with our current backup experience counters like how many rows backups up each row was very helpful. This should be a easy fix but since backup is reusing WALPlayer and ExportSnapshot existing code. I am not sure how to add the custom counters there ? I can work on this if required . > Counters for Backup Job > --- > > Key: HBASE-18871 > URL: https://issues.apache.org/jira/browse/HBASE-18871 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Vishal Khandelwal > Labels: backup > > There should be Map Reduce counter for backup jobs which would be helpful to > analysis the perf and data for backup > #1 Number of Rows backed up > #2 Number of WAL Files processed > #3 Number of HFiles created > etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180213#comment-16180213 ] Josh Elser commented on HBASE-16769: bq. I can not close this issue as the subtask is still open. Can we move that task out? Or should we wait for the sub issue closure? I moved it out of a sub-issue. Should not be an alpha4 blocker and shouldn't block this issue from being resolved. > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch, > HBASE-16769_V4.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18870) Hbase Backup should set the details to MR jobs
[ https://issues.apache.org/jira/browse/HBASE-18870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Khandelwal reassigned HBASE-18870: - Assignee: Vishal Khandelwal > Hbase Backup should set the details to MR jobs > -- > > Key: HBASE-18870 > URL: https://issues.apache.org/jira/browse/HBASE-18870 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Minor > Labels: backup > > For Incremental and Full Backups job names should be set correctly. It would > make the debug and monitoring better. > Current : > incremental -> WALPlayer_1506332227619 > ExportSnapshot-snapshot_1506323760362_default_t1 > Proposed : > incremental -> Backup_Incremental__ > Full -> Backups_Full__ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18870) Hbase Backup should set the details to MR jobs
[ https://issues.apache.org/jira/browse/HBASE-18870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180212#comment-16180212 ] Vishal Khandelwal commented on HBASE-18870: --- I can provide the fix will assign to myself > Hbase Backup should set the details to MR jobs > -- > > Key: HBASE-18870 > URL: https://issues.apache.org/jira/browse/HBASE-18870 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Vishal Khandelwal >Priority: Minor > Labels: backup > > For Incremental and Full Backups job names should be set correctly. It would > make the debug and monitoring better. > Current : > incremental -> WALPlayer_1506332227619 > ExportSnapshot-snapshot_1506323760362_default_t1 > Proposed : > incremental -> Backup_Incremental__ > Full -> Backups_Full__ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18731: -- Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed Release Note: The following methods in QuotaSettings were annotated InterfaceAudience.Private; they are for internal use only in hbase-2.0.0 buildSetQuotaRequestProto(final QuotaSettings settings) setupSetQuotaRequest(SetQuotaRequest.Builder builder) Status: Resolved (was: Patch Available) > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18873) Hide protobufs in GlobalQuotaSettings
[ https://issues.apache.org/jira/browse/HBASE-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-18873: --- Issue Type: Improvement (was: Sub-task) Parent: (was: HBASE-16769) > Hide protobufs in GlobalQuotaSettings > - > > Key: HBASE-18873 > URL: https://issues.apache.org/jira/browse/HBASE-18873 > Project: HBase > Issue Type: Improvement >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Critical > Fix For: 2.0.0-alpha-4 > > > HBASE-18807 cleaned up direct protobuf use in the Coprocessor APIs for > quota-related functions. However, one new POJO introduced to hide these > protocol buffers still exposes PBs via some methods. > We should try to hide those as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180208#comment-16180208 ] Anoop Sam John commented on HBASE-16769: Pushed to master and branch-2. Thanks all for the reviews. [~elserj] I can not close this issue as the subtask is still open. Can we move that task out? Or should we wait for the sub issue closure? > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch, > HBASE-16769_V4.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18731: -- Fix Version/s: (was: 3.0.0) > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180209#comment-16180209 ] stack commented on HBASE-18731: --- I pushed the branch-1 deprecation patch to all branches from branch-1.1 on. Thanks [~busbey] > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18652: -- Resolution: Fixed Fix Version/s: (was: 1.5.0) Status: Resolved (was: Patch Available) Pushed the v3 branch-1 patch to branch-1 and branch-1.4 after checking that the failed test passes locally. Also pushed the addendum. Thanks for the nice patch [~biju74] (pardon the delay). > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Fix For: 1.4.0, 2.0.0-alpha-4 > > Attachments: 18652-branch-1-V3.0.PATCH, HBASE-18652-addendum.patch, > HBASE-18652-BRANCH-1.PATCH, HBASE-18652-branch-1-v1.0.patch, > HBASE-18652-branch-1-V2.0.PATCH, HBASE-18652-branch-1-V3.0.PATCH, > HBASE-18652-branch-1-V3.0.PATCH, HBASE-18652.PATCH, HBASE-18652-V1.0.PATCH, > HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18786) FileNotFoundException should not be silently handled for primary region replicas
[ https://issues.apache.org/jira/browse/HBASE-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180202#comment-16180202 ] Hudson commented on HBASE-18786: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3780 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3780/]) Amend HBASE-18786 FileNotFoundException should not be silently handled (apurtell: rev b145286f36d1cdd2cec72ec694bbd3d8b62e3765) * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCorruptedRegionStoreFile.java > FileNotFoundException should not be silently handled for primary region > replicas > > > Key: HBASE-18786 > URL: https://issues.apache.org/jira/browse/HBASE-18786 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Reporter: Ashu Pachauri >Assignee: Andrew Purtell > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18786-branch-1.patch, HBASE-18786-branch-1.patch, > HBASE-18786.patch, HBASE-18786.patch > > > This is a follow up for HBASE-18186. > FileNotFoundException while scanning from a primary region replica can be > indicative of a more severe problem. Handling them silently can cause many > underlying issues go undetected. We should either > 1. Hard fail the regionserver if there is a FNFE on a primary region replica, > OR > 2. Report these exceptions as some region / server level metric so that these > can be proactively investigated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18875) Thrift server supports read-only mode
[ https://issues.apache.org/jira/browse/HBASE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180201#comment-16180201 ] Hudson commented on HBASE-18875: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3780 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3780/]) HBASE-18875 Thrift server supports read-only mode (tedyu: rev cfb6a54f69b1d847142afab56bdc1504638f118d) * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftHBaseServiceHandler.java * (add) hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandlerWithReadOnly.java > Thrift server supports read-only mode > - > > Key: HBASE-18875 > URL: https://issues.apache.org/jira/browse/HBASE-18875 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-4 > > Attachments: 18875.add, HBASE-18875.branch-1.v1.patch, > HBASE-18875.patch, HBASE-18875.v1.patch > > > Provide option for thrift server to support read-only mode. > To start the thrift server, use the -ro option or set hbase.thrift.readonly > to true. > false: Both read and write request are permitted.(all methods) > true : Only the read request is permitted. (only get/scan method) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15867) Move HBase replication tracking from ZooKeeper to HBase
[ https://issues.apache.org/jira/browse/HBASE-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180199#comment-16180199 ] Zheng Hu commented on HBASE-15867: -- [~stack], May I have this issue ? > Move HBase replication tracking from ZooKeeper to HBase > --- > > Key: HBASE-15867 > URL: https://issues.apache.org/jira/browse/HBASE-15867 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Joseph > > Move the WAL file and offset tracking out of ZooKeeper and into an HBase > table called hbase:replication. > The largest three new changes will be two classes ReplicationTableBase, > TableBasedReplicationQueues, and TableBasedReplicationQueuesClient. As of now > ReplicationPeers and HFileRef's tracking will not be implemented. Subtasks > have been filed for these two jobs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18879) Hbase FilterList cause KeyOnlyFilter not work
ZHA_Moonlight created HBASE-18879:
-
Summary: Hbase FilterList cause KeyOnlyFilter not work
Key: HBASE-18879
URL: https://issues.apache.org/jira/browse/HBASE-18879
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 1.2.4
Environment: OS： Red Hat 4.4.7-11
Hadoop: 2.6.4
Hbase: 1.2.4
Reporter: ZHA_Moonlight
when use FilterList and KeyOnlyFilter together, if we put KeyOnlyFilter before
FilterList, the KeyOnlyFilter may not work, means it will also grab the cell
values:
{code:java}
List filters = new ArrayList();
Filter filter1 = new SingleColumnValueFilter(Bytes.toBytes("cf"),
Bytes.toBytes("column1"),
CompareOp.EQUAL, Bytes.toBytes("value1"));
Filter filter2 = new SingleColumnValueFilter(Bytes.toBytes("cf"),
Bytes.toBytes("column1"),
CompareOp.EQUAL, Bytes.toBytes("value2"));
filters.add(filter1);
filters.add(filter2);
FilterList filterListAll = new FilterList(Operator.MUST_PASS_ALL,
new KeyOnlyFilter(),
new FilterList(Operator.MUST_PASS_ONE, filters));
{code}
use the above code as filter to scan a table, it will return the cells with
value instead of only return the key, if we put KeyOnlyFilter after FilterList
as following, it works well.
{code:java}
FilterList filterListAll = new FilterList(Operator.MUST_PASS_ALL,
new FilterList(Operator.MUST_PASS_ONE, filters),
new KeyOnlyFilter());
{code}
the cause should due to the following code at hbase-client FilterList.java
{code:java}
@Override
@edu.umd.cs.findbugs.annotations.SuppressWarnings(value="SF_SWITCH_FALLTHROUGH",
justification="Intentional")
public ReturnCode filterKeyValue(Cell v) throws IOException {
this.referenceKV = v;
// Accumulates successive transformation of every filter that includes the
Cell:
Cell transformed = v;
ReturnCode rc = operator == Operator.MUST_PASS_ONE?
ReturnCode.SKIP: ReturnCode.INCLUDE;
int listize = filters.size();
for (int i = 0; i < listize; i++) {
Filter filter = filters.get(i);
if (operator == Operator.MUST_PASS_ALL) {
if (filter.filterAllRemaining()) {
return ReturnCode.NEXT_ROW;
}
{color:red}ReturnCode code = filter.filterKeyValue(v);
{color}switch (code) {
// Override INCLUDE and continue to evaluate.
case INCLUDE_AND_NEXT_COL:
rc = ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs
SF_SWITCH_FALLTHROUGH
case INCLUDE:
{color:red} transformed = filter.transformCell(transformed);{color}
continue;
case SEEK_NEXT_USING_HINT:
seekHintFilter = filter;
return code;
default:
return code;
}
}
{code}
notice the {color:red} label line, first line is a recursive invocation, it
will assign a Cell results to the FilterList.transformedKV(we call it A), the
results is from the FilterList with 2 SingleColumnValueFilter, so A with
contains the cell value, while the second line with return A to the var
transformed.
back to the following loop, we can see the FilterList return results is var
"transformed " which will override in each loop, so the value is determined by
the last filter, so the order of KeyOnlyFilter will impact the results.
{code:java}
Cell transformed = v;
ReturnCode rc = operator == Operator.MUST_PASS_ONE?
ReturnCode.SKIP: ReturnCode.INCLUDE;
int listize = filters.size();
for (int i = 0; i < listize; i++) {
Filter filter = filters.get(i);
if (operator == Operator.MUST_PASS_ALL) {
if (filter.filterAllRemaining()) {
return ReturnCode.NEXT_ROW;
}
ReturnCode code = filter.filterKeyValue(v);
switch (code) {
// Override INCLUDE and continue to evaluate.
case INCLUDE_AND_NEXT_COL:
rc = ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs
SF_SWITCH_FALLTHROUGH
case INCLUDE:
transformed = filter.transformCell(transformed);
continue;
case SEEK_NEXT_USING_HINT:
seekHintFilter = filter;
return code;
default:
return code;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18411) Dividing FiterList into two separate sub-classes: FilterListWithOR , FilterListWithAND
[ https://issues.apache.org/jira/browse/HBASE-18411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180193#comment-16180193 ] Duo Zhang commented on HBASE-18411: --- I've already pushed HBASE-18160 to master and branch-2. You can upload the patch now [~openinx]. > Dividing FiterList into two separate sub-classes: FilterListWithOR , > FilterListWithAND > > > Key: HBASE-18411 > URL: https://issues.apache.org/jira/browse/HBASE-18411 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Peter Somogyi > Attachments: HBASE-18411.v1.patch, HBASE-18411.v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18160) Fix incorrect logic in FilterList.filterKeyValue
[
https://issues.apache.org/jira/browse/HBASE-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180190#comment-16180190
]
Duo Zhang commented on HBASE-18160:
---
Pushed to master and branch-2.
Wait for the pre commit result for branch-1.4-.
> Fix incorrect logic in FilterList.filterKeyValue
> -
>
> Key: HBASE-18160
> URL: https://issues.apache.org/jira/browse/HBASE-18160
> Project: HBase
> Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
> Fix For: 1.4.0, 1.2.6, 1.3.2, 2.0.0-alpha-3
>
> Attachments: filter-and-map.txt, filter-or-map.txt,
> HBASE-18160.branch-1.3.v1.patch, HBASE-18160.branch-1.3.v1.patch,
> HBASE-18160.branch-1.4.v1.patch, HBASE-18160.branch-1.4.v1.patch,
> HBASE-18160.v1.patch, HBASE-18160.v2.patch, HBASE-18160.v2.patch,
> HBASE-18160.v3.patch, HBASE-18160.v4.patch, HBASE-18160.v5.patch,
> HBASE-18160.v6.patch, HBASE-18160.v6.patch, HBASE-18160.v7.patch,
> HBASE-18160.v8.patch
>
>
> As HBASE-17678 said, there are two problems in FilterList.filterKeyValue
> implementation:
> 1. FilterList did not consider INCLUDE_AND_SEEK_NEXT_ROW case( seems like
> INCLUDE_AND_SEEK_NEXT_ROW is a newly added case, and the dev forgot to
> consider FilterList), So if a user use INCLUDE_AND_SEEK_NEXT_ROW in his own
> Filter and wrapped by a FilterList, it'll throw an
> IllegalStateException("Received code is not valid.").
> 2. For FilterList with MUST_PASS_ONE, if filter-A in filter list return
> INCLUDE and filter-B in filter list return INCLUDE_AND_NEXT_COL, the
> FilterList will return INCLUDE_AND_NEXT_COL finally. According to the
> mininal step rule , It's incorrect. (filter list with MUST_PASS_ONE choose
> the mininal step among filters in filter list. Let's call it: The Mininal
> Step Rule).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18298) RegionServerServices Interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180189#comment-16180189 ] Anoop Sam John commented on HBASE-18298: hmm... Agree Stack. In the ZK eg: which is changed in this patch, the zk connection been created in start only. May be we need an eg: which will create connection (as u mentioned) for a table object create and do puts to that table on every mutate (some thing like this). > RegionServerServices Interface cleanup for CP expose > > > Key: HBASE-18298 > URL: https://issues.apache.org/jira/browse/HBASE-18298 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18298.patch, HBASE-18298_V2.patch, > HBASE-18298_V3.patch, HBASE-18298_V4.patch, HBASE-18298_V5.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18649) Deprecate KV Usage in MR to move to Cells in 3.0
[ https://issues.apache.org/jira/browse/HBASE-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180186#comment-16180186 ] stack commented on HBASE-18649: --- bq. The branch-2 code will be made to work with MapReduceCell only and we can mark things as incompatible change. So, a MR job that is from branch-1 going against a branch-2 cluster will fail? I think that violates our attempt at making it so a branch-1 client can work against a branch-2 cluster? What you think [~ram_krish] Does MR have to be incompatible? But it seems that this is case only for some types of custom MR jobs and that majority will keep working? What you think? > Deprecate KV Usage in MR to move to Cells in 3.0 > > > Key: HBASE-18649 > URL: https://issues.apache.org/jira/browse/HBASE-18649 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 2.0.0-alpha-2 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 3.0.0, 2.1.0, 2.0.0-alpha-4 > > Attachments: HBASE-18649_branch-2.patch, HBASE-18649_master_2.patch, > HBASE-18649_master_3.patch, HBASE-18649_master_5.patch, > HBASE-18649_master_6.patch, HBASE-18649_master.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18298) RegionServerServices Interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180183#comment-16180183 ] stack commented on HBASE-18298: --- I saw an interesting case today where a CP was being used to keep up a Secondary Index. They were getting the Table instance from CoprocessorEnvironment which turned into the creation of a new Connection every time they wanted to put a few edits to the secondary index table. It was really heavyweight though the 'ordained' way of talking to another table. If they do their own Connection and connection to ZK, it should be at start of the CP and keep it around (cache meta locations, etc.). We should write this up or do an example. Yeah, if they use the RS Connection, could bring the RS down so probably appropriate we not let them use the RS internal Connection; it aligns w/ the principals being applied to the CP refactor. > RegionServerServices Interface cleanup for CP expose > > > Key: HBASE-18298 > URL: https://issues.apache.org/jira/browse/HBASE-18298 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18298.patch, HBASE-18298_V2.patch, > HBASE-18298_V3.patch, HBASE-18298_V4.patch, HBASE-18298_V5.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18875) Thrift server supports read-only mode
[ https://issues.apache.org/jira/browse/HBASE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180181#comment-16180181 ] Hudson commented on HBASE-18875: FAILURE: Integrated in Jenkins build HBase-2.0 #579 (See [https://builds.apache.org/job/HBase-2.0/579/]) HBASE-18875 Thrift server supports read-only mode (tedyu: rev bb81e9f3ca669bc17b20e3157f2728309b4a3f3a) * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftHBaseServiceHandler.java * (add) hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandlerWithReadOnly.java * (edit) hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java > Thrift server supports read-only mode > - > > Key: HBASE-18875 > URL: https://issues.apache.org/jira/browse/HBASE-18875 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-4 > > Attachments: 18875.add, HBASE-18875.branch-1.v1.patch, > HBASE-18875.patch, HBASE-18875.v1.patch > > > Provide option for thrift server to support read-only mode. > To start the thrift server, use the -ro option or set hbase.thrift.readonly > to true. > false: Both read and write request are permitted.(all methods) > true : Only the read request is permitted. (only get/scan method) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18786) FileNotFoundException should not be silently handled for primary region replicas
[ https://issues.apache.org/jira/browse/HBASE-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180182#comment-16180182 ] Hudson commented on HBASE-18786: FAILURE: Integrated in Jenkins build HBase-2.0 #579 (See [https://builds.apache.org/job/HBase-2.0/579/]) Amend HBASE-18786 FileNotFoundException should not be silently handled (apurtell: rev fa0b0934dbbe0b21fe248add62bd1e385d1d4408) * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCorruptedRegionStoreFile.java > FileNotFoundException should not be silently handled for primary region > replicas > > > Key: HBASE-18786 > URL: https://issues.apache.org/jira/browse/HBASE-18786 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Reporter: Ashu Pachauri >Assignee: Andrew Purtell > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18786-branch-1.patch, HBASE-18786-branch-1.patch, > HBASE-18786.patch, HBASE-18786.patch > > > This is a follow up for HBASE-18186. > FileNotFoundException while scanning from a primary region replica can be > indicative of a more severe problem. Handling them silently can cause many > underlying issues go undetected. We should either > 1. Hard fail the regionserver if there is a FNFE on a primary region replica, > OR > 2. Report these exceptions as some region / server level metric so that these > can be proactively investigated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18815) We need to pass something like CompactionRequest in CP to give user some information about the compaction
[ https://issues.apache.org/jira/browse/HBASE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180173#comment-16180173 ] stack commented on HBASE-18815: --- Thanks [~Apache9] > We need to pass something like CompactionRequest in CP to give user some > information about the compaction > - > > Key: HBASE-18815 > URL: https://issues.apache.org/jira/browse/HBASE-18815 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang > Fix For: 2.0.0-alpha-4 > > > For example, it is a major compaction or minor compaction. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18815) We need to pass something like CompactionRequest in CP to give user some information about the compaction
[ https://issues.apache.org/jira/browse/HBASE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180169#comment-16180169 ] Duo Zhang commented on HBASE-18815: --- [~stack] In HBASE-18453. > We need to pass something like CompactionRequest in CP to give user some > information about the compaction > - > > Key: HBASE-18815 > URL: https://issues.apache.org/jira/browse/HBASE-18815 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang > Fix For: 2.0.0-alpha-4 > > > For example, it is a major compaction or minor compaction. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18770) We should not allow RegionObserver.preBulkLoadHFile to bypass the default behavior
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180167#comment-16180167 ] stack commented on HBASE-18770: --- Thanks lads. I think one of our crew is up for doing this one. We'll be back. > We should not allow RegionObserver.preBulkLoadHFile to bypass the default > behavior > -- > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang > Fix For: 2.0.0-alpha-4 > > > As now we do not allow users to create a StoreFile instance. Users can still > select the files to be bulk loaded by modifying the familyPaths passed in. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18815) We need to pass something like CompactionRequest in CP to give user some information about the compaction
[ https://issues.apache.org/jira/browse/HBASE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180165#comment-16180165 ] stack commented on HBASE-18815: --- Do you know when the change was [~anoop.hbase]? Thanks. > We need to pass something like CompactionRequest in CP to give user some > information about the compaction > - > > Key: HBASE-18815 > URL: https://issues.apache.org/jira/browse/HBASE-18815 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang > Fix For: 2.0.0-alpha-4 > > > For example, it is a major compaction or minor compaction. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[
https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180159#comment-16180159
]
Hadoop QA commented on HBASE-18010:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} HBASE-18010 does not apply to branch-2. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18010 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888992/HBASE-18010-branch-2.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8786/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Connect CellChunkMap to be used for flattening in CompactingMemStore
>
>
> Key: HBASE-18010
> URL: https://issues.apache.org/jira/browse/HBASE-18010
> Project: HBase
> Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>Assignee: Anastasia Braginsky
> Fix For: 2.0.0, 3.0.0
>
> Attachments: HBASE-18010-branch-2.patch, HBASE-18010-branch-2.patch,
> HBASE-18010-V04.patch, HBASE-18010-V06.patch, HBASE-18010-V07.patch,
> HBASE-18010-V08.patch, HBASE-18010-V09.patch, HBASE-18010-V10.patch,
> HBASE-18010-V11.patch
>
>
> The CellChunkMap helps to create a new type of ImmutableSegment, where the
> index (CellSet's delegatee) is going to be CellChunkMap. No big cells or
> upserted cells are going to be supported here.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18786) FileNotFoundException should not be silently handled for primary region replicas
[ https://issues.apache.org/jira/browse/HBASE-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180155#comment-16180155 ] Hudson commented on HBASE-18786: FAILURE: Integrated in Jenkins build HBase-1.4 #931 (See [https://builds.apache.org/job/HBase-1.4/931/]) Amend HBASE-18786 FileNotFoundException should not be silently handled (apurtell: rev e6e7ae4aaeab277961ba0f468ef6a5088c18e305) * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCorruptedRegionStoreFile.java > FileNotFoundException should not be silently handled for primary region > replicas > > > Key: HBASE-18786 > URL: https://issues.apache.org/jira/browse/HBASE-18786 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Reporter: Ashu Pachauri >Assignee: Andrew Purtell > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18786-branch-1.patch, HBASE-18786-branch-1.patch, > HBASE-18786.patch, HBASE-18786.patch > > > This is a follow up for HBASE-18186. > FileNotFoundException while scanning from a primary region replica can be > indicative of a more severe problem. Handling them silently can cause many > underlying issues go undetected. We should either > 1. Hard fail the regionserver if there is a FNFE on a primary region replica, > OR > 2. Report these exceptions as some region / server level metric so that these > can be proactively investigated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16290) Dump summary of callQueue content; can help debugging
[ https://issues.apache.org/jira/browse/HBASE-16290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreeram Venkatasubramanian updated HBASE-16290: --- Attachment: HBASE-16290.master.003.patch > Dump summary of callQueue content; can help debugging > - > > Key: HBASE-16290 > URL: https://issues.apache.org/jira/browse/HBASE-16290 > Project: HBase > Issue Type: Bug > Components: Operability >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Sreeram Venkatasubramanian > Labels: beginner > Fix For: 2.0.0 > > Attachments: DebugDump_screenshot.png, HBASE-16290.master.003.patch, > Sample Summary.txt > > > Being able to get a clue what is in a backedup callQueue could give insight > on what is going on on a jacked server. Just needs to summarize count, sizes, > call types. Useful debugging. In a servlet? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[ https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18010: --- Attachment: HBASE-18010-branch-2.patch retry > Connect CellChunkMap to be used for flattening in CompactingMemStore > > > Key: HBASE-18010 > URL: https://issues.apache.org/jira/browse/HBASE-18010 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 2.0.0, 3.0.0 > > Attachments: HBASE-18010-branch-2.patch, HBASE-18010-branch-2.patch, > HBASE-18010-V04.patch, HBASE-18010-V06.patch, HBASE-18010-V07.patch, > HBASE-18010-V08.patch, HBASE-18010-V09.patch, HBASE-18010-V10.patch, > HBASE-18010-V11.patch > > > The CellChunkMap helps to create a new type of ImmutableSegment, where the > index (CellSet's delegatee) is going to be CellChunkMap. No big cells or > upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[ https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18010: --- Status: Patch Available (was: Open) > Connect CellChunkMap to be used for flattening in CompactingMemStore > > > Key: HBASE-18010 > URL: https://issues.apache.org/jira/browse/HBASE-18010 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 2.0.0, 3.0.0 > > Attachments: HBASE-18010-branch-2.patch, HBASE-18010-branch-2.patch, > HBASE-18010-V04.patch, HBASE-18010-V06.patch, HBASE-18010-V07.patch, > HBASE-18010-V08.patch, HBASE-18010-V09.patch, HBASE-18010-V10.patch, > HBASE-18010-V11.patch > > > The CellChunkMap helps to create a new type of ImmutableSegment, where the > index (CellSet's delegatee) is going to be CellChunkMap. No big cells or > upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[ https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18010: --- Status: Open (was: Patch Available) > Connect CellChunkMap to be used for flattening in CompactingMemStore > > > Key: HBASE-18010 > URL: https://issues.apache.org/jira/browse/HBASE-18010 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 2.0.0, 3.0.0 > > Attachments: HBASE-18010-branch-2.patch, HBASE-18010-branch-2.patch, > HBASE-18010-V04.patch, HBASE-18010-V06.patch, HBASE-18010-V07.patch, > HBASE-18010-V08.patch, HBASE-18010-V09.patch, HBASE-18010-V10.patch, > HBASE-18010-V11.patch > > > The CellChunkMap helps to create a new type of ImmutableSegment, where the > index (CellSet's delegatee) is going to be CellChunkMap. No big cells or > upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16290) Dump summary of callQueue content; can help debugging
[ https://issues.apache.org/jira/browse/HBASE-16290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreeram Venkatasubramanian updated HBASE-16290: --- Attachment: (was: HBASE-16290.master.002.patch) > Dump summary of callQueue content; can help debugging > - > > Key: HBASE-16290 > URL: https://issues.apache.org/jira/browse/HBASE-16290 > Project: HBase > Issue Type: Bug > Components: Operability >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Sreeram Venkatasubramanian > Labels: beginner > Fix For: 2.0.0 > > Attachments: DebugDump_screenshot.png, Sample Summary.txt > > > Being able to get a clue what is in a backedup callQueue could give insight > on what is going on on a jacked server. Just needs to summarize count, sizes, > call types. Useful debugging. In a servlet? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16290) Dump summary of callQueue content; can help debugging
[ https://issues.apache.org/jira/browse/HBASE-16290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreeram Venkatasubramanian updated HBASE-16290: --- Attachment: (was: HBASE-16290.master.003.patch) > Dump summary of callQueue content; can help debugging > - > > Key: HBASE-16290 > URL: https://issues.apache.org/jira/browse/HBASE-16290 > Project: HBase > Issue Type: Bug > Components: Operability >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Sreeram Venkatasubramanian > Labels: beginner > Fix For: 2.0.0 > > Attachments: DebugDump_screenshot.png, Sample Summary.txt > > > Being able to get a clue what is in a backedup callQueue could give insight > on what is going on on a jacked server. Just needs to summarize count, sizes, > call types. Useful debugging. In a servlet? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16290) Dump summary of callQueue content; can help debugging
[ https://issues.apache.org/jira/browse/HBASE-16290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreeram Venkatasubramanian updated HBASE-16290: --- Attachment: (was: HBASE-16290.master.001.patch) > Dump summary of callQueue content; can help debugging > - > > Key: HBASE-16290 > URL: https://issues.apache.org/jira/browse/HBASE-16290 > Project: HBase > Issue Type: Bug > Components: Operability >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Sreeram Venkatasubramanian > Labels: beginner > Fix For: 2.0.0 > > Attachments: DebugDump_screenshot.png, HBASE-16290.master.002.patch, > HBASE-16290.master.003.patch, Sample Summary.txt > > > Being able to get a clue what is in a backedup callQueue could give insight > on what is going on on a jacked server. Just needs to summarize count, sizes, > call types. Useful debugging. In a servlet? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18298) RegionServerServices Interface cleanup for CP expose
[
https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180150#comment-16180150
]
Anoop Sam John commented on HBASE-18298:
Sorry [~Apache9].. The RB is not having latest patch. All ur concerns are
addressed in the latest patch attached here.. Let me upload the latest to the
RB also this time. Really sorry for missing that.
bq.The below cast is hackery for now because this class is deprecated and going
away
Ya this is going away and the type casting is fine IMO
{quote}
he below creates a new zk instance?
176 ZooKeeper zk = new ZooKeeper(ensemble, sessionTimeout, null);
177 re.getSharedData().putIfAbsent(zkkey, new ZKWatcher(zk));
It gets closed? You think we don't have to reuse because this an example?
{quote}
I think missed adding the close.. Let me address.. Ya it is just an eg: that
we show.. If the CP user has to use the zk for some reason, we should not
expose our zk instance objects to them.. The ZooKeeperWatcher will have
methods even close() in it.. Any mistake in using this will result in our RS
process of no use. So if they want, let them have own ways of reading what is
the quorum and ports to use and start own zk client.
bq.nit: s/getFromOnlineRegions/getOnlineRegion/ singular.
These all existing methods. Just moved to new interface. All in OnlineRegions
having this 's' at end.
Region getFromOnlineRegions(String encodedRegionName); - Means get a region
from all online regions
bq.And later, should we allow getting reference to zk? Though AccessController
should be coming internal so wouldn't worry about it
As said above, we should not expose our zk client at RS to the CP users. Our
AC and VC needs zk and there is no other way but to do this type casting..
This also suggest that we should be having our AC and VC as core impl not as CP
impl :-)
> RegionServerServices Interface cleanup for CP expose
>
>
> Key: HBASE-18298
> URL: https://issues.apache.org/jira/browse/HBASE-18298
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18298.patch, HBASE-18298_V2.patch,
> HBASE-18298_V3.patch, HBASE-18298_V4.patch, HBASE-18298_V5.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18826) Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface
[ https://issues.apache.org/jira/browse/HBASE-18826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-18826: -- Attachment: HBASE-18826-v1.patch Fix failed UTs. > Use HStore instead of Store in our own code base and remove unnecessary > methods in Store interface > -- > > Key: HBASE-18826 > URL: https://issues.apache.org/jira/browse/HBASE-18826 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18826.patch, HBASE-18826-v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18815) We need to pass something like CompactionRequest in CP to give user some information about the compaction
[ https://issues.apache.org/jira/browse/HBASE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180136#comment-16180136 ] Anoop Sam John commented on HBASE-18815: We used to pass CompactionRequest into CPs and that was having some helpful methods for CPs. Now we pass CompactionLifeCycleTracker instead. WOuld be nice to give ways for CP users to get those info. > We need to pass something like CompactionRequest in CP to give user some > information about the compaction > - > > Key: HBASE-18815 > URL: https://issues.apache.org/jira/browse/HBASE-18815 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang > Fix For: 2.0.0-alpha-4 > > > For example, it is a major compaction or minor compaction. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18770) We should not allow RegionObserver.preBulkLoadHFile to bypass the default behavior
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180133#comment-16180133 ] Anoop Sam John commented on HBASE-18770: We consider the bypass set or not in code as of now.. But as per RSRpcServices#bulkLoadHFile, when bypass is done, we mark the response as bulk load failed. So IMO any way the impl was buggy.. We can remove the bypass support from this hook and doc clearly > We should not allow RegionObserver.preBulkLoadHFile to bypass the default > behavior > -- > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang > Fix For: 2.0.0-alpha-4 > > > As now we do not allow users to create a StoreFile instance. Users can still > select the files to be bulk loaded by modifying the familyPaths passed in. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18786) FileNotFoundException should not be silently handled for primary region replicas
[ https://issues.apache.org/jira/browse/HBASE-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180127#comment-16180127 ] Hudson commented on HBASE-18786: FAILURE: Integrated in Jenkins build HBase-1.5 #75 (See [https://builds.apache.org/job/HBase-1.5/75/]) Amend HBASE-18786 FileNotFoundException should not be silently handled (apurtell: rev d74bdc310085470bf573b07daea8ae03c8007987) * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCorruptedRegionStoreFile.java > FileNotFoundException should not be silently handled for primary region > replicas > > > Key: HBASE-18786 > URL: https://issues.apache.org/jira/browse/HBASE-18786 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Reporter: Ashu Pachauri >Assignee: Andrew Purtell > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18786-branch-1.patch, HBASE-18786-branch-1.patch, > HBASE-18786.patch, HBASE-18786.patch > > > This is a follow up for HBASE-18186. > FileNotFoundException while scanning from a primary region replica can be > indicative of a more severe problem. Handling them silently can cause many > underlying issues go undetected. We should either > 1. Hard fail the regionserver if there is a FNFE on a primary region replica, > OR > 2. Report these exceptions as some region / server level metric so that these > can be proactively investigated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18806) VerifyRep by snapshot need not to restore snapshot for each mapper
[
https://issues.apache.org/jira/browse/HBASE-18806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180121#comment-16180121
]
Zheng Hu commented on HBASE-18806:
--
Hadoop QA again.
> VerifyRep by snapshot need not to restore snapshot for each mapper
> --
>
> Key: HBASE-18806
> URL: https://issues.apache.org/jira/browse/HBASE-18806
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0-alpha-2
>Reporter: Zheng Hu
>Assignee: Zheng Hu
> Attachments: HBASE-18806.v1.patch, HBASE-18806.v2.patch,
> HBASE-18806.v3.patch, HBASE-18806.v3.patch
>
>
> In following method stack, seems like each mapper task will restore the
> snapshot. If we verify replication by a snapshot which has many hfiles,
> then we will take long time to restore snapshot. In our cluster, we took
> ~30min for the snapshot restoring when verify a big table.
> {code}
> Verifier.map
> |> replicatedScanner = new TableSnapshotScanner(...)
> |>
> TableSnapshotScanner.init()
>
> |-> RestoreSnapshotHelper.copySnapshotForScanner
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18806) VerifyRep by snapshot need not to restore snapshot for each mapper
[
https://issues.apache.org/jira/browse/HBASE-18806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-18806:
-
Attachment: HBASE-18806.v3.patch
> VerifyRep by snapshot need not to restore snapshot for each mapper
> --
>
> Key: HBASE-18806
> URL: https://issues.apache.org/jira/browse/HBASE-18806
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0-alpha-2
>Reporter: Zheng Hu
>Assignee: Zheng Hu
> Attachments: HBASE-18806.v1.patch, HBASE-18806.v2.patch,
> HBASE-18806.v3.patch, HBASE-18806.v3.patch
>
>
> In following method stack, seems like each mapper task will restore the
> snapshot. If we verify replication by a snapshot which has many hfiles,
> then we will take long time to restore snapshot. In our cluster, we took
> ~30min for the snapshot restoring when verify a big table.
> {code}
> Verifier.map
> |> replicatedScanner = new TableSnapshotScanner(...)
> |>
> TableSnapshotScanner.init()
>
> |-> RestoreSnapshotHelper.copySnapshotForScanner
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18845) TestReplicationSmallTests fails after HBASE-14004
[ https://issues.apache.org/jira/browse/HBASE-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180115#comment-16180115 ] Duo Zhang commented on HBASE-18845: --- Any other concerns [~tedyu]? Thanks. > TestReplicationSmallTests fails after HBASE-14004 > - > > Key: HBASE-18845 > URL: https://issues.apache.org/jira/browse/HBASE-18845 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0, 2.0.0-alpha-3 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 3.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-18845.patch > > > testEmptyWALRecovery and testVerifyRepJob -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18411) Dividing FiterList into two separate sub-classes: FilterListWithOR , FilterListWithAND
[ https://issues.apache.org/jira/browse/HBASE-18411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-18411: - Attachment: HBASE-18411.v1.patch > Dividing FiterList into two separate sub-classes: FilterListWithOR , > FilterListWithAND > > > Key: HBASE-18411 > URL: https://issues.apache.org/jira/browse/HBASE-18411 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Peter Somogyi > Attachments: HBASE-18411.v1.patch, HBASE-18411.v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18762) Canary sink type cast error
[ https://issues.apache.org/jira/browse/HBASE-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-18762. Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.5.0 1.4.0 3.0.0 2.0.0 Pushed to branch-1.4 and up > Canary sink type cast error > --- > > Key: HBASE-18762 > URL: https://issues.apache.org/jira/browse/HBASE-18762 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18762.001.patch > > > When running the main method of Canary.java, we see the following error: > Exception in thread "main" java.lang.ClassCastException: > org.apache.hadoop.hbase.tool.Canary$RegionServerStdOutSink cannot be cast to > org.apache.hadoop.hbase.tool.Canary$RegionStdOutSink > at org.apache.hadoop.hbase.tool.Canary.newMonitor(Canary.java:911) > at org.apache.hadoop.hbase.tool.Canary.run(Canary.java:796) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.hbase.tool.Canary.main(Canary.java:1571) > This happens because we typecast the sink depending on the mode (zookeeper > mode/region server mode) that Canary is configured in. In case no mode is > specified, we typecast the sink into _RegionStdOutSink_. In general, it is > possible to provide inconsistent mode and sink types while running Canary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18830) TestCanaryTool does not check Canary monitor's error code
[ https://issues.apache.org/jira/browse/HBASE-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-18830. Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.5.0 1.4.0 3.0.0 2.0.0 Pushed to branch-1.4 and up > TestCanaryTool does not check Canary monitor's error code > - > > Key: HBASE-18830 > URL: https://issues.apache.org/jira/browse/HBASE-18830 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18830.001.patch > > > None of the tests inside TestCanaryTool check Canary monitor's error code. > Thus, it is possible that the monitor has registered an error and yet the > tests pass. We should check the value returned by the _ToolRunner.run()_ > method inside each unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18867) maven enforcer plugin needs update to work with jdk9
[
https://issues.apache.org/jira/browse/HBASE-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180099#comment-16180099
]
Peter Somogyi commented on HBASE-18867:
---
[~busbey], using this patch the enforcer error is solved for me however later
on the build fails for hbase-spark module. Were you able to run a successful
build?
I was using Oracle java 9:
java version "9"
Java(TM) SE Runtime Environment (build 9+181)
Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)
{code}
[ERROR] error: error while loading package, Missing dependency 'object
java.lang.Object in compiler mirror', required by
/Users/peter.somogyi/.m2/repository/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar(scala/package.class)
[ERROR] error: error while loading package, Missing dependency 'object
java.lang.Object in compiler mirror', required by
/Users/peter.somogyi/.m2/repository/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar(scala/runtime/package.class)
[ERROR] error: scala.reflect.internal.MissingRequirementError: object
java.lang.Object in compiler mirror not found.
[ERROR] at
scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:16)
[ERROR] at
scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:17)
{code}
> maven enforcer plugin needs update to work with jdk9
>
>
> Key: HBASE-18867
> URL: https://issues.apache.org/jira/browse/HBASE-18867
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Affects Versions: 2.0.0-alpha-3
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-18867.0.patch
>
>
> build fails under jdk9, even when targeting java 8
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-enforcer-plugin:1.4:enforce
> (min-maven-min-java-banned-xerces) on project hbase: Execution
> min-maven-min-java-banned-xerces of goal
> org.apache.maven.plugins:maven-enforcer-plugin:1.4:enforce failed: An API
> incompatibility was encountered while executing
> org.apache.maven.plugins:maven-enforcer-plugin:1.4:enforce:
> java.lang.ExceptionInInitializerError: null
> [ERROR] -
> [ERROR] realm =plugin>org.apache.maven.plugins:maven-enforcer-plugin:1.4
> [ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
> [ERROR] urls[0] =
> file:/Users/busbey/.m2/repository/org/apache/maven/plugins/maven-enforcer-plugin/1.4/maven-enforcer-plugin-1.4.jar
> [ERROR] urls[1] =
> file:/Users/busbey/.m2/repository/org/codehaus/mojo/extra-enforcer-rules/1.0-beta-6/extra-enforcer-rules-1.0-beta-6.jar
> [ERROR] urls[2] =
> file:/Users/busbey/.m2/repository/org/apache/maven/shared/maven-dependency-tree/2.2/maven-dependency-tree-2.2.jar
> [ERROR] urls[3] =
> file:/Users/busbey/.m2/repository/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar
> [ERROR] urls[4] =
> file:/Users/busbey/.m2/repository/org/eclipse/aether/aether-util/0.9.0.M2/aether-util-0.9.0.M2.jar
> [ERROR] urls[5] =
> file:/Users/busbey/.m2/repository/org/apache/maven/shared/maven-common-artifact-filters/1.4/maven-common-artifact-filters-1.4.jar
> [ERROR] urls[6] =
> file:/Users/busbey/.m2/repository/com/ibm/icu/icu4j/56.1/icu4j-56.1.jar
> [ERROR] urls[7] =
> file:/Users/busbey/.m2/repository/backport-util-concurrent/backport-util-concurrent/3.1/backport-util-concurrent-3.1.jar
> [ERROR] urls[8] =
> file:/Users/busbey/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.11/plexus-interpolation-1.11.jar
> [ERROR] urls[9] =
> file:/Users/busbey/.m2/repository/org/slf4j/slf4j-jdk14/1.5.6/slf4j-jdk14-1.5.6.jar
> [ERROR] urls[10] =
> file:/Users/busbey/.m2/repository/org/slf4j/jcl-over-slf4j/1.5.6/jcl-over-slf4j-1.5.6.jar
> [ERROR] urls[11] =
> file:/Users/busbey/.m2/repository/org/apache/maven/reporting/maven-reporting-api/2.2.1/maven-reporting-api-2.2.1.jar
> [ERROR] urls[12] =
> file:/Users/busbey/.m2/repository/org/apache/maven/doxia/doxia-sink-api/1.1/doxia-sink-api-1.1.jar
> [ERROR] urls[13] =
> file:/Users/busbey/.m2/repository/org/apache/maven/doxia/doxia-logging-api/1.1/doxia-logging-api-1.1.jar
> [ERROR] urls[14] =
> file:/Users/busbey/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar
> [ERROR] urls[15] =
> file:/Users/busbey/.m2/repository/org/codehaus/plexus/plexus-interactivity-api/1.0-alpha-4/plexus-interactivity-api-1.0-alpha-4.jar
> [ERROR] urls[16] =
> file:/Users/busbey/.m2/repository/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar
> [ERROR] urls[17] =
>
[jira] [Commented] (HBASE-18875) Thrift server supports read-only mode
[ https://issues.apache.org/jira/browse/HBASE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180091#comment-16180091 ] Guangxu Cheng commented on HBASE-18875: --- Thanks [~yuzhih...@gmail.com] for review. > Thrift server supports read-only mode > - > > Key: HBASE-18875 > URL: https://issues.apache.org/jira/browse/HBASE-18875 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-4 > > Attachments: 18875.add, HBASE-18875.branch-1.v1.patch, > HBASE-18875.patch, HBASE-18875.v1.patch > > > Provide option for thrift server to support read-only mode. > To start the thrift server, use the -ro option or set hbase.thrift.readonly > to true. > false: Both read and write request are permitted.(all methods) > true : Only the read request is permitted. (only get/scan method) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18878) Use Optional instead of Nullable annotation in ObserverContext to better describe that the caller may not be presented
Duo Zhang created HBASE-18878: - Summary: Use Optional instead of Nullable annotation in ObserverContext to better describe that the caller may not be presented Key: HBASE-18878 URL: https://issues.apache.org/jira/browse/HBASE-18878 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang I've already done lots of Nullable to Optional change when purging the interfaces for CP. This is a big one so open a separated issue for it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18411) Dividing FiterList into two separate sub-classes: FilterListWithOR , FilterListWithAND
[ https://issues.apache.org/jira/browse/HBASE-18411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180079#comment-16180079 ] Zheng Hu commented on HBASE-18411: -- bq. could you upload the patch to reviewboard? [~psomogyi], HBASE-18411 is dependent on HBASE-18160, and HBASE-18160 haven't been committed into git rep yet. so uploading the HBASE-18411.v1.patch to RB will be failed. I upload it for the initial version, after with HBASE-18411 resolved, I'll put the patch to review board :-) > Dividing FiterList into two separate sub-classes: FilterListWithOR , > FilterListWithAND > > > Key: HBASE-18411 > URL: https://issues.apache.org/jira/browse/HBASE-18411 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Peter Somogyi > Attachments: HBASE-18411.v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18875) Thrift server supports read-only mode
[ https://issues.apache.org/jira/browse/HBASE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18875: --- Attachment: 18875.add Addendum fixes compilation. > Thrift server supports read-only mode > - > > Key: HBASE-18875 > URL: https://issues.apache.org/jira/browse/HBASE-18875 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-4 > > Attachments: 18875.add, HBASE-18875.branch-1.v1.patch, > HBASE-18875.patch, HBASE-18875.v1.patch > > > Provide option for thrift server to support read-only mode. > To start the thrift server, use the -ro option or set hbase.thrift.readonly > to true. > false: Both read and write request are permitted.(all methods) > true : Only the read request is permitted. (only get/scan method) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18160) Fix incorrect logic in FilterList.filterKeyValue
[
https://issues.apache.org/jira/browse/HBASE-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-18160:
-
Attachment: HBASE-18160.branch-1.3.v1.patch
HBASE-18160.branch-1.4.v1.patch
Run Hadoop QA again.
> Fix incorrect logic in FilterList.filterKeyValue
> -
>
> Key: HBASE-18160
> URL: https://issues.apache.org/jira/browse/HBASE-18160
> Project: HBase
> Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
> Fix For: 1.4.0, 1.2.6, 1.3.2, 2.0.0-alpha-3
>
> Attachments: filter-and-map.txt, filter-or-map.txt,
> HBASE-18160.branch-1.3.v1.patch, HBASE-18160.branch-1.3.v1.patch,
> HBASE-18160.branch-1.4.v1.patch, HBASE-18160.branch-1.4.v1.patch,
> HBASE-18160.v1.patch, HBASE-18160.v2.patch, HBASE-18160.v2.patch,
> HBASE-18160.v3.patch, HBASE-18160.v4.patch, HBASE-18160.v5.patch,
> HBASE-18160.v6.patch, HBASE-18160.v6.patch, HBASE-18160.v7.patch,
> HBASE-18160.v8.patch
>
>
> As HBASE-17678 said, there are two problems in FilterList.filterKeyValue
> implementation:
> 1. FilterList did not consider INCLUDE_AND_SEEK_NEXT_ROW case( seems like
> INCLUDE_AND_SEEK_NEXT_ROW is a newly added case, and the dev forgot to
> consider FilterList), So if a user use INCLUDE_AND_SEEK_NEXT_ROW in his own
> Filter and wrapped by a FilterList, it'll throw an
> IllegalStateException("Received code is not valid.").
> 2. For FilterList with MUST_PASS_ONE, if filter-A in filter list return
> INCLUDE and filter-B in filter list return INCLUDE_AND_NEXT_COL, the
> FilterList will return INCLUDE_AND_NEXT_COL finally. According to the
> mininal step rule , It's incorrect. (filter list with MUST_PASS_ONE choose
> the mininal step among filters in filter list. Let's call it: The Mininal
> Step Rule).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Reopened] (HBASE-18875) Thrift server supports read-only mode
[
https://issues.apache.org/jira/browse/HBASE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reopened HBASE-18875:
{code}
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR]
/testptch/hbase/hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandlerWithReadOnly.java:[88,22]
error: cannot find symbol
[INFO] 1 error
{code}
> Thrift server supports read-only mode
> -
>
> Key: HBASE-18875
> URL: https://issues.apache.org/jira/browse/HBASE-18875
> Project: HBase
> Issue Type: New Feature
> Components: Thrift
>Affects Versions: 2.0.0, 3.0.0, 1.5.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-4
>
> Attachments: HBASE-18875.branch-1.v1.patch, HBASE-18875.patch,
> HBASE-18875.v1.patch
>
>
> Provide option for thrift server to support read-only mode.
> To start the thrift server, use the -ro option or set hbase.thrift.readonly
> to true.
> false: Both read and write request are permitted.(all methods)
> true : Only the read request is permitted. (only get/scan method)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18875) Thrift server supports read-only mode
[
https://issues.apache.org/jira/browse/HBASE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180055#comment-16180055
]
Hadoop QA commented on HBASE-18875:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
17s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
25s{color} | {color:red} hbase-thrift in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
25s{color} | {color:red} hbase-thrift in the patch failed with JDK v1.8.0_144.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 26s{color}
| {color:red} hbase-thrift in the patch failed with JDK v1.8.0_144. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
27s{color} | {color:red} hbase-thrift in the patch failed with JDK v1.7.0_151.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 27s{color}
| {color:red} hbase-thrift in the patch failed with JDK v1.7.0_151. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
51s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
15s{color} | {color:red} The patch causes 14 errors with Hadoop v2.6.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m
28s{color} | {color:red} The patch causes 14 errors with Hadoop v2.6.2. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
46s{color} | {color:red} The patch causes 14 errors with Hadoop v2.6.3. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
1s{color} | {color:red} The patch causes 14 errors with Hadoop v2.6.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m
17s{color} | {color:red} The patch causes 14 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
32s{color} | {color:red} The patch causes 14 errors with Hadoop v2.7.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 11m
50s{color} | {color:red} The patch causes 14 errors with Hadoop v2.7.2. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 13m
5s{color} | {color:red} The patch causes 14 errors with Hadoop v2.7.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} |
[jira] [Updated] (HBASE-18875) Thrift server supports read-only mode
[ https://issues.apache.org/jira/browse/HBASE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18875: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0-alpha-4 1.5.0 1.4.0 Status: Resolved (was: Patch Available) > Thrift server supports read-only mode > - > > Key: HBASE-18875 > URL: https://issues.apache.org/jira/browse/HBASE-18875 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-4 > > Attachments: HBASE-18875.branch-1.v1.patch, HBASE-18875.patch, > HBASE-18875.v1.patch > > > Provide option for thrift server to support read-only mode. > To start the thrift server, use the -ro option or set hbase.thrift.readonly > to true. > false: Both read and write request are permitted.(all methods) > true : Only the read request is permitted. (only get/scan method) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18762) Canary sink type cast error
[ https://issues.apache.org/jira/browse/HBASE-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180019#comment-16180019 ] Andrew Purtell commented on HBASE-18762: Thanks, lgtm Will commit shortly > Canary sink type cast error > --- > > Key: HBASE-18762 > URL: https://issues.apache.org/jira/browse/HBASE-18762 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Attachments: HBASE-18762.001.patch > > > When running the main method of Canary.java, we see the following error: > Exception in thread "main" java.lang.ClassCastException: > org.apache.hadoop.hbase.tool.Canary$RegionServerStdOutSink cannot be cast to > org.apache.hadoop.hbase.tool.Canary$RegionStdOutSink > at org.apache.hadoop.hbase.tool.Canary.newMonitor(Canary.java:911) > at org.apache.hadoop.hbase.tool.Canary.run(Canary.java:796) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.hbase.tool.Canary.main(Canary.java:1571) > This happens because we typecast the sink depending on the mode (zookeeper > mode/region server mode) that Canary is configured in. In case no mode is > specified, we typecast the sink into _RegionStdOutSink_. In general, it is > possible to provide inconsistent mode and sink types while running Canary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18875) Thrift server supports read-only mode
[ https://issues.apache.org/jira/browse/HBASE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guangxu Cheng updated HBASE-18875: -- Attachment: HBASE-18875.branch-1.v1.patch branch-1.v1.patch can be applied to branch-1/branch-1.3/branch-1.4 normally. > Thrift server supports read-only mode > - > > Key: HBASE-18875 > URL: https://issues.apache.org/jira/browse/HBASE-18875 > Project: HBase > Issue Type: New Feature > Components: Thrift >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Attachments: HBASE-18875.branch-1.v1.patch, HBASE-18875.patch, > HBASE-18875.v1.patch > > > Provide option for thrift server to support read-only mode. > To start the thrift server, use the -ro option or set hbase.thrift.readonly > to true. > false: Both read and write request are permitted.(all methods) > true : Only the read request is permitted. (only get/scan method) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18830) TestCanaryTool does not check Canary monitor's error code
[ https://issues.apache.org/jira/browse/HBASE-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180018#comment-16180018 ] Andrew Purtell commented on HBASE-18830: lgtm Will commit > TestCanaryTool does not check Canary monitor's error code > - > > Key: HBASE-18830 > URL: https://issues.apache.org/jira/browse/HBASE-18830 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Attachments: HBASE-18830.001.patch > > > None of the tests inside TestCanaryTool check Canary monitor's error code. > Thus, it is possible that the monitor has registered an error and yet the > tests pass. We should check the value returned by the _ToolRunner.run()_ > method inside each unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18845) TestReplicationSmallTests fails after HBASE-14004
[ https://issues.apache.org/jira/browse/HBASE-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180017#comment-16180017 ] Duo Zhang commented on HBASE-18845: --- The replication will keep reading the same file if it is still opened for writing. In the old code, we determine if a file is still opened for writing by checking if it is the only one file in the replication queue, so inject a new file to the queue will make progress, and now this is changed, so if you want the replication queue to be advanced then you need to roll the WAL explicitly. > TestReplicationSmallTests fails after HBASE-14004 > - > > Key: HBASE-18845 > URL: https://issues.apache.org/jira/browse/HBASE-18845 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0, 2.0.0-alpha-3 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 3.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-18845.patch > > > testEmptyWALRecovery and testVerifyRepJob -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18770) We should not allow RegionObserver.preBulkLoadHFile to bypass the default behavior
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180015#comment-16180015 ] Duo Zhang commented on HBASE-18770: --- We should allow, please see the description of this issue... I think the correct way is to change the List passed in... The operation which I want to stop is that, in RegionObserver.preBulkLoadHFile, you can call ObserverContext.bypass to bypass the default behavior, which means we will not do bulk load at the upper layer. As for now, it seems impossible for CP users to load a StoreFile without touching IA.Private classes, so it does not make sense to still let users bypass the default behavior as they can not do it by themselves... Thanks. > We should not allow RegionObserver.preBulkLoadHFile to bypass the default > behavior > -- > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang > Fix For: 2.0.0-alpha-4 > > > As now we do not allow users to create a StoreFile instance. Users can still > select the files to be bulk loaded by modifying the familyPaths passed in. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18815) We need to pass something like CompactionRequest in CP to give user some information about the compaction
[ https://issues.apache.org/jira/browse/HBASE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180009#comment-16180009 ] Duo Zhang commented on HBASE-18815: --- Yes, [~anoopsamjohn] said that CompactionRequest may carry some useful information for CP users, for example, if the compaction is major or not. I think we can add a parameter for some of the compact-related methods which are called after we generate the CompactionRequest. Thanks. > We need to pass something like CompactionRequest in CP to give user some > information about the compaction > - > > Key: HBASE-18815 > URL: https://issues.apache.org/jira/browse/HBASE-18815 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang > Fix For: 2.0.0-alpha-4 > > > For example, it is a major compaction or minor compaction. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18786) FileNotFoundException should not be silently handled for primary region replicas
[ https://issues.apache.org/jira/browse/HBASE-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180008#comment-16180008 ] Andrew Purtell commented on HBASE-18786: Pushed addendum to remove now invalid unit test To https://git-wip-us.apache.org/repos/asf/hbase.git 5a3d9cb225..d74bdc3100 branch-1 -> branch-1 0c93008dd6..e6e7ae4aae branch-1.4 -> branch-1.4 bb81e9f3ca..fa0b0934db branch-2 -> branch-2 cfb6a54f69..b145286f36 master -> master > FileNotFoundException should not be silently handled for primary region > replicas > > > Key: HBASE-18786 > URL: https://issues.apache.org/jira/browse/HBASE-18786 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Reporter: Ashu Pachauri >Assignee: Andrew Purtell > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18786-branch-1.patch, HBASE-18786-branch-1.patch, > HBASE-18786.patch, HBASE-18786.patch > > > This is a follow up for HBASE-18186. > FileNotFoundException while scanning from a primary region replica can be > indicative of a more severe problem. Handling them silently can cause many > underlying issues go undetected. We should either > 1. Hard fail the regionserver if there is a FNFE on a primary region replica, > OR > 2. Report these exceptions as some region / server level metric so that these > can be proactively investigated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18876) Backup create command fails to take queue parameter as option
[ https://issues.apache.org/jira/browse/HBASE-18876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179996#comment-16179996 ] Hudson commented on HBASE-18876: FAILURE: Integrated in Jenkins build HBase-2.0 #578 (See [https://builds.apache.org/job/HBase-2.0/578/]) HBASE-18876 Backup create command fails to take queue parameter as (tedyu: rev ad60bc5f6052c66259d0b55d619ff1087221cb09) * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupDriver.java > Backup create command fails to take queue parameter as option > - > > Key: HBASE-18876 > URL: https://issues.apache.org/jira/browse/HBASE-18876 > Project: HBase > Issue Type: Bug >Reporter: Amit Kabra >Assignee: Amit Kabra > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18876.patch > > > Backup help shows we can set queue using -q parameter. > -q Yarn queue name to run backup restore command on > But when we give ./hbase backup create full hdfs://localhost:8020/test/ -t > test1 -q hbase > It throws following error "Error when parsing command-line arguments: > Unrecognized option: -q" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18870) Hbase Backup should set the details to MR jobs
[ https://issues.apache.org/jira/browse/HBASE-18870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179909#comment-16179909 ] Vladimir Rodionov commented on HBASE-18870: --- Hey, [~vishk]. Mind assigning this to yourself? That is easy fix. > Hbase Backup should set the details to MR jobs > -- > > Key: HBASE-18870 > URL: https://issues.apache.org/jira/browse/HBASE-18870 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Vishal Khandelwal >Priority: Minor > Labels: backup > > For Incremental and Full Backups job names should be set correctly. It would > make the debug and monitoring better. > Current : > incremental -> WALPlayer_1506332227619 > ExportSnapshot-snapshot_1506323760362_default_t1 > Proposed : > incremental -> Backup_Incremental__ > Full -> Backups_Full__ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18871) Counters for Backup Job
[ https://issues.apache.org/jira/browse/HBASE-18871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179905#comment-16179905 ] Vladimir Rodionov commented on HBASE-18871: --- We have some counters in BackupInfo and standard counters for MR jobs : bytes copied, number of files. This needs to be enchanced in BackupInfo in a future > Counters for Backup Job > --- > > Key: HBASE-18871 > URL: https://issues.apache.org/jira/browse/HBASE-18871 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Vishal Khandelwal > Labels: backup > > There should be Map Reduce counter for backup jobs which would be helpful to > analysis the perf and data for backup > #1 Number of Rows backed up > #2 Number of WAL Files processed > #3 Number of HFiles created > etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18876) Backup create command fails to take queue parameter as option
[ https://issues.apache.org/jira/browse/HBASE-18876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179900#comment-16179900 ] Vladimir Rodionov commented on HBASE-18876: --- Thank you for the patch, [~amitkabraiiit] > Backup create command fails to take queue parameter as option > - > > Key: HBASE-18876 > URL: https://issues.apache.org/jira/browse/HBASE-18876 > Project: HBase > Issue Type: Bug >Reporter: Amit Kabra >Assignee: Amit Kabra > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18876.patch > > > Backup help shows we can set queue using -q parameter. > -q Yarn queue name to run backup restore command on > But when we give ./hbase backup create full hdfs://localhost:8020/test/ -t > test1 -q hbase > It throws following error "Error when parsing command-line arguments: > Unrecognized option: -q" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18872) Backup scaling for multiple table and millions of row
[ https://issues.apache.org/jira/browse/HBASE-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov reassigned HBASE-18872: - Assignee: Vladimir Rodionov > Backup scaling for multiple table and millions of row > - > > Key: HBASE-18872 > URL: https://issues.apache.org/jira/browse/HBASE-18872 > Project: HBase > Issue Type: Improvement >Reporter: Vishal Khandelwal >Assignee: Vladimir Rodionov > > I did a simple experiment of loading ~200 million rows on a table 1 and > nothing in a table 2. This test was done on a local cluster ~ approx 3-4 > containers were running in parallel. The focus of the test was not on how > much time backup takes but on time spent on the table were no data has been > changed. > *Table without Data -->* > Elapsed: 44mins, 52sec > Average Map Time 3sec > Average Shuffle Time 2mins, 35sec > Average Merge Time0sec > Average Reduce Time 0sec > Map : 2052 > Reduce : 1 > *Table with Data -->* > Elapsed: 1hrs, 44mins, 10sec > Average Map Time 4sec > Average Shuffle Time 37sec > Average Merge Time3sec > Average Reduce Time 47sec > Map : 2052 > Reduce : 64 > All above numbers are a single node cluster so not many mappers run in > parallel. but let's extrapolate this to 20 node cluster, with ~100 tables and > data size to be backed up various for approx 2000 Wals, let us say each 20 > node can process 3 containers i.e 60 wals in parallel. assume 3 sec are spent > in each WALs i.e. 6000\ 60 sec --> 100 per table --> 1 sec for all > tables. > ~166 mins --> ~2.7 hrs only for filtering. This does not seem to be scale. > (These are just rough numbers from a basic test). As all parsing is O (m > (WALS) * n (Tables)) > Main intend of this test is to see even the backup of very less churning > table might take good amount for just filtering the data. As number of table > or data increases, this does not seem scalable > Even i can see from our current cluster numbers easily close to 100 table, > 200 millions rows, 200 -300 GB. > I would suggest that we should have filtering to parse WALs once and to > segregate in multiple WALs per table --> hFiles from per table wals. ( just a > rough idea). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18183) Region interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179896#comment-16179896 ] Umesh Agashe edited comment on HBASE-18183 at 9/25/17 10:45 PM: [~anoop.hbase], thanks for your reply. I am looking into HBASE-18703 which is regarding inconsistency between doMiniBathMutate() and processRowsWithLocks(). The only internal implementation of RowProcessor is MultiRowMutationProcessor. This is used by mutateRows() methods. I am modifying doMiniBatchMutate so that it can be called by mutateRows() methods (with and without locks as an explicit argument). This leaves processRowsWithLocks() and it's use of RowProcessor. Currently BaseRowProcessorEndpoint instantiates and calls processRowsWithLocks(). It may be sufficient, if we can modify processRowsWithLocks() to call doMiniBatchMutate() and CP hooks can be used. This is a heads up about what I am thinking regarding HBASE-18703. Perhaps when patch is ready we can discuss this further. was (Author: uagashe): [~anoop.hbase], thanks for your reply. I am looking into HBASE-18703 which is regarding inconsistency between doMiniBathMutate() and processRowsWithLocks(). The only internal implementation of RowProcessor is MultiRowMutationProcessor. This is used by mutateRows() methods. I am modifying doMiniBatchMutate so that it can be called by mutateRows() methods (with and without locks as an explicit argument). This leaves processRowsWithLocks() and it's use of RowProcessor. Currently BaseRowProcessorEndpoint instantiates and calls processRowsWithLocks(). It may be sufficient, if we can modify processRowsWithLocks() to call doMiniBatchMutate() and CP hooks can be used. This heads up on what I am thinking regarding HBASE-18703. Perhaps when patch is ready we can discuss this further. > Region interface cleanup for CP expose > -- > > Key: HBASE-18183 > URL: https://issues.apache.org/jira/browse/HBASE-18183 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18859) Purge PB from BulkLoadObserver
[
https://issues.apache.org/jira/browse/HBASE-18859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179897#comment-16179897
]
Hadoop QA commented on HBASE-18859:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 7m
1s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
0s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
37m 29s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}116m
0s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
25s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}183m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18859 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888931/HBASE-18859.master.002.patch
|
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6c320f430295 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a5f8443 |
| Default Java | 1.8.0_144 |
|
[jira] [Commented] (HBASE-18183) Region interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179896#comment-16179896 ] Umesh Agashe commented on HBASE-18183: -- [~anoop.hbase], thanks for your reply. I am looking into HBASE-18703 which is regarding inconsistency between doMiniBathMutate() and processRowsWithLocks(). The only internal implementation of RowProcessor is MultiRowMutationProcessor. This is used by mutateRows() methods. I am modifying doMiniBatchMutate so that it can be called by mutateRows() methods (with and without locks as an explicit argument). This leaves processRowsWithLocks() and it's use of RowProcessor. Currently BaseRowProcessorEndpoint instantiates and calls processRowsWithLocks(). It may be sufficient, if we can modify processRowsWithLocks() to call doMiniBatchMutate() and CP hooks can be used. This heads up on what I am thinking regarding HBASE-18703. Perhaps when patch is ready we can discuss this further. > Region interface cleanup for CP expose > -- > > Key: HBASE-18183 > URL: https://issues.apache.org/jira/browse/HBASE-18183 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
