[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16195095#comment-16195095
]
Hudson commented on HBASE-17732:
Results for branch HBASE-18467, done in 4 hr 24 min and counting
[build #136 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18467/136/]:
FAILURE
details (if available):
(x) *{color:red}-1 overall{color}*
Committer, please check your recent inclusion of a patch for this issue.
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18467/136//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 checks{color}
-- For more information [see jdk8
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18467/136//JDK8_Nightly_Build_Report/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Labels: incompatible
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch, HBASE-17732.master.011.patch,
> HBASE-17732.master.012.patch, HBASE-17732.master.013.patch,
> HBASE-17732.master.014.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185176#comment-16185176
]
Hudson commented on HBASE-17732:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3795 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3795/])
HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732 (appy: rev
74d0adce61fc39ef9d1ec2254dcd0f53181bb62c)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRowProcessorEndpoint.java
* (edit)
hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/RefreshHFilesEndpoint.java
* (edit)
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/token/TokenProvider.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpoint.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorEndpoint.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorServiceBackwardCompatiblity.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/Coprocessor.java
* (edit)
hbase-endpoint/src/main/java/org/apache/hadoop/hbase/coprocessor/AggregateImplementation.java
* (edit)
hbase-endpoint/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/WALCoprocessor.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/ProtobufCoprocessorService.java
* (add)
dev-support/design-docs/Coprocessor_Design_Improvements-Use_composition_instead_of_inheritance-HBASE-17732-2017_09_27.pdf
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
* (edit)
hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/RowCountEndpoint.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerCoprocessorHost.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpointWithErrors.java
* (edit)
hbase-endpoint/src/main/java/org/apache/hadoop/hbase/coprocessor/Export.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/regionserver/TestServerCustomProtocol.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpointNullResponse.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestAsyncCoprocessorEndpoint.java
* (edit)
hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/BulkDeleteEndpoint.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MultiRowMutationEndpoint.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java
HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732; (stack:
rev b5b4108fcedeea4e25e86f426e313ffa51229d10)
* (add) dev-support/design-docs/README.txt
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/Coprocessor.java
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Labels: incompatible
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch, HBASE-17732.master.011.patch,
> HBASE-17732.master.012.patch, HBASE-17732.master.013.patch,
> HBASE-17732.master.014.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185161#comment-16185161
]
Hudson commented on HBASE-17732:
FAILURE: Integrated in Jenkins build HBase-2.0 #594 (See
[https://builds.apache.org/job/HBase-2.0/594/])
HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732 (appy: rev
a6a303816cc429db6ed90f634b2d6126f0e8d8fc)
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpointWithErrors.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MultiRowMutationEndpoint.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java
* (edit)
hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/BulkDeleteEndpoint.java
* (edit)
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java
* (edit)
hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/RowCountEndpoint.java
* (edit)
hbase-endpoint/src/main/java/org/apache/hadoop/hbase/coprocessor/Export.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpoint.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRowProcessorEndpoint.java
* (edit)
hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/RefreshHFilesEndpoint.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/regionserver/TestServerCustomProtocol.java
* (edit)
hbase-endpoint/src/main/java/org/apache/hadoop/hbase/coprocessor/AggregateImplementation.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/WALCoprocessor.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpointNullResponse.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorEndpoint.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerCoprocessorHost.java
* (add)
dev-support/design-docs/Coprocessor_Design_Improvements-Use_composition_instead_of_inheritance-HBASE-17732-2017_09_27.pdf
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/Coprocessor.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/ProtobufCoprocessorService.java
* (edit)
hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestAsyncCoprocessorEndpoint.java
* (edit)
hbase-endpoint/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorServiceBackwardCompatiblity.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/token/TokenProvider.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732; (stack:
rev ac933f7af2188f1c834a4d5334880c5f6d5dbc5c)
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/Coprocessor.java
* (add) dev-support/design-docs/README.txt
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Labels: incompatible
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch, HBASE-17732.master.011.patch,
> HBASE-17732.master.012.patch, HBASE-17732.master.013.patch,
> HBASE-17732.master.014.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183685#comment-16183685
]
Anoop Sam John commented on HBASE-17732:
Done. HBASE-18898
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Labels: incompatible
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch, HBASE-17732.master.011.patch,
> HBASE-17732.master.012.patch, HBASE-17732.master.013.patch,
> HBASE-17732.master.014.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183648#comment-16183648
]
Appy commented on HBASE-17732:
--
bq. Can we have a way in the code (before trying to call the hook) to know
whether the user has implemented one particular hook or not?
You're right. I too gave a lot of thought on this unnecessary creation of extra
'Operation' object for each and every hook if there's a loaded CP which only
overrides few (or even worse, one) hook(s).
It is definitely doable using reflection! I have a rough design in mind but not
the bandwidth to work on it right now. But let's get the discussion going.
Create a jira?
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Labels: incompatible
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch, HBASE-17732.master.011.patch,
> HBASE-17732.master.012.patch, HBASE-17732.master.013.patch,
> HBASE-17732.master.014.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183640#comment-16183640
]
Anoop Sam John commented on HBASE-17732:
This is great stuff from u. One Q...
Now we can have diff kind of Observer hooks going into diff classes. This is
nice.
Can we have a way in the code (before trying to call the hook) to know
whether the user has implemented one particular hook or not? eg: On
write related hooks only prePut() might be what the user CP
implemented. All others are just dummy impl from the interface. Can
we have a way for the core code to know this and avoid the call to
other dummy hooks fully?Some times we do some processing for just
calling CP hooks (Say we have to make a POJO out of PB object for
calling) and if the user CP not impl this hook, we can avoid this
extra work fully.The pain of this will be more when we have to
later deprecate one hook and add new. So the dummy impl in new hook
has to call the old one and that might be doing some extra work
normally.
If the CP f/w itself is having a way to tell this, the core code can make use.
What am expecting is some thing like in PB way where we can call
CPObject.hasPre(), then CPObject. pre ().. Should not like asking
users to impl this extra ugly thing. When the CP instance is loaded in the
RS/HM, that object will be having this info also. Just some thinking and
sharing with u. May be for thinking and later changes.
Thanks for this nice refactoring.
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Labels: incompatible
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch, HBASE-17732.master.011.patch,
> HBASE-17732.master.012.patch, HBASE-17732.master.013.patch,
> HBASE-17732.master.014.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[ https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183482#comment-16183482 ] Hudson commented on HBASE-17732: FAILURE: Integrated in Jenkins build HBase-2.0 #588 (See [https://builds.apache.org/job/HBase-2.0/588/]) HBASE-17732 Coprocessor Design Improvements (appy: rev 0c883a23c57e16b212c69193edd8e5d01306b823) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckMOB.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/WALObserver.java * (edit) hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestAsyncCoprocessorEndpoint.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALFactory.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterCoprocessor.java * (edit) hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/BulkDeleteEndpoint.java * (edit) hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/ExampleMasterObserverWithMetrics.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckReplicas.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAbort.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController3.java * (edit) hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/ExampleRegionObserverWithMetrics.java * (edit) hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/ErrorThrowingGetObserver.java * (edit) hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorEndpoint.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/BaseTestHBaseFsck.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestWALObserver.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithAbort.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/NoOpScanPolicyObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicaWithCluster.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestResultFromCoprocessor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestNamespaceCommands.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverScannerOpenHook.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCoprocessorWhitelistMasterObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SecureBulkLoadManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/CoprocessorWhitelistMasterObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncAdminBuilder.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorMetrics.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTimeout.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/WALCoprocessor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestScanEarlyTermination.java * (add) hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorServiceBackwardCompatibility.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MultiRowMutationEndpoint.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorStop.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/SingletonCoprocessorService.java * (edit)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[ https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183473#comment-16183473 ] Hudson commented on HBASE-17732: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3789 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3789/]) HBASE-17732 Coprocessor Design Improvements (appy: rev 97513466c05f5eaadb94425c98098063ac374098) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java * (edit) hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroups.java * (edit) hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpointNullResponse.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverForAddingMutationsFromCoprocessors.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckMOB.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseEnvironment.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorConfiguration.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestScannerRetriableFailure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java * (edit) hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/BulkDeleteEndpoint.java * (edit) hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithTTLs.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCoprocessorHost.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SampleRegionWALObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncAdminBuilder.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/CoprocessorEnvironment.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSettingTimeoutOnBlockingPoint.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SampleRegionWALCoprocessor.java * (edit) hbase-endpoint/src/main/java/org/apache/hadoop/hbase/coprocessor/AggregateImplementation.java * (add) hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorServiceBackwardCompatibility.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithAbort.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsReplication.java * (edit) hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/ExampleMasterObserverWithMetrics.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityReplication.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClient.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTimeout.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestResultFromCoprocessor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestFSWAL.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestWithDisabledAuthorization.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientOperationInterrupt.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionServerObserver.java * (edit) hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/RowCountEndpoint.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterSpaceQuotaObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestBlockEvictionFromClient.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALFactory.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestCoprocessorScanPolicy.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestServerBusyException.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/CoprocessorWhitelistMasterObserver.java * (edit)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183142#comment-16183142
]
Appy commented on HBASE-17732:
--
h4. Commit message
{noformat}
HBASE-17732 Coprocessor Design Improvements
--
TL;DR
--
We are moving from Inheritence
- Observer *is* Coprocessor
- FooService *is* CoprocessorService
To Composition
- Coprocessor *has* Observer
- Coprocessor *has* Service
--
Design Changes
--
- Adds four new interfaces - MasterCoprocessor, RegionCoprocessor,
RegionServierCoprocessor,
WALCoprocessor
- These new *Coprocessor interfaces have a get*Observer() function for each
observer type
supported by them.
- Added Coprocessor#getService() to base interface. All extending
*Coprocessor interfaces will
get it from the base interface.
- Added BulkLoadObserver hooks to RegionCoprocessorHost instad of
SecureBulkLoadManager doing its
own trickery.
- CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP
internals.
Deleted if can, else marked @VisibleForTesting.
--
Backward Compatibility
--
- Old coprocessors implementing *Observer won't get loaded (no backward
compatibility guarantees).
- Third party coprocessors only implementing Coprocessor will not get
loaded (just like Observers).
- Old coprocessors implementing CoprocessorService (for master/region host)
/SingletonCoprocessorService (for RegionServer host) will continue to
work with 2.0.
- Added test to ensure backward compatibility of
CoprocessorService/SingletonCoprocessorService
- Note that if a coprocessor implements both observer and service in same
class, its service
component will continue to work but it's observer component won't work.
--
Notes
--
Did a side-by-side comparison of CPs in master and after patch. These
coprocessors which were just
CoprocessorService earlier, needed a home in some coprocessor in new
design. For most it was clear
since they were using a particular type of environment. Some were tricky.
- JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener
makes sense for
processes?)
- RSGroupAdminEndpoint --> MasterCP
- VisibilityController -> MasterCP and RegionCP
These were converted to RegionCoprocessor because they were using
RegionCoprocessorEnvironment
which can only come from a RegionCPHost.
- AggregateImplementation
- BaseRowProcessorEndpoint
- BulkDeleteEndpoint
- Export
- RefreshHFilesEndpoint
- RowCountEndpoint
- MultiRowMutationEndpoint
- SecureBulkLoadEndpoint
- TokenProvider
{noformat}
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch, HBASE-17732.master.011.patch,
> HBASE-17732.master.012.patch, HBASE-17732.master.013.patch,
> HBASE-17732.master.014.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183136#comment-16183136
]
Appy commented on HBASE-17732:
--
Rebased to latest master.
Updated the commit message. Will post it here too for summary.
Like stack said, minor teething issues will be addressed by a followup
HBASE-18884
Will commit to master and branch-2 after doing a local clean install.
Will update the release note on this jira.
Thanks everyone. Thanks [~stack] and [~apurtell] for the reviews.
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch, HBASE-17732.master.011.patch,
> HBASE-17732.master.012.patch, HBASE-17732.master.013.patch,
> HBASE-17732.master.014.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182070#comment-16182070
]
stack commented on HBASE-17732:
---
Porting here my rb review from just now:
{code}
I went over Andrew's great review and noticed that you've addressed his
comments or moved out his suggestions and remainder to a new issue which is
fine I think. I skimmed about half the patch to notice what is new (suggestions
from our chat today).
+1 on commit. There will be teething issues to do in follow-ups but lets get
the bulk in.
The below you can address on commit or in follow-on (You will probably have
conflicts when you go to push because of patch that just went in..)
{code}
Sweet.
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch, HBASE-17732.master.011.patch,
> HBASE-17732.master.012.patch, HBASE-17732.master.013.patch,
> HBASE-17732.master.014.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181990#comment-16181990
]
Hadoop QA commented on HBASE-17732:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
49s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 91 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 11m
43s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
37m 34s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
25s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}117m
45s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
28s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
13s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
19s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
13s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
5s{color} | {color:green}
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181981#comment-16181981
]
Appy commented on HBASE-17732:
--
Yay, hadoop QA passed!
Here's the list of minor followup improvements HBASE-18884.
[~stack] [~apurtell] What do you say? Is this good to go in?
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch,
> HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
> HBASE-17732.master.010.patch, HBASE-17732.master.011.patch,
> HBASE-17732.master.012.patch, HBASE-17732.master.013.patch,
> HBASE-17732.master.014.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181949#comment-16181949
]
Hadoop QA commented on HBASE-17732:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 91 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 10m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
24s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
27s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 50s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
35s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 89m
50s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
44s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
53s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
16s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
7s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
57s{color} | {color:green}
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180804#comment-16180804
]
Hadoop QA commented on HBASE-17732:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 91 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 10m
41s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
48s{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
39m 6s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
51s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}102m
36s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
11s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
16s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
16s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
2s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180475#comment-16180475
]
Hadoop QA commented on HBASE-17732:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 91 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 14m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
21s{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
2s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
40m 12s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
36s{color} | {color:red} hbase-server generated 2 new + 1 unchanged - 0 fixed =
3 total (was 1) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
39s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 22m 29s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 10m 3s{color}
| {color:red} hbase-mapreduce in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 10s{color}
| {color:red} hbase-thrift in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 5m 56s{color}
| {color:red} hbase-endpoint in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 17m 42s{color}
| {color:red} hbase-backup in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179684#comment-16179684
]
Hadoop QA commented on HBASE-17732:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 83 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 11m
23s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
54s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
36m 29s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
27s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}100m
44s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
39s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
0s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
5s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
43s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
3s{color} | {color:green}
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178996#comment-16178996
]
Hadoop QA commented on HBASE-17732:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 83 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 10m
59s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 2m
33s{color} | {color:red} patch has 10 errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 51s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
28s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 89m
57s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
29s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
45s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
51s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
5s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
57s{color} | {color:green} hbase-examples in the
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178610#comment-16178610
]
Hadoop QA commented on HBASE-17732:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 82 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 11m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 2m
31s{color} | {color:red} patch has 10 errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
36m 31s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
36s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}104m 44s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
16s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
55s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
41s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
54s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
50s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
38s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
13s{color} | {color:green} hbase-examples in the patch
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178457#comment-16178457
]
Hadoop QA commented on HBASE-17732:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-17732 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-17732 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888765/HBASE-17732.master.007.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8762/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch, HBASE-17732.master.005.patch,
> HBASE-17732.master.006.patch, HBASE-17732.master.007.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177471#comment-16177471
]
Hadoop QA commented on HBASE-17732:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 18m
1s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 40 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 7m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 2m
9s{color} | {color:red} patch has 10 errors when building our shaded downstream
artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 22s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
41s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 57m 24s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
0s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
42s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 5m 35s{color}
| {color:red} hbase-endpoint in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
7s{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 2m
50s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}161m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.coprocessor.TestCoprocessorStop |
| | hadoop.hbase.coprocessor.TestHTableWrapper |
| |
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177448#comment-16177448
]
Hadoop QA commented on HBASE-17732:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 25 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 9m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
55s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
36m 20s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
35s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 23s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 5s{color}
| {color:red} hbase-thrift in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
39s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 53s{color}
| {color:red} hbase-endpoint in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 41s{color}
| {color:red} hbase-examples in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 2m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}165m 19s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.tool.TestSecureLoadIncrementalHFilesSplitRecovery |
| | hadoop.hbase.TestJMXListener |
| |
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175525#comment-16175525
]
Josh Elser commented on HBASE-17732:
Will try to give you a heads-up assuming that HBASE-18807 lands before you get
this one in, [~appy].
Should be a simple conflict to resolve by switching the {{QuotaProtos.Quotas}}
object with the new object {{GlobalQuotaSettings}} (assuming the class name
doesn't change).
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159941#comment-16159941
]
Duo Zhang commented on HBASE-17732:
---
Skimmed the design doc. I'm not fully understand but seems OK. No big concerns
from me.
Thanks.
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157930#comment-16157930
]
Appy commented on HBASE-17732:
--
Thanks [~chia7712]!
Filed HBASE-17732 to follow up with docs changes.
Linked to HBASE-18169 which is about CP API improvements in 2.0.
Will address other concerns in followup.
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157922#comment-16157922
]
Appy commented on HBASE-17732:
--
Hi [~anoopsamjohn], [~Apache9], [~ghelmling],
[~stack] suggested that it would be good to get your thoughts on this one.
Please oblige me. Thanks
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156779#comment-16156779
]
Chia-Ping Tsai commented on HBASE-17732:
Nice work. Many of docs will get stale. :) We need a issue to introduce this
new coprocessor architecture in our docs. What kind of coprocessor information
we should show in the web ui of Master/RegionServer/Region? Does JMXListener(or
other classes which implement Coprocessor only) work in ur design? BTW, there
are other issues about the improvement of coprocessor architecture. Could we
open a umbralla to address them in 2.0? Thanks.
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156636#comment-16156636
]
Appy commented on HBASE-17732:
--
I updated the link in the discussion thread but forgot to update it here. Sorry
about that. Done now.
Quick ref:
https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156527#comment-16156527
]
Chia-Ping Tsai commented on HBASE-17732:
I can't access the link of design doc. Could you share the doc to us? Thanks.
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-rOpQmZuixCDua-LT9X4/edit
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156359#comment-16156359
]
Hadoop QA commented on HBASE-17732:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 40 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
21s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 74m 24s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
18s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
40s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 5m 52s{color}
| {color:red} hbase-endpoint in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
12s{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 2m
29s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}160m 47s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.security.access.TestAccessController |
| | hadoop.hbase.client.TestEnableTable |
| | hadoop.hbase.client.TestFromClientSide3 |
| | hadoop.hbase.client.TestServerBusyException |
| | hadoop.hbase.TestJMXConnectorServer |
| | hadoop.hbase.client.TestHCM |
| | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | hadoop.hbase.client.TestFromClientSide |
| | hadoop.hbase.regionserver.TestScannerWithCorruptHFile |
| |
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142815#comment-16142815
]
Hadoop QA commented on HBASE-17732:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 24 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 7s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
51s{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed =
1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
54s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 44m 33s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 4m 20s{color}
| {color:red} hbase-thrift in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
24s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 55s{color}
| {color:red} hbase-endpoint in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 6s{color}
| {color:red} hbase-examples in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
57s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}151m 40s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Uncallable method
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost$RegionEnvironment)
defined in anonymous class At RegionCoprocessorHost.java:anonymous class At
RegionCoprocessorHost.java:[lines 519-520] |
| Failed junit tests | hadoop.hbase.regionserver.TestKeepDeletes |
| | hadoop.hbase.coprocessor.TestCoprocessorConfiguration |
| |
