[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[
https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832408#comment-17832408
]
Hudson commented on HBASE-27657:
Results for branch branch-2
[build #1022 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1022/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1022/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1022/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1022/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1022/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Connection and Request Attributes
> -
>
> Key: HBASE-27657
> URL: https://issues.apache.org/jira/browse/HBASE-27657
> Project: HBase
> Issue Type: New Feature
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> Currently we have the ability to set Operation attributes, via
> Get.setAttribute, etc. It would be useful to be able to set attributes at the
> request and connection level.
> These levels can result in less duplication. For example, send some
> attributes once per connection instead of for every one of the millions of
> requests a connection might send. Or send once for the request, instead of
> duplicating on every operation in a multi request.
> Additionally, the Connection and RequestHeader are more globally available on
> the server side. Both can be accessed via RpcServer.getCurrentCall(), which
> is useful in various integration points – coprocessors, custom queues,
> quotas, slow log, etc. Operation attributes are harder to access because you
> need to parse the raw Message into the appropriate type to get access to the
> getter.
> I was thinking adding two new methods to Connection interface:
> - setAttribute (and getAttribute/getAttributes)
> - setRequestAttributeProvider
> Any Connection attributes would be set onto the ConnectionHeader during
> initialization. The RequestAttributeProvider would be called when creating
> each RequestHeader.
> An alternative to setRequestAttributeProvider would be to add this into
> HBaseRpcController, which can already be customized via site configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[
https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832374#comment-17832374
]
Hudson commented on HBASE-27657:
Results for branch branch-2.6
[build #86 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/86/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/86/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/86/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/86/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/86/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Connection and Request Attributes
> -
>
> Key: HBASE-27657
> URL: https://issues.apache.org/jira/browse/HBASE-27657
> Project: HBase
> Issue Type: New Feature
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> Currently we have the ability to set Operation attributes, via
> Get.setAttribute, etc. It would be useful to be able to set attributes at the
> request and connection level.
> These levels can result in less duplication. For example, send some
> attributes once per connection instead of for every one of the millions of
> requests a connection might send. Or send once for the request, instead of
> duplicating on every operation in a multi request.
> Additionally, the Connection and RequestHeader are more globally available on
> the server side. Both can be accessed via RpcServer.getCurrentCall(), which
> is useful in various integration points – coprocessors, custom queues,
> quotas, slow log, etc. Operation attributes are harder to access because you
> need to parse the raw Message into the appropriate type to get access to the
> getter.
> I was thinking adding two new methods to Connection interface:
> - setAttribute (and getAttribute/getAttributes)
> - setRequestAttributeProvider
> Any Connection attributes would be set onto the ConnectionHeader during
> initialization. The RequestAttributeProvider would be called when creating
> each RequestHeader.
> An alternative to setRequestAttributeProvider would be to add this into
> HBaseRpcController, which can already be customized via site configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[
https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748887#comment-17748887
]
Hudson commented on HBASE-27657:
Results for branch branch-2
[build #855 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/855/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/855/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/855/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/855/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/855/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Connection and Request Attributes
> -
>
> Key: HBASE-27657
> URL: https://issues.apache.org/jira/browse/HBASE-27657
> Project: HBase
> Issue Type: New Feature
>Reporter: Bryan Beaudreault
>Assignee: Ray Mattingly
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> Currently we have the ability to set Operation attributes, via
> Get.setAttribute, etc. It would be useful to be able to set attributes at the
> request and connection level.
> These levels can result in less duplication. For example, send some
> attributes once per connection instead of for every one of the millions of
> requests a connection might send. Or send once for the request, instead of
> duplicating on every operation in a multi request.
> Additionally, the Connection and RequestHeader are more globally available on
> the server side. Both can be accessed via RpcServer.getCurrentCall(), which
> is useful in various integration points – coprocessors, custom queues,
> quotas, slow log, etc. Operation attributes are harder to access because you
> need to parse the raw Message into the appropriate type to get access to the
> getter.
> I was thinking adding two new methods to Connection interface:
> - setAttribute (and getAttribute/getAttributes)
> - setRequestAttributeProvider
> Any Connection attributes would be set onto the ConnectionHeader during
> initialization. The RequestAttributeProvider would be called when creating
> each RequestHeader.
> An alternative to setRequestAttributeProvider would be to add this into
> HBaseRpcController, which can already be customized via site configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748850#comment-17748850 ] Bryan Beaudreault commented on HBASE-27657: --- Pushed to branch-2, branch-3, and master. I added release notes including usage info and info around incompatible changes to the constructors of a few pluggable but Private and LimitedPrivate classes. This should be ok for a minor version change per our docs around Private and LimitedPrivate. Thanks for all the work here [~rmdmattingly], and for the design review [~zhangduo]! > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > Fix For: 2.6.0, 3.0.0-beta-1 > > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[
https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747130#comment-17747130
]
Hudson commented on HBASE-27657:
Results for branch master
[build #876 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/876/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/876/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/876/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/876/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Connection and Request Attributes
> -
>
> Key: HBASE-27657
> URL: https://issues.apache.org/jira/browse/HBASE-27657
> Project: HBase
> Issue Type: New Feature
>Reporter: Bryan Beaudreault
>Assignee: Ray Mattingly
>Priority: Major
>
> Currently we have the ability to set Operation attributes, via
> Get.setAttribute, etc. It would be useful to be able to set attributes at the
> request and connection level.
> These levels can result in less duplication. For example, send some
> attributes once per connection instead of for every one of the millions of
> requests a connection might send. Or send once for the request, instead of
> duplicating on every operation in a multi request.
> Additionally, the Connection and RequestHeader are more globally available on
> the server side. Both can be accessed via RpcServer.getCurrentCall(), which
> is useful in various integration points – coprocessors, custom queues,
> quotas, slow log, etc. Operation attributes are harder to access because you
> need to parse the raw Message into the appropriate type to get access to the
> getter.
> I was thinking adding two new methods to Connection interface:
> - setAttribute (and getAttribute/getAttributes)
> - setRequestAttributeProvider
> Any Connection attributes would be set onto the ConnectionHeader during
> initialization. The RequestAttributeProvider would be called when creating
> each RequestHeader.
> An alternative to setRequestAttributeProvider would be to add this into
> HBaseRpcController, which can already be customized via site configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[
https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747126#comment-17747126
]
Hudson commented on HBASE-27657:
Results for branch branch-3
[build #21 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/21/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/21/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/21/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/21/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Connection and Request Attributes
> -
>
> Key: HBASE-27657
> URL: https://issues.apache.org/jira/browse/HBASE-27657
> Project: HBase
> Issue Type: New Feature
>Reporter: Bryan Beaudreault
>Assignee: Ray Mattingly
>Priority: Major
>
> Currently we have the ability to set Operation attributes, via
> Get.setAttribute, etc. It would be useful to be able to set attributes at the
> request and connection level.
> These levels can result in less duplication. For example, send some
> attributes once per connection instead of for every one of the millions of
> requests a connection might send. Or send once for the request, instead of
> duplicating on every operation in a multi request.
> Additionally, the Connection and RequestHeader are more globally available on
> the server side. Both can be accessed via RpcServer.getCurrentCall(), which
> is useful in various integration points – coprocessors, custom queues,
> quotas, slow log, etc. Operation attributes are harder to access because you
> need to parse the raw Message into the appropriate type to get access to the
> getter.
> I was thinking adding two new methods to Connection interface:
> - setAttribute (and getAttribute/getAttributes)
> - setRequestAttributeProvider
> Any Connection attributes would be set onto the ConnectionHeader during
> initialization. The RequestAttributeProvider would be called when creating
> each RequestHeader.
> An alternative to setRequestAttributeProvider would be to add this into
> HBaseRpcController, which can already be customized via site configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744180#comment-17744180 ] Duo Zhang commented on HBASE-27657: --- Then let's use the solution in the design doc. Creating a table is lightweighted as it does not create any network connections, just some objects in memory. Thanks. > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744150#comment-17744150 ] Bryan Beaudreault commented on HBASE-27657: --- Yea batch requests are very common for us, and the attributes we want to send can in some cases be long. We don’t want to duplicate that for 100+ operations in a batch. I suppose an alternative would be to modify Table/AsyncTable to add an attributes argument to the batch methods. It might be a bit confusing. What do you think? > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744041#comment-17744041 ] Duo Zhang commented on HBASE-27657: --- o, just took a quick pass on the design doc, it is very straight-forward. I think this is an acceptable way to pass request attributes. Just a minor question, have you guys considered adding an 'attributes' field to the Operation class? What is the problem for adding the field there? Because it can not suit the batch methods? Thanks. > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744037#comment-17744037 ] Duo Zhang commented on HBASE-27657: --- Will take a look today, Thanks! > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743955#comment-17743955 ] Ray Mattingly commented on HBASE-27657: --- I wrote up a draft PR demonstrating a TableBuilder request attributes implementation here: https://github.com/apache/hbase/pull/5326 > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743225#comment-17743225 ] Ray Mattingly commented on HBASE-27657: --- I've updated [the design doc|https://docs.google.com/document/d/1cGEmUn2kAPhn_Q18DvAOhigtCbnQ8ia6oV4OvMb5DyU/edit?usp=sharing] to reflect our new TableBuilder interface for request attributes and have a branch which has proved the concept. Does this seem like a more suitable design to you [~zhangduo]? > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742194#comment-17742194 ] Ray Mattingly commented on HBASE-27657: --- Thanks for the feedback here, that's a fair criticism for sure. I'm going to explore whether we can add support for request header configuration in the TableBuilder > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742037#comment-17742037 ] Duo Zhang commented on HBASE-27657: --- I still do not get the point here... If you want to expose connection/request attributes to usres, you need to change the IA.Public APIs so users can set the attributes? For example, you add a new parameter for createConnection, which is for setting connection level attributes, so for request level attributes, you'd better do the same? Find a proper place to add a method for setting attributes per client , instead of relying on a customized rpc controller implementation. Thanks. > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[
https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741660#comment-17741660
]
Bryan Beaudreault commented on HBASE-27657:
---
That's correct – clients can't really access the HBaseRpcController directly.
They can provide their own implementation, which can in turn call their own
code within it. But they can't "call" methods on the controller themselves or
get access to an instance of it.
For us, most of the attributes we'd want to set have values that are available
statically in the client processes – such as environment variables or static
thread locals for API request contexts. So for us, we can create our own
HBaseRpcController impl which pulls on those. For example:
{code:java}
public static class MyRpcController extends HBaseRpcController {
@Override
public Map getAttributes() {
return ImmutableMap.builder()
.put("apiCaller", Bytes.toBytes(ApiContext.current().getCallerString())
.put("workerType", Bytes.toBytes(System.getenv("WORKER_TYPE"))
.build();
}
} {code}
> Connection and Request Attributes
> -
>
> Key: HBASE-27657
> URL: https://issues.apache.org/jira/browse/HBASE-27657
> Project: HBase
> Issue Type: New Feature
>Reporter: Bryan Beaudreault
>Assignee: Ray Mattingly
>Priority: Major
>
> Currently we have the ability to set Operation attributes, via
> Get.setAttribute, etc. It would be useful to be able to set attributes at the
> request and connection level.
> These levels can result in less duplication. For example, send some
> attributes once per connection instead of for every one of the millions of
> requests a connection might send. Or send once for the request, instead of
> duplicating on every operation in a multi request.
> Additionally, the Connection and RequestHeader are more globally available on
> the server side. Both can be accessed via RpcServer.getCurrentCall(), which
> is useful in various integration points – coprocessors, custom queues,
> quotas, slow log, etc. Operation attributes are harder to access because you
> need to parse the raw Message into the appropriate type to get access to the
> getter.
> I was thinking adding two new methods to Connection interface:
> - setAttribute (and getAttribute/getAttributes)
> - setRequestAttributeProvider
> Any Connection attributes would be set onto the ConnectionHeader during
> initialization. The RequestAttributeProvider would be called when creating
> each RequestHeader.
> An alternative to setRequestAttributeProvider would be to add this into
> HBaseRpcController, which can already be customized via site configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741604#comment-17741604 ] Ray Mattingly commented on HBASE-27657: --- Is the HBaseRpcController trivially available to clients when generating requests? I think inaccessibility is the main blocker for just adding getters/setters there > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740224#comment-17740224 ] Duo Zhang commented on HBASE-27657: --- Any updates here? > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739300#comment-17739300 ] Duo Zhang commented on HBASE-27657: --- Seems I do not have the permission to comment on the design doc. My question is about the request attributes, why not just add new methods in HBaseRpcController for adding and getting request attributes? > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27657) Connection and Request Attributes
[ https://issues.apache.org/jira/browse/HBASE-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17737337#comment-17737337 ] Ray Mattingly commented on HBASE-27657: --- I've written up [a basic design doc|https://docs.google.com/document/d/1cGEmUn2kAPhn_Q18DvAOhigtCbnQ8ia6oV4OvMb5DyU/edit?usp=sharing] to pair with [our initial PR|https://github.com/apache/hbase/pull/5306]. > Connection and Request Attributes > - > > Key: HBASE-27657 > URL: https://issues.apache.org/jira/browse/HBASE-27657 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > > Currently we have the ability to set Operation attributes, via > Get.setAttribute, etc. It would be useful to be able to set attributes at the > request and connection level. > These levels can result in less duplication. For example, send some > attributes once per connection instead of for every one of the millions of > requests a connection might send. Or send once for the request, instead of > duplicating on every operation in a multi request. > Additionally, the Connection and RequestHeader are more globally available on > the server side. Both can be accessed via RpcServer.getCurrentCall(), which > is useful in various integration points – coprocessors, custom queues, > quotas, slow log, etc. Operation attributes are harder to access because you > need to parse the raw Message into the appropriate type to get access to the > getter. > I was thinking adding two new methods to Connection interface: > - setAttribute (and getAttribute/getAttributes) > - setRequestAttributeProvider > Any Connection attributes would be set onto the ConnectionHeader during > initialization. The RequestAttributeProvider would be called when creating > each RequestHeader. > An alternative to setRequestAttributeProvider would be to add this into > HBaseRpcController, which can already be customized via site configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
