[jira] [Commented] (HBASE-18842) The hbase shell clone_snaphost command returns bad error message
[
https://issues.apache.org/jira/browse/HBASE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198811#comment-16198811
]
Jesse Yates commented on HBASE-18842:
-
Failure looks unrelated and passed with #3860.
> The hbase shell clone_snaphost command returns bad error message
>
>
> Key: HBASE-18842
> URL: https://issues.apache.org/jira/browse/HBASE-18842
> Project: HBase
> Issue Type: Bug
>Reporter: Thoralf Gutierrez
>Assignee: Jesse Yates
>Priority: Minor
> Fix For: 2.0.0, 3.0.0
>
> Attachments:
> 0001-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0002-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0003-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0004-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0005-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch
>
>
> When you call the hbase shell clone_snapshot command with a target namespace
> that doesn't exist, you get an error message, but the variable used to
> identify the inexistent namespace is wrong:
> {noformat}
> hbase(main):001:0> clone_snapshot 'someSnapshotName',
> 'someNamespaceName:someTableName'
> ERROR: Unknown namespace someSnapshotName!
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot data.
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
> hbase> clone_snapshot 'snapshotName', 'namespace:tableName'
> {noformat}
> It should rather say:
> {noformat}
> ERROR: Unknown namespace someNamespaceName!
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18842) The hbase shell clone_snaphost command returns bad error message
[
https://issues.apache.org/jira/browse/HBASE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesse Yates updated HBASE-18842:
Fix Version/s: 2.0.0
> The hbase shell clone_snaphost command returns bad error message
>
>
> Key: HBASE-18842
> URL: https://issues.apache.org/jira/browse/HBASE-18842
> Project: HBase
> Issue Type: Bug
>Reporter: Thoralf Gutierrez
>Assignee: Jesse Yates
>Priority: Minor
> Fix For: 2.0.0, 3.0.0
>
> Attachments:
> 0001-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0002-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0003-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0004-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0005-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch
>
>
> When you call the hbase shell clone_snapshot command with a target namespace
> that doesn't exist, you get an error message, but the variable used to
> identify the inexistent namespace is wrong:
> {noformat}
> hbase(main):001:0> clone_snapshot 'someSnapshotName',
> 'someNamespaceName:someTableName'
> ERROR: Unknown namespace someSnapshotName!
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot data.
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
> hbase> clone_snapshot 'snapshotName', 'namespace:tableName'
> {noformat}
> It should rather say:
> {noformat}
> ERROR: Unknown namespace someNamespaceName!
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18842) The hbase shell clone_snaphost command returns bad error message
[
https://issues.apache.org/jira/browse/HBASE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesse Yates updated HBASE-18842:
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
> The hbase shell clone_snaphost command returns bad error message
>
>
> Key: HBASE-18842
> URL: https://issues.apache.org/jira/browse/HBASE-18842
> Project: HBase
> Issue Type: Bug
>Reporter: Thoralf Gutierrez
>Assignee: Jesse Yates
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments:
> 0001-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0002-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0003-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0004-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0005-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch
>
>
> When you call the hbase shell clone_snapshot command with a target namespace
> that doesn't exist, you get an error message, but the variable used to
> identify the inexistent namespace is wrong:
> {noformat}
> hbase(main):001:0> clone_snapshot 'someSnapshotName',
> 'someNamespaceName:someTableName'
> ERROR: Unknown namespace someSnapshotName!
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot data.
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
> hbase> clone_snapshot 'snapshotName', 'namespace:tableName'
> {noformat}
> It should rather say:
> {noformat}
> ERROR: Unknown namespace someNamespaceName!
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18842) The hbase shell clone_snaphost command returns bad error message
[
https://issues.apache.org/jira/browse/HBASE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179321#comment-16179321
]
Jesse Yates edited comment on HBASE-18842 at 9/25/17 5:00 PM:
--
Talked about this with [~ThorGutierrez] offline - we are just going to stick
with the current ruby style (i.e. if over guard clauses) to keep things
consistent. lint issue is just failure to recognize java class.
was (Author: jesse_yates):
Talked about this with [~ThorGutierrez] offline - we are just going to stick
with the current ruby style (if over guard clauses) to keep things consistent.
lint issue is just failure to recognize java class.
> The hbase shell clone_snaphost command returns bad error message
>
>
> Key: HBASE-18842
> URL: https://issues.apache.org/jira/browse/HBASE-18842
> Project: HBase
> Issue Type: Bug
>Reporter: Thoralf Gutierrez
>Assignee: Jesse Yates
>Priority: Minor
> Attachments:
> 0001-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0002-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch
>
>
> When you call the hbase shell clone_snapshot command with a target namespace
> that doesn't exist, you get an error message, but the variable used to
> identify the inexistent namespace is wrong:
> {noformat}
> hbase(main):001:0> clone_snapshot 'someSnapshotName',
> 'someNamespaceName:someTableName'
> ERROR: Unknown namespace someSnapshotName!
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot data.
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
> hbase> clone_snapshot 'snapshotName', 'namespace:tableName'
> {noformat}
> It should rather say:
> {noformat}
> ERROR: Unknown namespace someNamespaceName!
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18842) The hbase shell clone_snaphost command returns bad error message
[
https://issues.apache.org/jira/browse/HBASE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179321#comment-16179321
]
Jesse Yates commented on HBASE-18842:
-
Talked about this with [~ThorGutierrez] offline - we are just going to stick
with the current ruby style (if over guard clauses) to keep things consistent.
lint issue is just failure to recognize java class.
> The hbase shell clone_snaphost command returns bad error message
>
>
> Key: HBASE-18842
> URL: https://issues.apache.org/jira/browse/HBASE-18842
> Project: HBase
> Issue Type: Bug
>Reporter: Thoralf Gutierrez
>Assignee: Jesse Yates
>Priority: Minor
> Attachments:
> 0001-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0002-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch
>
>
> When you call the hbase shell clone_snapshot command with a target namespace
> that doesn't exist, you get an error message, but the variable used to
> identify the inexistent namespace is wrong:
> {noformat}
> hbase(main):001:0> clone_snapshot 'someSnapshotName',
> 'someNamespaceName:someTableName'
> ERROR: Unknown namespace someSnapshotName!
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot data.
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
> hbase> clone_snapshot 'snapshotName', 'namespace:tableName'
> {noformat}
> It should rather say:
> {noformat}
> ERROR: Unknown namespace someNamespaceName!
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-18842) The hbase shell clone_snaphost command returns bad error message
[
https://issues.apache.org/jira/browse/HBASE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesse Yates reassigned HBASE-18842:
---
Assignee: Jesse Yates
> The hbase shell clone_snaphost command returns bad error message
>
>
> Key: HBASE-18842
> URL: https://issues.apache.org/jira/browse/HBASE-18842
> Project: HBase
> Issue Type: Bug
>Reporter: Thoralf Gutierrez
>Assignee: Jesse Yates
>Priority: Minor
>
> When you call the hbase shell clone_snapshot command with a target namespace
> that doesn't exist, you get an error message, but the variable used to
> identify the inexistent namespace is wrong:
> {noformat}
> hbase(main):001:0> clone_snapshot 'someSnapshotName',
> 'someNamespaceName:someTableName'
> ERROR: Unknown namespace someSnapshotName!
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot data.
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
> hbase> clone_snapshot 'snapshotName', 'namespace:tableName'
> {noformat}
> It should rather say:
> {noformat}
> ERROR: Unknown namespace someNamespaceName!
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18842) The hbase shell clone_snaphost command returns bad error message
[
https://issues.apache.org/jira/browse/HBASE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170850#comment-16170850
]
Jesse Yates commented on HBASE-18842:
-
Any chance for a patch [~ThorGutierrez]? Looks like we could do a light hack
where commands could optionally handle exceptions to generate custom ruby
exceptions.
> The hbase shell clone_snaphost command returns bad error message
>
>
> Key: HBASE-18842
> URL: https://issues.apache.org/jira/browse/HBASE-18842
> Project: HBase
> Issue Type: Bug
>Reporter: Thoralf Gutierrez
>Priority: Minor
>
> When you call the hbase shell clone_snapshot command with a target namespace
> that doesn't exist, you get an error message, but the variable used to
> identify the inexistent namespace is wrong:
> {noformat}
> hbase(main):001:0> clone_snapshot 'someSnapshotName',
> 'someNamespaceName:someTableName'
> ERROR: Unknown namespace someSnapshotName!
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot data.
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
> hbase> clone_snapshot 'snapshotName', 'namespace:tableName'
> {noformat}
> It should rather say:
> {noformat}
> ERROR: Unknown namespace someNamespaceName!
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-15704) Refactoring: Move HFileArchiver from backup to its own package
[ https://issues.apache.org/jira/browse/HBASE-15704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257332#comment-15257332 ] Jesse Yates commented on HBASE-15704: - They were from some of the work Salesforce was doing at the time around supporting our backups. However, as everything is moving towards the "new" backup impl (based on wals - that still happening?), I don't know if we need to keep it around. > Refactoring: Move HFileArchiver from backup to its own package > -- > > Key: HBASE-15704 > URL: https://issues.apache.org/jira/browse/HBASE-15704 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-15704-v2.patch > > > This class is in backup package (as well as backup/examples classes) but is > not backup - related. Move examples classes to hbase-examples package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15629) Backport HBASE-14703 to 0.98+
[ https://issues.apache.org/jira/browse/HBASE-15629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238221#comment-15238221 ] Jesse Yates commented on HBASE-15629: - Later this week work for you fellas? > Backport HBASE-14703 to 0.98+ > - > > Key: HBASE-15629 > URL: https://issues.apache.org/jira/browse/HBASE-15629 > Project: HBase > Issue Type: Task >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 0.98.19 > > Attachments: HBASE-15629-0.98.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15222416#comment-15222416 ] Jesse Yates commented on HBASE-14703: - Sorry, got side-tracked with real life. I don't think I'm going to be able to make 0.98 patch soon. Maybe [~chenheng] feels up to it? > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-branch-1.patch, HBASE-14703-branch-1.v1.patch, > HBASE-14703-branch-1_v1.patch, HBASE-14703-start.patch, > HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209709#comment-15209709 ] Jesse Yates commented on HBASE-14703: - oh, ok. my bad. please ignore me :) > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-branch-1.patch, HBASE-14703-branch-1.v1.patch, > HBASE-14703-branch-1_v1.patch, HBASE-14703-start.patch, > HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209705#comment-15209705 ] Jesse Yates commented on HBASE-14703: - Yeah, the conflicts. Generally, you should resolve those and cleanup the commit, AFAIK. > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-branch-1.patch, HBASE-14703-branch-1.v1.patch, > HBASE-14703-branch-1_v1.patch, HBASE-14703-start.patch, > HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209691#comment-15209691
]
Jesse Yates commented on HBASE-14703:
-
did you cherry pick that over? The commit message is... concerning:
{code}
commit ee33bf0c7d539a63e2bf84c888ff8cf8bb57b7f6
Author: Jesse Yates
Date: Fri Mar 4 19:07:59 2016 -0800
HBASE-14703 HTable.mutateRow does not collect stats (Heng Chen)
Conflicts:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerImpl.java
hbase-client/src/main/java/org/apache/hadoop/hbase/client/StatsTrackingRpcRetryingCaller.java
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
{code}
> HTable.mutateRow does not collect stats
> ---
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-branch-1.patch, HBASE-14703-branch-1.v1.patch,
> HBASE-14703-branch-1_v1.patch, HBASE-14703-start.patch,
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch,
> HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch,
> HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch,
> HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch,
> HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch,
> HBASE-14703_v9.patch
>
>
> We are trying to fix the stats implementation, by moving it out of the Result
> object and into an Rpc payload (but not the 'cell payload', just as part of
> the values returned from the request). This change will also us use easily
> switch to AsyncProcess as the executor, and support stats, for nearly all the
> rpc calls. However, that means when you upgrade the client or server, you
> will lose stats visibility until the other side is upgraded. We could keep
> around the Result based stats storage to accommodate the old api and send
> both stats back from the server (in each result and in the rpc payload).
> Note that we will still be wire compatible - protobufs mean we can just ride
> over the lack of information.
> The other tricky part of this is that Result has a
> non-InterfaceAudience.Private getStatistics() method (along with two
> InterfaceAudience.Private addResults and setStatistics methods), so we might
> need a release to deprecate the getStats() method before throwing it out?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184309#comment-15184309 ] Jesse Yates commented on HBASE-14703: - Great, thanks for the info [~apurtell]. I'll keep it in mind. > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184306#comment-15184306 ] Jesse Yates commented on HBASE-14703: - And to be fair, there was a long, unresolved discussion on the dev list as to what _should_ be backported to branches. Some of the argument was in favor of bring more things back, some of the argument was in favor of just bringing back bug fixes and keeping the improvements (which can have their own bugs) off the 'stable' branch. Again, this didn't get resolved, but doesn't help clarify project policies on this sort of stuff. I'm just trying to help here, hence garnering discussion around where it should go, especially as it is a kinda-sorta breaking change. > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184301#comment-15184301 ] Jesse Yates commented on HBASE-14703: - My understanding was that, _as a project_ we were not backporting features past an upstream branch point. So, if an upstream RM didn't want it, then the downstream shouldn't get it. IIRC, that came up for the client push back the first time around (but I'm probably thinking of another JIRA). Seeing as none of the other upstream RMs had weighed in, I took Sean's comments as what was expected to go in... he still RM'ing 1.2 line, yes? My apologies that I'm not up on the latest guidelines, as was going on previous discussions I had been involved with. You are still getting your code Mr. Purtell :) > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184014#comment-15184014 ] Jesse Yates commented on HBASE-14703: - Fair enough. I'll try to find some time for a 0.98 version this week. > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-14703: Resolution: Fixed Status: Resolved (was: Patch Available) Going to mark this closed based on [~busbey]'s comments. I believe it is standard that unless something isn't backported to a newer branch, it doesn't go any further back. Thanks for hanging in through all this [~chenheng]. > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15181944#comment-15181944 ] Jesse Yates commented on HBASE-14703: - And FWIW, looks like the build failure started in #746 with the same cause, so I think we are good on the commit. > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15181943#comment-15181943 ] Jesse Yates commented on HBASE-14703: - [~busbey] yes, that is still the case. However, it is a temporary outage until upgrade on a afaik not widely used feature that only impacts performance, not correctness. I'm leaning to not backporting b/c it is technically an incompatibilitity of feature (though still wire compatible - thanks protobuf!). Hence, kicking it back to the RMs :) Thanks for the pointer on branch-2 - I'm a bit behind in the latest dev state. > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15181829#comment-15181829 ] Jesse Yates commented on HBASE-14703: - I think we do have a [branch-2|https://github.com/apache/hbase/tree/branch-2] and its not exactly master, so going to take me a little bit to cut a new version of the patch. > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15181826#comment-15181826 ] Jesse Yates commented on HBASE-14703: - I committed to master and will work on a version for branch-2. I'm tentative to move this any further back due to the recent discussion on the dev list on what should and shouldn't be moved back. Its a nice improvement, but does introduce some changes to the AsyncProcess. I'll leave it up the RMs if they want it (but I'll do my best to do the work, if they want it backported). thoughts [~apurtell] [~ndimiduk] [~mantonov] [~busbey]? > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 3.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-14703: Fix Version/s: (was: 1.4.0) (was: 0.98.18) (was: 1.1.4) (was: 1.2.1) (was: 1.3.0) > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 3.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14703) HTable.mutateRow does not collect stats
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-14703: Summary: HTable.mutateRow does not collect stats (was: not collect stats when call HTable.mutateRow ) > HTable.mutateRow does not collect stats > --- > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 3.0.0, 1.3.0, 1.2.1, 1.1.4, 0.98.18, 1.4.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-14703: Fix Version/s: 1.4.0 0.98.18 1.1.4 1.2.1 1.3.0 3.0.0 > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 3.0.0, 1.3.0, 1.2.1, 1.1.4, 0.98.18, 1.4.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-14703: Status: Open (was: Patch Available) > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-14703: Status: Patch Available (was: Open) > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15163430#comment-15163430 ] Jesse Yates commented on HBASE-14703: - Mind taking a look [~chenheng]? > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, > HBASE-14703_v7.patch, HBASE-14703_v8.patch, HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153591#comment-15153591 ] Jesse Yates commented on HBASE-14703: - Sorry [~chenheng] - fell off my queue. Thanks for the reminder. Promise I will look at it this weekend. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061245#comment-15061245
]
Jesse Yates commented on HBASE-14703:
-
[~chenheng] I think making the changes form RB puts this patch into the
committable zone. Maybe fix the checkstyle issues? However, I've got some
family stuff going on at the moment (why I haven't been responsive recently).
Maybe [~apurtell] or [~stack] might be able to help push this over the goal
line? I should be able to be back to working come the new year, if you can wait
that long.
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch,
> HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch,
> HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch,
> HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch,
> HBASE-14703_v9.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034461#comment-15034461
]
Jesse Yates commented on HBASE-14703:
-
Didn't really need to, but now that it is, lets rev from RB
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch,
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch,
> HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch,
> HBASE-14703_v8.patch, HBASE-14703_v9.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026991#comment-15026991
]
Jesse Yates commented on HBASE-14703:
-
I think that right now, its not the best solution. The cost comes not
immediately, but suppose we don't finish changing the other interfaces. Then we
have mutateRows and checkAndMutate being the only two cases where we use the
new class and they way it looks is substantially different than every other
method call. By keeping it simple, it makes it easier for people to understand
what is happening as it is all written out and not hidden away.
We should either be refactoring or changing behavior; when you combine both its
hard to understand what the 'correct' solution is and makes it harder for
others to grok the change. In this case, we are making a functionality change,
so we shouldn't be too changing how things are organized (note
PayloadCarryingCallable is a _functionality_ change as we needed a new base
class to run the mutate callables). I think we should be trying for the
smallest possible change to accomplish our goals.
Is it so bad to just hold off on adding the new layer until we do the other
methods as part of async? I bet we will actually figure out that we want a
different heriarchy and mechanism for managing the callables once we look at it
holistically, which will make the work here pointless. If I'm wrong, then we
can just put the class from here and use it directly.
Fair enough?
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch,
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch,
> HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026202#comment-15026202
]
Jesse Yates commented on HBASE-14703:
-
Hey [~chenheng], can you attach that as a full patch? The way I find people
using addendums is generally for two things: (1) bug fixes immediately after
the fact, (2) helping to show changes. For instance, I used it previously to
show how you go follow up, but its a large change and it would have been hard
to keep track of how it fit onto the base patch, were it a single patch. For
smaller changes, especially that aren't expected to be committed, a new patch
and version is fine, and for really small changes - checkstyles, spelling, etc
- doing point patches is ok (so a 7.1).
The functional problem in particular here is that QA won't by able to handle
addendums, so making it patch available doesn't actually run the test :(
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch,
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch,
> HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026207#comment-15026207
]
Jesse Yates commented on HBASE-14703:
-
That patch just seems a bit much for right now. It makes the code change we are
putting in bigger, but doesn't really add any more clarity. How about coming
back to that when we do the changes for the rest of the methods? I tend to shy
away from adding heirarchy when its only being slightly used. WDYT?
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch,
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch,
> HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesse Yates updated HBASE-14703:
Attachment: HBASE-14703-v6_with-check-and-mutate.patch
Attaching a patch on top of v6 to support checkAndMutate with the new style.
Takes advantage of the 'exists' flag in Result to track processed state.
Haven't tested against the whole test suite, but TestCheckAndMutate passes for
me locally.
WDYT [~chenheng]?
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch,
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch,
> HBASE-14703_v6.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025023#comment-15025023
]
Jesse Yates commented on HBASE-14703:
-
With the amount of copied code, we will definitely want to consider making a
new callable class to that does a lot of the work, but that's for a future
patch. FWIW, this wasn't too complicated once I figured out how manage the
processed state, which is a testament to this new style.
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch,
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch,
> HBASE-14703_v6.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022668#comment-15022668
]
Jesse Yates commented on HBASE-14703:
-
{quote}
optional bool enabled = 4 [default = false];
Excuse me, what's the purpose for this option.
{quote}
Its just come cruft left in my example addendum from when I was trying to
figure out how to better implement stats. Unless you use it anywhere - I didn't
see it - lets pull it out.
bq. But not remove RegionLoadStats in ResultOrException for PB parser not
crashed when user upgrade cluster.
What do you mean? Its an optional field, so if its not there on the wire, PB
will just ignore it (which is why PB generates #hasLoadStats() method).
bq. Maybe we should add process flag back into MultiResponse. wdyt?
ClientProtos.MultiResponse (the PB) or MultiResponse (o.a.h.hbase.client
class)? I think by marking it as an EMPTY_RESULT we are implicitly saying
'processed'. The only other option for that value is as an exception, right?
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch,
> HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch,
> HBASE-14703_v5.patch, HBASE-14703_v6.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022589#comment-15022589
]
Jesse Yates commented on HBASE-14703:
-
Maybe [~apurtell] can give us some advice on the compatibility story here (I
forget :-/).
I think we can just do the fix in master as that cuts a new major release.
Worst case is you lose stats until both sides of the wire are upgraded. We
would also remove the public Result#getStats() method as that again becomes an
internal value, but I'm not sure if we can do that in a major release. Further,
I don't know how valuable it is to backport except some of the cleanup from
removing mutateRows statistics tracking and the statsTrackingRpcController
(since they never gets used anyways).
TL;DR on the JIRA so far:
We are trying to fix the stats implementation, by moving it out of the Result
object and into an Rpc payload (but not the 'cell payload', just as part of the
values returned from the request). This change will also us use easily switch
to AsyncProcess as the executor, and support stats, for nearly all the rpc
calls. However, that means when you upgrade the client or server, you will lose
stats visibility until the other side is upgraded. We could keep around the
Result based stats storage to accommodate the old api and send both stats back
from the server (in each result and in the rpc payload).
Note that we will still be wire compatible - protobufs mean we can just ride
over the lack of information.
The other tricky part of this is that Result has a
non-InterfaceAudience.Private getStatistics() method (along with two
InterfaceAudience.Private addResults and setStatistics methods), so we might
need a release to deprecate the getStats() method before throwing it out(?).
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14702_v5.2_addendum-addendum.patch,
> HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch,
> HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch,
> HBASE-14703_v5.patch, HBASE-14703_v6.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15015093#comment-15015093
]
Jesse Yates commented on HBASE-14703:
-
bq. This issue is about mutateRow and its statistics. And after this issue, the
framework will be build
What do you mean by this?
If you are agreeing with the direction I was suggesting, do you want to crack
at a patch for this one to do the cleanup? I'll can start filling follow ups
and pinging the dev list with the scope of changes and getting feedback.
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch,
> HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch,
> HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014002#comment-15014002
]
Jesse Yates commented on HBASE-14703:
-
Looking back to the original purpose, its actually about ripping out the
StatsTrackingRpcRetryingCaller. With the new implementation envisioned, I don't
think we will still need it.
Since we are working in 'master' there is no reason to put in a 'hack' fix and
then go back and fix it properly right after - we should just do it "right" the
first time. Right just means a bigger change, which takes a bit longer
How about we make this patch the cleanup of "stuff that doesn't work". That
would be the statsTrackingRpcRetryingCaller and all the stats work in
RsRpcServices where we attempt to attach the statistics to the result from the
mutateRows computation.
Then follow up JIRAs look like:
- Support stats tracking for all calls (Umbrella)
-- Use async everywhere in HTable calls
-- Move statistics to per-call
-- unify statistic collection to an interface
wdyt?
Thanks for being patient through all this - sometimes it takes some exploration
and trial/error to figure out what works.
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch,
> HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch,
> HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011480#comment-15011480
]
Jesse Yates commented on HBASE-14703:
-
bq. remove some code unused in RpcRetryingCallerImpl
I left it, to preserve the current implementation. I don't know where else it
would be used, but can't hurt to not change things.
Then here, in AsyncProcess
{code}
723 // As mutateRow, if statistic off, nothing will
return. So let's stop.
724 actionsInProgress.set(0);
{code}
This doesn't feel right. In receiveMultiAction -> setResult we just decrement
the actionsInProgress, rather than set it to 0. We don't have any tests that
check running ops concurrently, so I think it would be better to refactor
AsyncProcess a little bit more to unify the logic a bit
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch,
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch,
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesse Yates updated HBASE-14703:
Attachment: HBASE-14703-5.2-addendum.patch
I'm starting to think that to support stats across all the mutation based calls
needs to be reimplememented to support other calls. I'm attaching an addendum
to 5.2 (so apply 5.2 and then this patch; makes it easier to see where I'm
going with the bigger change).
Here's what I'm thinking instead.
Results should not have a regionStatistic as there are potentially multiple
results for a single region. Instead, MultiResponse should carry the
statistics. In the RsRpcServices, we can collect statistics about all the
regions affected by the call and then return them once per region Then in
asyncProcess we just update all the region stats at once when we receive the
MultiResponse.
Note this is also cleaner in cases where we return multiple results for the
same region; where previously we would return multiple region load stats, now
we can return just the most recent.
Also, did a little of cleanup to support using this is in the mutateRows by
adding an empty result for each mutateRow action that was successfully
processed.
My only concern is about the viablilty of implementing this w/o breaking wire
compatibility. Since this is just adding protobuf fields it will still make the
puts correctly, but in the case of upgrading, the statistics will not be
correct (actually, missing) from until both sides are upgraded. I don't think
this is a widely used feature yet, so I'm not too worried, but thought I should
mention it.
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch,
> HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch,
> HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch,
> HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch,
> HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesse Yates updated HBASE-14703:
Status: Open (was: Patch Available)
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch,
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch,
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesse Yates updated HBASE-14703:
Attachment: HBASE-14703-v4.patch
I like this a lot more. Still wasn't a fan of the switching we had to do in the
async process, the time calculation logic semi-copy from retryingcaller and the
lack of tracking for the operation.
So, I took a crack at it. Now, the timeout tracking is encapsulated and reused
in the RetryingCaller and in the callable. I think its necessary in the
callable, since that is the thing being reused across calls. Then I added a new
callable class with the expectation of almost getting rid of
RegionServerCallable everywhere (except for Coprocessor calls) in favor of a
'PayloadCarryingCallable'; the new callable sets the
PayloadCarryingRpcController, allowing it to be canceled and saving the
creation in _every single use_ of RegionServerCallable that we'd eventually
replace. This lets us unify the interfaces in the progress tracking and cancel
the mutateRows call as well.
[~chenheng], your turn. What do you think?
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch,
> HBASE-14703-v4.patch, HBASE-14703.patch, HBASE-14703_v1.patch,
> HBASE-14703_v2.patch, HBASE-14703_v3.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesse Yates updated HBASE-14703:
Attachment: HBASE-14703-v4.1.patch
Looks like I was a little bit behind master. Lets try this version.
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch,
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch,
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007720#comment-15007720
]
Jesse Yates commented on HBASE-14703:
-
sorry [~chenheng], got a bit caught up with stuff. Will take a look tomorrow.
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch,
> HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch,
> HBASE-14703_v3.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999564#comment-14999564
]
Jesse Yates commented on HBASE-14703:
-
Looking closer at the patch, I'm a bit concerned with how retries will work.
Since the AsyncProcess handles retries, we shouldn't be calling the
callable.callWithRetries, right? Then we would get two sets of retries with the
callable nested under the AsyncProcess. Can't we just use the standard AP
timeout then as well (its not something the user changes per-call)?
bq. it seems that processed flag has no relates with results
Ah, yes. For this, IIRC its all or nothing with the check so all you need to
send back is a single 'everything worked' flag, rather than a flag per
mutation, which would all be the same value. You added that with the
'processed' flag in the MultiResponse, but that's only going to be applicable
for the checkAndXXX cases, right?
I don't particularly like bolting on a processed flag onto the MulitResponse,
just to support this one case. Looking at the checkAnd methods, they all
follow the same basic logic and return a MutateResponse#processed, except for
checkAndMutate. That makes me think we can do the same thing we are doing here
for the other checkAndXXX methods, but fork logic a little bit in AsyncProcess
to support that kind of processing instead (as a follow up JIRA, not here).
In the meantime, lets just focus on supporting the mutateRow and then later we
can fix checkAndMutate along with the other checkAnd methods.
For mutateRow, maybe its better to just check to see if it has any Results,
rather than checking to see if there is a statistic tracker? It ties more into
why the ResponseConverter won't work, rather than why there are no Results in
the response and leave that information in a comment around the check for the
count of results.
Good stuff Heng Chen
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch,
> HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997093#comment-14997093
]
Jesse Yates commented on HBASE-14703:
-
Getting there! There seems to be a whole bunch of mechanics to get the
'processed' result out of the AP - why can't you just use the results[] option
and use that to get the processed bit? If that works, you can throw out a ton
of changes.
Other comments:
- can we add an overloaded createAsyncRequestFuture() that doesn't take a
callable or a timeout? Would just call your version but with null, timeout
(saves a bunch of changes w/o loss of clarity)
- why the switch in mutateRow around if there is a statistic tracker for
returning a value? seems like it simplify the code in the async process too.
- Extra newline in MultiServerCallable
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch,
> HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14715) Add javadocs to DelegatingRetryingCallable
[ https://issues.apache.org/jira/browse/HBASE-14715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988200#comment-14988200 ] Jesse Yates commented on HBASE-14715: - Thanks andrew! > Add javadocs to DelegatingRetryingCallable > -- > > Key: HBASE-14715 > URL: https://issues.apache.org/jira/browse/HBASE-14715 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Jesse Yates >Assignee: Jesse Yates >Priority: Trivial > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16 > > Attachments: hbase-14715-trunk-v0.patch > > > Follow up cleanup from HBASE-9049 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14985952#comment-14985952
]
Jesse Yates commented on HBASE-14703:
-
I think we also need to cover checkAndMutate - it has the same logic as mutate.
Otherwise, yeah, this seems like an improvement. Couldn't we also drop the
StatsTrackingRpcRetryingCaller then as well?
Minor nit, IMO its a bit 'nicer' to use static checking (especially in IDEs
that don't catch strings when doing renames) for the error handling, so
something like:
{code}
} catch(RetriesExhaustedWithDetailsException e) {
for(Throwable rootCause: e.getCauses()){
if(rootCause instanceof NoSuchColumnFamilyException){
return;
}
}
throw e;
{code}
bq. I found two difference point between original code and the patch.
It doesn't seem to far fetch though to add support with another 2 parameters
when doing the submission to the async process. If not, at the very least we
could refactor the mutate/checkAndMutate code to use the same path and fix the
stats collecting there
> not collect stats when call HTable.mutateRow
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch,
> HBASE-14703.patch, HBASE-14703_v1.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-14715) Add javadocs to DelegatingRetryingCallable
[ https://issues.apache.org/jira/browse/HBASE-14715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates resolved HBASE-14715. - Resolution: Fixed Fix Version/s: 2.0.0 Committed to trunk. Thanks for the reviews fellas. > Add javadocs to DelegatingRetryingCallable > -- > > Key: HBASE-14715 > URL: https://issues.apache.org/jira/browse/HBASE-14715 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Jesse Yates >Assignee: Jesse Yates >Priority: Trivial > Fix For: 2.0.0 > > Attachments: hbase-14715-trunk-v0.patch > > > Follow up cleanup from HBASE-9049 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) update the per-region stats twice for the call on return
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982007#comment-14982007
]
Jesse Yates commented on HBASE-14703:
-
Yes, in TestClientPushback. Originally, I was suggesting adding a test case
that uses mocks to test that we don't double-update stats. The purpose of the
original test was ensure we are getting stats back, but it wasn't checking that
we are double updating b/c it wasn't getting in the path of the updates and you
can't tell a double update from a single-update when the pushback value is the
same.
bq. we can't test the situation which client NOT collect server pushback.
Welll, there we would use two different tests - the original one and the new,
mock-based one.
> update the per-region stats twice for the call on return
>
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14703) update the per-region stats twice for the call on return
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesse Yates updated HBASE-14703:
Attachment: HBASE-14703-start.patch
So I was getting confused with the result returned in the client-small-scan
that, I think, is just used to lookup the region; note that this does _not_
update any stats b/c the form of the result is 'wrong' for stats (its an array
of results).
Adding a second put, I see that only the asyncProcess check is used to collect
stats from the result.
The best I can tell is the StatsTrackingRpcRetryingCaller was there to catch
cases where we don't go through the multi() calls.
I think the "right" way to fix this, following the original intent of the code,
is to add support for unwrapping results from the other, non-put calls.
Currently, things like mutateRows _don't return a result_, even though there is
information there. Thus, the right thing to do would seem to be unwrapping that
result and letting the StatsTrackingRpcCaller handle the result's stats as we
would expect.
I'm attaching a patch that would be *starting point* for doing this - we would
need to add support in a couple more places as well.
If you apply this patch and then drop breakpoints in at
RSRpcServices#mutateRows and StatsTrackingRpcRetryingCaller#callWithRetries.
You'll see that the RS does in fact send along the stats information, but
HTable#mutateRow returns only retuns any kind of result after the changes in
the patch.
> update the per-region stats twice for the call on return
>
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-start.patch, HBASE-14703.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14703) update the per-region stats twice for the call on return
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesse Yates updated HBASE-14703:
Status: Open (was: Patch Available)
> update the per-region stats twice for the call on return
>
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-start.patch, HBASE-14703.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14118) minicluster maven plugin
[
https://issues.apache.org/jira/browse/HBASE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980942#comment-14980942
]
Jesse Yates commented on HBASE-14118:
-
Ok, so looking through the [ip clearance
template|http://incubator.apache.org/ip-clearance/ip-clearance-template.html] I
think that having WibiData's CLA *and all active committers have a signed CLA
on record* should be enough.
>From our end, we can check the CLA:
{quote}
Identify name recorded for software grant: the name of the grant as recorded in
the foundation/officers area, in either grants.txt or cclas.txt, so that the
grant can be easily identified
{quote}
However, we need a Chair/Member of ASF to do the actual work ([~apurtell]?
[~stack]?) The rest of the process we need to follow is outlined in steps 5-8
in the [ip clearance
template|http://incubator.apache.org/ip-clearance/ip-clearance-template.html].
Did as much as I could, which was checking the dependencies and starting the
clearance form: https://gist.github.com/jyates/67a54dc8c1f23257c925
Guess we (HBase) need to have a vote as well to accept the code.
> minicluster maven plugin
>
>
> Key: HBASE-14118
> URL: https://issues.apache.org/jira/browse/HBASE-14118
> Project: HBase
> Issue Type: New Feature
> Components: integration tests, util
>Affects Versions: 2.0.0
>Reporter: Jesse Yates
> Attachments:
> hbase-maven-plugin-c444b9b6228327266094df266a99daa4f53958c4.tar.gz
>
>
> In the [failsafe docs
> |https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html] they
> describe being able to startup a jetty server before your integration tests
> and then bring it down after the tests. We should be able to provide the same
> for the mini-cluster for downstream projects and cross-version compatibility
> testing.
> The folks over in Kiji made this great [plugin
> |https://github.com/kijiproject/hbase-maven-plugin] that lets you do just
> that! We should see about bringing that under the HBase umbrella proper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14703) update the per-region stats twice for the call on return
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981065#comment-14981065
]
Jesse Yates commented on HBASE-14703:
-
Ok, looking closer at trunk now - looks like you are right [~chenheng].
Thinking about it, it would be a hairy patch to insert a mock stats into the
connection to check for a bug from a class that will no longer exist.
I can commit if you fellas are happy with it
> update the per-region stats twice for the call on return
>
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-5790) ZKUtil deleteRecursively should be a recoverable operation
[ https://issues.apache.org/jira/browse/HBASE-5790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-5790: --- Assignee: (was: Jesse Yates) > ZKUtil deleteRecursively should be a recoverable operation > -- > > Key: HBASE-5790 > URL: https://issues.apache.org/jira/browse/HBASE-5790 > Project: HBase > Issue Type: Improvement >Reporter: Jesse Yates > Labels: zookeeper > Attachments: java_HBASE-5790-v1.patch, java_HBASE-5790.patch > > > As of 3.4.3 Zookeeper now has full, multi-operation transaction. This means > we can wholesale delete chunks of the zk tree and ensure that we don't have > any pesky recursive delete issues where we delete the children of a node, but > then a child joins before deletion of the parent. Even without transactions, > this should be the behavior, but it is possible to make it much cleaner now > that we have this new feature in zk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-5815) ZKUtil.createWithParents should be transactional
[ https://issues.apache.org/jira/browse/HBASE-5815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-5815: --- Assignee: (was: Jesse Yates) > ZKUtil.createWithParents should be transactional > > > Key: HBASE-5815 > URL: https://issues.apache.org/jira/browse/HBASE-5815 > Project: HBase > Issue Type: Improvement >Reporter: Jesse Yates > Labels: zookeeper > Attachments: java_HBASE-5815-v0.patch > > > Should solve a lot of tricky corner cases when you create the parent, get an > update to watchers (who read ZK, but find no children) and then create the > children. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-6867) Globally consistent snapshots
[ https://issues.apache.org/jira/browse/HBASE-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates resolved HBASE-6867. Resolution: Won't Fix > Globally consistent snapshots > - > > Key: HBASE-6867 > URL: https://issues.apache.org/jira/browse/HBASE-6867 > Project: HBase > Issue Type: New Feature > Components: snapshots >Reporter: Jesse Yates >Assignee: Jesse Yates > > Take a globally consistent snapshot of a table (where global only includes > all regionservers hosting the table). The is one of the two end results of > the 'take a snapshot' jiras. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-7958) Statistics per-column family per-region
[ https://issues.apache.org/jira/browse/HBASE-7958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates resolved HBASE-7958. Resolution: Won't Fix > Statistics per-column family per-region > --- > > Key: HBASE-7958 > URL: https://issues.apache.org/jira/browse/HBASE-7958 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.95.2 >Reporter: Jesse Yates >Assignee: Jesse Yates > Attachments: hbase-7958-v0-parent.patch, hbase-7958-v0.patch, > hbase-7958_rough-cut-v0.patch > > > Originating from this discussion on the dev list: > http://search-hadoop.com/m/coDKU1urovS/Simple+stastics+per+region/v=plain > Essentially, we should have built-in statistics gathering for HBase tables. > This allows clients to have a better understanding of the distribution of > keys within a table and a given region. We could also surface this > information via the UI. > There are a couple different proposals from the email, the overview is this: > We add in something on compactions that gathers stats about the keys that are > written and then we surface them to a table. > The possible proposals include: > *How to implement it?* > # Coprocessors - > ** advantage - it easily plugs in and people could pretty easily add their > own statistics. > ** disadvantage - UI elements would also require this, we get into dependent > loading, which leads down the OSGi path. Also, these CPs need to be installed > _after_ all the other CPs on compaction to ensure they see exactly what gets > written (doable, but a pain) > # Built into HBase as a custom scanner > ** advantage - always goes in the right place and no need to muck about with > loading CPs etc. > ** disadvantage - less pluggable, at least for the initial cut > *Where do we store data?* > # .META. > ** advantage - its an existing table, so we can jam it into another CF there > ** disadvantage - this would make META much larger, possibly leading to > splits AND will make it much harder for other processes to read the info > # A new stats table > ** advantage - cleanly separates out the information from META > ** disadvantage - should use a 'system table' idea to prevent accidental > deletion, manipulation by arbitrary clients, but still allow clients to read > it. > Once we have this framework, we can then move to an actual implementation of > various statistics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-5354) Source to standalone deployment script
[ https://issues.apache.org/jira/browse/HBASE-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates resolved HBASE-5354. Resolution: Won't Fix Seems like this is covered by the docker image. Probably needs updating if someone wants to pick this up later > Source to standalone deployment script > -- > > Key: HBASE-5354 > URL: https://issues.apache.org/jira/browse/HBASE-5354 > Project: HBase > Issue Type: New Feature > Components: build, scripts >Affects Versions: 0.94.0 >Reporter: Jesse Yates >Assignee: Jesse Yates >Priority: Minor > Attachments: bash_HBASE-5354-v0.patch, bash_HBASE-5354-v1.patch, > bash_HBASE-5354.patch > > > Automating the testing of source code in a 'real' instance can be a bit of a > pain, even getting it into standalone mode. > Steps you need to go through: > 1) Build the project > 2) Copy it to the deployment directory > 3) Shutdown the current cluster (if it is running) > 4) Untar the tar > 5) Update the configs to point to a local data cluster > 6) Startup the new deployment > Yeah, its not super difficult, but it would be nice to just have a script to > make it button push easy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) update the per-region stats twice for the call on return
[
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979630#comment-14979630
]
Jesse Yates commented on HBASE-14703:
-
Hmm, I remember this code being a bit tricky with a couple of code paths
needing to be covered to ensure the stats are collected. I would like to see a
test change, probably in TestClientPushback for actually verifying the amount
of pushback we should see, so instead of:
{code}
long backoffTime = backoffPolicy.getBackoffTime(server, regionName,
serverStats);
assertNotEquals("Reported load does not produce a backoff", backoffTime, 0);
{code}
you could mock out the server stats to specify the load it _should_ be getting
and then ensure that the backoff time is the same. If the values are as
expected, then we should be good to go.
I'll take a deeper look at the code tomorrow to see how it all fits together -
right now its way out RAM, in cold storage
> update the per-region stats twice for the call on return
>
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}} by
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
> @Override
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> @Override
> public T callWithoutRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
> }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do
> update again.
> {code}
> // update the stats about the region, if its a user table. We don't want to
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
> result = ResultStatsUtil.updateStats(result,
> AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-9049) Generalize ServerCallable creation to support custom callables
[ https://issues.apache.org/jira/browse/HBASE-9049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979634#comment-14979634 ] Jesse Yates commented on HBASE-9049: [~stack] see HBASE-14715 for follow up on those docs. thanks for keeping me honest > Generalize ServerCallable creation to support custom callables > -- > > Key: HBASE-9049 > URL: https://issues.apache.org/jira/browse/HBASE-9049 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.2, 0.94.11 >Reporter: Jesse Yates >Assignee: Jesse Yates > Attachments: hbase-9049-trunk-v0.patch, hbase-9049-trunk-v1.patch, > hbase-9049-trunk-v2.patch, hbase-9049-trunk-v3.patch, > hbase-9049-trunk-v4.patch > > > Currently, sever callables are instantiated via direct calls. Instead, we can > use a single factory and that allows more specialized callable > implementations, for instance, using a circuit-breaker pattern (or the > Hystrix implementation!) to minimize attempts to contact the server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14715) Add javadocs to DelegatingRetryingCallable
Jesse Yates created HBASE-14715: --- Summary: Add javadocs to DelegatingRetryingCallable Key: HBASE-14715 URL: https://issues.apache.org/jira/browse/HBASE-14715 Project: HBase Issue Type: Improvement Components: Client Reporter: Jesse Yates Assignee: Jesse Yates Priority: Trivial Follow up cleanup from HBASE-9049 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14715) Add javadocs to DelegatingRetryingCallable
[ https://issues.apache.org/jira/browse/HBASE-14715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-14715: Attachment: hbase-14715-trunk-v0.patch Simple patch for DelegatingRetryingCallable + documenting the type param on RetryingCallable because if that's not doc'ed I can't properly make assertions about it, in the delegating version :) > Add javadocs to DelegatingRetryingCallable > -- > > Key: HBASE-14715 > URL: https://issues.apache.org/jira/browse/HBASE-14715 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Jesse Yates >Assignee: Jesse Yates >Priority: Trivial > Attachments: hbase-14715-trunk-v0.patch > > > Follow up cleanup from HBASE-9049 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14118) minicluster maven plugin
[ https://issues.apache.org/jira/browse/HBASE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979581#comment-14979581 ] Jesse Yates commented on HBASE-14118: - Sad face - any time for this [~gwu]? > minicluster maven plugin > > > Key: HBASE-14118 > URL: https://issues.apache.org/jira/browse/HBASE-14118 > Project: HBase > Issue Type: New Feature > Components: integration tests, util >Affects Versions: 2.0.0 >Reporter: Jesse Yates > Attachments: > hbase-maven-plugin-c444b9b6228327266094df266a99daa4f53958c4.tar.gz > > > In the [failsafe docs > |https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html] they > describe being able to startup a jetty server before your integration tests > and then bring it down after the tests. We should be able to provide the same > for the mini-cluster for downstream projects and cross-version compatibility > testing. > The folks over in Kiji made this great [plugin > |https://github.com/kijiproject/hbase-maven-plugin] that lets you do just > that! We should see about bringing that under the HBase umbrella proper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9049) Generalize ServerCallable creation to support custom callables
[ https://issues.apache.org/jira/browse/HBASE-9049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14941883#comment-14941883 ] Jesse Yates commented on HBASE-9049: Thinking about this.. it seems like we might want to keep the class around in the interest of utility for users. If we do, then it should have some sort of docs (even if just minimal) XOR we throw out the class. Yes, its a simple class and anyone could rewrite it, but this saves everyone from rewriting it. I dunno, maybe ppl don't need it? Or the PMC wants a slimmer codebase? What say you mr. [~stack]? > Generalize ServerCallable creation to support custom callables > -- > > Key: HBASE-9049 > URL: https://issues.apache.org/jira/browse/HBASE-9049 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.2, 0.94.11 >Reporter: Jesse Yates >Assignee: Jesse Yates > Attachments: hbase-9049-trunk-v0.patch, hbase-9049-trunk-v1.patch, > hbase-9049-trunk-v2.patch, hbase-9049-trunk-v3.patch, > hbase-9049-trunk-v4.patch > > > Currently, sever callables are instantiated via direct calls. Instead, we can > use a single factory and that allows more specialized callable > implementations, for instance, using a circuit-breaker pattern (or the > Hystrix implementation!) to minimize attempts to contact the server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9049) Generalize ServerCallable creation to support custom callables
[ https://issues.apache.org/jira/browse/HBASE-9049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906245#comment-14906245 ] Jesse Yates commented on HBASE-9049: Yeah, I'm on vacation for a little bit, but can do it when I get back (next week?). I thought DelegatingRetryingCallable would be pretty self explanatory, but if it isn't used, that's an issue. > Generalize ServerCallable creation to support custom callables > -- > > Key: HBASE-9049 > URL: https://issues.apache.org/jira/browse/HBASE-9049 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.2, 0.94.11 >Reporter: Jesse Yates >Assignee: Jesse Yates > Attachments: hbase-9049-trunk-v0.patch, hbase-9049-trunk-v1.patch, > hbase-9049-trunk-v2.patch, hbase-9049-trunk-v3.patch, > hbase-9049-trunk-v4.patch > > > Currently, sever callables are instantiated via direct calls. Instead, we can > use a single factory and that allows more specialized callable > implementations, for instance, using a circuit-breaker pattern (or the > Hystrix implementation!) to minimize attempts to contact the server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13480) ShortCircuitConnection doesn't short-circuit all calls as expected
[
https://issues.apache.org/jira/browse/HBASE-13480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703488#comment-14703488
]
Jesse Yates commented on HBASE-13480:
-
Does marking it deprecated and point ppl to the new method make everyone happy
then? :)
ShortCircuitConnection doesn't short-circuit all calls as expected
--
Key: HBASE-13480
URL: https://issues.apache.org/jira/browse/HBASE-13480
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: Josh Elser
Assignee: Jingcheng Du
Fix For: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.3
Attachments: HBASE-13480.patch
Noticed the following situation in debugging unexpected unit tests failures
in HBASE-13351.
{{ConnectionUtils#createShortCircuitHConnection(Connection, ServerName,
AdminService.BlockingInterface, ClientService.BlockingInterface)}} is
intended to avoid the extra RPC by calling the server's instantiation of the
protobuf rpc stub directly for the AdminService and ClientService.
The problem is that this is insufficient to actually avoid extra remote
RPCs as all other calls to the Connection are routed to a real Connection
instance. As such, any object created by the real Connection (such as an
HTable) will use the real Connection, not the SSC.
The end result is that
{{MasterRpcService#reportRegionStateTransition(RpcController,
ReportRegionStateTransitionRequest)}} will make additional remote RPCs over
what it thinks is an SSC through a {{Get}} on {{HTable}} which was
constructed using the SSC, but the {{Get}} itself will use the underlying
real Connection instead of the SSC. With insufficiently sized thread pools,
this has been observed to result in RPC deadlock in the HMaster where an RPC
attempts to make another RPC but there are no more threads available to
service the second RPC so the first RPC blocks indefinitely.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13480) ShortCircuitConnection doesn't short-circuit all calls as expected
[
https://issues.apache.org/jira/browse/HBASE-13480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702466#comment-14702466
]
Jesse Yates commented on HBASE-13480:
-
I'm not very surprised this has been causing issues in the master - it was
originally intended just a regionserver local thing only (and as such, any
calls to the master were expected to go via an RPC) and wanted a minimum impact
change. Anyways, this seems to be a more general approach, which is nice :).
I'm not concerned about changing the signature [~lhofhansl] - the class is
already marked InterfaceAudience.Private, so any changes are to our discretion,
no? I would like to see tests around the changes to ensure we don't get into
this situation again though. Otherwise, LGTM
ShortCircuitConnection doesn't short-circuit all calls as expected
--
Key: HBASE-13480
URL: https://issues.apache.org/jira/browse/HBASE-13480
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: Josh Elser
Assignee: Jingcheng Du
Fix For: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.3
Attachments: HBASE-13480.patch
Noticed the following situation in debugging unexpected unit tests failures
in HBASE-13351.
{{ConnectionUtils#createShortCircuitHConnection(Connection, ServerName,
AdminService.BlockingInterface, ClientService.BlockingInterface)}} is
intended to avoid the extra RPC by calling the server's instantiation of the
protobuf rpc stub directly for the AdminService and ClientService.
The problem is that this is insufficient to actually avoid extra remote
RPCs as all other calls to the Connection are routed to a real Connection
instance. As such, any object created by the real Connection (such as an
HTable) will use the real Connection, not the SSC.
The end result is that
{{MasterRpcService#reportRegionStateTransition(RpcController,
ReportRegionStateTransitionRequest)}} will make additional remote RPCs over
what it thinks is an SSC through a {{Get}} on {{HTable}} which was
constructed using the SSC, but the {{Get}} itself will use the underlying
real Connection instead of the SSC. With insufficiently sized thread pools,
this has been observed to result in RPC deadlock in the HMaster where an RPC
attempts to make another RPC but there are no more threads available to
service the second RPC so the first RPC blocks indefinitely.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14118) minicluster maven plugin
[ https://issues.apache.org/jira/browse/HBASE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631626#comment-14631626 ] Jesse Yates commented on HBASE-14118: - The other reason to bring it into HBase is that the hbase-maven-plugin hasn't seen any activity in the last year or so, whereas I think the HBase folks will give it more love than its seen in a while. Talked with [~gwu] offline (one of the committers there) and seemed receptive to bringing it in. My biggest question is around licensing; its already Apache 2.0 licensed, what do we need to do to bring it into HBase (besides fixing it up to work on trunk)? [~apurtell] [~stack]? minicluster maven plugin Key: HBASE-14118 URL: https://issues.apache.org/jira/browse/HBASE-14118 Project: HBase Issue Type: New Feature Components: integration tests, util Affects Versions: 2.0.0 Reporter: Jesse Yates In the [failsafe docs |https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html] they describe being able to startup a jetty server before your integration tests and then bring it down after the tests. We should be able to provide the same for the mini-cluster for downstream projects and cross-version compatibility testing. The folks over in Kiji made this great [plugin |https://github.com/kijiproject/hbase-maven-plugin] that lets you do just that! We should see about bringing that under the HBase umbrella proper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14118) minicluster maven plugin
Jesse Yates created HBASE-14118: --- Summary: minicluster maven plugin Key: HBASE-14118 URL: https://issues.apache.org/jira/browse/HBASE-14118 Project: HBase Issue Type: New Feature Components: integration tests, util Affects Versions: 2.0.0 Reporter: Jesse Yates In the [failsafe docs |https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html] they describe being able to startup a jetty server before your integration tests and then bring it down after the tests. We should be able to provide the same for the mini-cluster for downstream projects and cross-version compatibility testing. The folks over in Kiji made this great [plugin |https://github.com/kijiproject/hbase-maven-plugin] that lets you do just that! We should see about bringing that under the HBase umbrella proper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14118) minicluster maven plugin
[ https://issues.apache.org/jira/browse/HBASE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631661#comment-14631661 ] Jesse Yates commented on HBASE-14118: - Basic Cross version compat testing? So you could test a new version of HBase against the minicluster from the plugin in the last version. At least, that's all I can think of right now. But still would dramatically help us in cutting down non-destructive, integration test times. For downstreamers, like Phoenix, they wouldn't need to spin up/down a minicluster for every test (which ends up being a majority of the time spent in the test). It just seems like something the hbase project should own too :) minicluster maven plugin Key: HBASE-14118 URL: https://issues.apache.org/jira/browse/HBASE-14118 Project: HBase Issue Type: New Feature Components: integration tests, util Affects Versions: 2.0.0 Reporter: Jesse Yates In the [failsafe docs |https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html] they describe being able to startup a jetty server before your integration tests and then bring it down after the tests. We should be able to provide the same for the mini-cluster for downstream projects and cross-version compatibility testing. The folks over in Kiji made this great [plugin |https://github.com/kijiproject/hbase-maven-plugin] that lets you do just that! We should see about bringing that under the HBase umbrella proper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14118) minicluster maven plugin
[ https://issues.apache.org/jira/browse/HBASE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631735#comment-14631735 ] Jesse Yates commented on HBASE-14118: - bq. Adding support, -Dminicluster.verson or whatever, for launching earlier or later releases when running integration tests, for checking forwards and backwards client compat. I think with this, you would actually just specify the version of the plugin to match the version of the minicluster you want to run, since we would release the plugin alongside the rest of the codebase. If we didn't have a minicluster plugin for that version, you are SOL (or we could back port it), but otherwise it will run that version of the HBase code. minicluster maven plugin Key: HBASE-14118 URL: https://issues.apache.org/jira/browse/HBASE-14118 Project: HBase Issue Type: New Feature Components: integration tests, util Affects Versions: 2.0.0 Reporter: Jesse Yates In the [failsafe docs |https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html] they describe being able to startup a jetty server before your integration tests and then bring it down after the tests. We should be able to provide the same for the mini-cluster for downstream projects and cross-version compatibility testing. The folks over in Kiji made this great [plugin |https://github.com/kijiproject/hbase-maven-plugin] that lets you do just that! We should see about bringing that under the HBase umbrella proper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14118) minicluster maven plugin
[ https://issues.apache.org/jira/browse/HBASE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-14118: Attachment: hbase-maven-plugin-c444b9b6228327266094df266a99daa4f53958c4.tar.gz Tarball of hbase-maven-plugin (with the commit ID from whence it came). minicluster maven plugin Key: HBASE-14118 URL: https://issues.apache.org/jira/browse/HBASE-14118 Project: HBase Issue Type: New Feature Components: integration tests, util Affects Versions: 2.0.0 Reporter: Jesse Yates Attachments: hbase-maven-plugin-c444b9b6228327266094df266a99daa4f53958c4.tar.gz In the [failsafe docs |https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html] they describe being able to startup a jetty server before your integration tests and then bring it down after the tests. We should be able to provide the same for the mini-cluster for downstream projects and cross-version compatibility testing. The folks over in Kiji made this great [plugin |https://github.com/kijiproject/hbase-maven-plugin] that lets you do just that! We should see about bringing that under the HBase umbrella proper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14118) minicluster maven plugin
[ https://issues.apache.org/jira/browse/HBASE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631730#comment-14631730 ] Jesse Yates commented on HBASE-14118: - Ah, ok. Sorry [~apurtell], should have RTFS. thanks! minicluster maven plugin Key: HBASE-14118 URL: https://issues.apache.org/jira/browse/HBASE-14118 Project: HBase Issue Type: New Feature Components: integration tests, util Affects Versions: 2.0.0 Reporter: Jesse Yates In the [failsafe docs |https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html] they describe being able to startup a jetty server before your integration tests and then bring it down after the tests. We should be able to provide the same for the mini-cluster for downstream projects and cross-version compatibility testing. The folks over in Kiji made this great [plugin |https://github.com/kijiproject/hbase-maven-plugin] that lets you do just that! We should see about bringing that under the HBase umbrella proper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14118) minicluster maven plugin
[ https://issues.apache.org/jira/browse/HBASE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631729#comment-14631729 ] Jesse Yates edited comment on HBASE-14118 at 7/17/15 6:48 PM: -- FWIW, spent a little bit bringing the plugin up to trunk ([github|https://github.com/jyates/hbase/tree/add-minicluster-plugin-14118]); well, with just hadoop 2.5.1 support, but not sure how we do version support anyways, so that will do the moment. Also, brought up my [next improvements|https://github.com/jyates/hbase/tree/add-plugin-improvements] to it that allow you to do things like running mvn org.apache.hbase:hbase-maven-plugin:run to bring up (and leave up) mini-cluster was (Author: jesse_yates): FWIW, spent a little bit bringing the plugin up to trunk ([github|https://github.com/jyates/hbase/tree/add-minicluster-plugin-14118]); well, with just hadoop 2.5.1 support, but not sure how we do version support anyways, so that will do the moment. Also, brought up my [next improvements|https://github.com/jyates/hbase/tree/add-minicluster-plugin-14118] to it that allow you to do things like running mvn org.apache.hbase:hbase-maven-plugin:run to bring up (and leave up) mini-cluster minicluster maven plugin Key: HBASE-14118 URL: https://issues.apache.org/jira/browse/HBASE-14118 Project: HBase Issue Type: New Feature Components: integration tests, util Affects Versions: 2.0.0 Reporter: Jesse Yates In the [failsafe docs |https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html] they describe being able to startup a jetty server before your integration tests and then bring it down after the tests. We should be able to provide the same for the mini-cluster for downstream projects and cross-version compatibility testing. The folks over in Kiji made this great [plugin |https://github.com/kijiproject/hbase-maven-plugin] that lets you do just that! We should see about bringing that under the HBase umbrella proper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14118) minicluster maven plugin
[ https://issues.apache.org/jira/browse/HBASE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631723#comment-14631723 ] Jesse Yates commented on HBASE-14118: - Do we need to bring it into the incubator? That seems a bit much for what is really not that much code (few hundred lines). I was hoping it would just fall in as a new module in HBase. If we need to, then we need to... and I'd be +1 for that too. minicluster maven plugin Key: HBASE-14118 URL: https://issues.apache.org/jira/browse/HBASE-14118 Project: HBase Issue Type: New Feature Components: integration tests, util Affects Versions: 2.0.0 Reporter: Jesse Yates In the [failsafe docs |https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html] they describe being able to startup a jetty server before your integration tests and then bring it down after the tests. We should be able to provide the same for the mini-cluster for downstream projects and cross-version compatibility testing. The folks over in Kiji made this great [plugin |https://github.com/kijiproject/hbase-maven-plugin] that lets you do just that! We should see about bringing that under the HBase umbrella proper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14118) minicluster maven plugin
[ https://issues.apache.org/jira/browse/HBASE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631729#comment-14631729 ] Jesse Yates commented on HBASE-14118: - FWIW, spent a little bit bringing the plugin up to trunk ([github|https://github.com/jyates/hbase/tree/add-minicluster-plugin-14118]); well, with just hadoop 2.5.1 support, but not sure how we do version support anyways, so that will do the moment. Also, brought up my [next improvements|https://github.com/jyates/hbase/tree/add-minicluster-plugin-14118] to it that allow you to do things like running mvn org.apache.hbase:hbase-maven-plugin:run to bring up (and leave up) mini-cluster minicluster maven plugin Key: HBASE-14118 URL: https://issues.apache.org/jira/browse/HBASE-14118 Project: HBase Issue Type: New Feature Components: integration tests, util Affects Versions: 2.0.0 Reporter: Jesse Yates In the [failsafe docs |https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html] they describe being able to startup a jetty server before your integration tests and then bring it down after the tests. We should be able to provide the same for the mini-cluster for downstream projects and cross-version compatibility testing. The folks over in Kiji made this great [plugin |https://github.com/kijiproject/hbase-maven-plugin] that lets you do just that! We should see about bringing that under the HBase umbrella proper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13885) ZK watches leaks during snapshots
[ https://issues.apache.org/jira/browse/HBASE-13885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587062#comment-14587062 ] Jesse Yates commented on HBASE-13885: - Seems reasonable to me :) ZK watches leaks during snapshots - Key: HBASE-13885 URL: https://issues.apache.org/jira/browse/HBASE-13885 Project: HBase Issue Type: Bug Components: snapshots Affects Versions: 0.98.12 Reporter: Abhishek Singh Chouhan Assignee: Lars Hofhansl Priority: Critical Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 13885-0.98-v2.txt, 13885-0.98-v3.txt, 13885-master.txt When taking snapshot of a table a watcher over /hbase/online-snapshot/abort/snapshot-name is created which is never cleared when the snapshot is successful. If we use snapshots to take backups daily we accumulate a lot of watches. Steps to reproduce - 1) Take snapshot of a table - snapshot 'table_1', 'abc' 2) Run the following on zk node or alternatively observe zk watches metric echo wchc | nc localhost 2181 /hbase/online-snapshot/abort/abc can be found. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13846) Run MiniCluster on top of other MiniDfsCluster
[ https://issues.apache.org/jira/browse/HBASE-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-13846: Resolution: Fixed Status: Resolved (was: Patch Available) Committed to 0.98, branch-1, branch-1.1, and master. Thanks for the reviews! Run MiniCluster on top of other MiniDfsCluster -- Key: HBASE-13846 URL: https://issues.apache.org/jira/browse/HBASE-13846 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Minor Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Attachments: hbase-13846-0.98-v0.patch, hbase-13846-master-v0.patch, hbase-13846-master-v1.patch Similar to how we don't start a mini-zk cluster when we already have one specified, this will skip starting a mini-dfs cluster if the user specifies a different one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13846) Run MiniCluster on top of other MiniDfsCluster
[ https://issues.apache.org/jira/browse/HBASE-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14579492#comment-14579492 ] Jesse Yates commented on HBASE-13846: - Thanks for taking a look Ted! I assume otherwise, you are +1? Posting another patch momentarily; if QA is happy, I'll commit unless I hear otherwise. Run MiniCluster on top of other MiniDfsCluster -- Key: HBASE-13846 URL: https://issues.apache.org/jira/browse/HBASE-13846 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Minor Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Attachments: hbase-13846-0.98-v0.patch, hbase-13846-master-v0.patch Similar to how we don't start a mini-zk cluster when we already have one specified, this will skip starting a mini-dfs cluster if the user specifies a different one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13846) Run MiniCluster on top of other MiniDfsCluster
[ https://issues.apache.org/jira/browse/HBASE-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-13846: Attachment: hbase-13846-master-v1.patch Updated master version - fixing javadoc, extra imports header. Run MiniCluster on top of other MiniDfsCluster -- Key: HBASE-13846 URL: https://issues.apache.org/jira/browse/HBASE-13846 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Minor Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Attachments: hbase-13846-0.98-v0.patch, hbase-13846-master-v0.patch, hbase-13846-master-v1.patch Similar to how we don't start a mini-zk cluster when we already have one specified, this will skip starting a mini-dfs cluster if the user specifies a different one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13846) Run MiniCluster on top of other MiniDfsCluster
[ https://issues.apache.org/jira/browse/HBASE-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-13846: Status: Patch Available (was: Open) Run MiniCluster on top of other MiniDfsCluster -- Key: HBASE-13846 URL: https://issues.apache.org/jira/browse/HBASE-13846 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Minor Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Attachments: hbase-13846-0.98-v0.patch, hbase-13846-master-v0.patch, hbase-13846-master-v1.patch Similar to how we don't start a mini-zk cluster when we already have one specified, this will skip starting a mini-dfs cluster if the user specifies a different one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13846) Run MiniCluster on top of other MiniDfsCluster
[ https://issues.apache.org/jira/browse/HBASE-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-13846: Status: Open (was: Patch Available) Run MiniCluster on top of other MiniDfsCluster -- Key: HBASE-13846 URL: https://issues.apache.org/jira/browse/HBASE-13846 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Minor Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Attachments: hbase-13846-0.98-v0.patch, hbase-13846-master-v0.patch, hbase-13846-master-v1.patch Similar to how we don't start a mini-zk cluster when we already have one specified, this will skip starting a mini-dfs cluster if the user specifies a different one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13846) Run MiniCluster on top of other MiniDfsCluster
[ https://issues.apache.org/jira/browse/HBASE-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-13846: Attachment: hbase-13846-0.98-v0.patch Patch on 0.98 (to support internal use cases). I'll whip up something for trunk and 1.1 shortly. Run MiniCluster on top of other MiniDfsCluster -- Key: HBASE-13846 URL: https://issues.apache.org/jira/browse/HBASE-13846 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.98.14, 1.2.0, 1.1.1 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Minor Attachments: hbase-13846-0.98-v0.patch Similar to how we don't start a mini-zk cluster when we already have one specified, this will skip starting a mini-dfs cluster if the user specifies a different one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13846) Run MiniCluster on top of other MiniDfsCluster
Jesse Yates created HBASE-13846: --- Summary: Run MiniCluster on top of other MiniDfsCluster Key: HBASE-13846 URL: https://issues.apache.org/jira/browse/HBASE-13846 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.98.14, 1.2.0, 1.1.1 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Minor Similar to how we don't start a mini-zk cluster when we already have one specified, this will skip starting a mini-dfs cluster if the user specifies a different one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13846) Run MiniCluster on top of other MiniDfsCluster
[ https://issues.apache.org/jira/browse/HBASE-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-13846: Fix Version/s: 1.1.1 1.2.0 0.98.14 2.0.0 Status: Patch Available (was: Open) Run MiniCluster on top of other MiniDfsCluster -- Key: HBASE-13846 URL: https://issues.apache.org/jira/browse/HBASE-13846 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Minor Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Attachments: hbase-13846-0.98-v0.patch, hbase-13846-master-v0.patch Similar to how we don't start a mini-zk cluster when we already have one specified, this will skip starting a mini-dfs cluster if the user specifies a different one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13846) Run MiniCluster on top of other MiniDfsCluster
[ https://issues.apache.org/jira/browse/HBASE-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-13846: Affects Version/s: 2.0.0 Run MiniCluster on top of other MiniDfsCluster -- Key: HBASE-13846 URL: https://issues.apache.org/jira/browse/HBASE-13846 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Minor Attachments: hbase-13846-0.98-v0.patch Similar to how we don't start a mini-zk cluster when we already have one specified, this will skip starting a mini-dfs cluster if the user specifies a different one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13846) Run MiniCluster on top of other MiniDfsCluster
[ https://issues.apache.org/jira/browse/HBASE-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-13846: Attachment: hbase-13846-master-v0.patch Patch for master. Same applies cleanly to branch-1 and branch-1.1 Run MiniCluster on top of other MiniDfsCluster -- Key: HBASE-13846 URL: https://issues.apache.org/jira/browse/HBASE-13846 Project: HBase Issue Type: Improvement Components: test Affects Versions: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Minor Attachments: hbase-13846-0.98-v0.patch, hbase-13846-master-v0.patch Similar to how we don't start a mini-zk cluster when we already have one specified, this will skip starting a mini-dfs cluster if the user specifies a different one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-4879) Add Constraint support to shell
[
https://issues.apache.org/jira/browse/HBASE-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498347#comment-14498347
]
Jesse Yates commented on HBASE-4879:
Hmm, I'm not sure when they got put under the private annotation. Indeed was
meant to be something used by users, not just limited to internal use (of which
there is none, AFAIK).
bq. Should this not be accessible, at least LimitedPrivate? But is it complete
enough to be used?
Yes, and yes. The tests are still running and passing (e.g.
TestConstraint.java), so constraints at least work. So it can be used, with
some caveats.
It has two main issues: (1) you still have to deploy a jar to the server that
contains the constraint (ala coprocessors or filters), and (2) its not useful
for many cases because as the docs say:
bq. Further, {@link Constraint Constraints} should probably not be used to
enforce cross-table references as it will cause tremendous write slowdowns, but
it is possible.
Obviously, there is no way around the second one, and the interface actually
goes out of its way to not allow you access to an HTable, to prevent this kind
of thing.
However, the flip side is that you have very limited utility. It becomes useful
for toy examples, but in real life its really limited to use. I don't know of
any users actively using constraints (though there was some interest when they
first came out).
Things that could be added to increase utility are: a limited HTable interface
that only accesses the local region and just for reads; a basic set of
constraint processors that uses the table properties to configure certain
constraints (e.g. all values must remain 0); easy shell support.
Your call if you want to include it - they can be useful, but no one has
actually used them :) I'm happy to write code/do reviews/commit improvements,
if people actually want them, but until there is an actual user who wants them,
I'm reluctant to put any more work into it.
Add Constraint support to shell
---
Key: HBASE-4879
URL: https://issues.apache.org/jira/browse/HBASE-4879
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: Jesse Yates
Follow-on ticket to HBASE-4605. Extend the shell functionality to include
altering a table to add a Constraint.
Discussion around this can be found at:
http://search-hadoop.com/m/EeYb3dezMM
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-4879) Add Constraint support to shell
[ https://issues.apache.org/jira/browse/HBASE-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14494809#comment-14494809 ] Jesse Yates commented on HBASE-4879: [~stack], you mean for the constraint stuff in general? The shell support never got finished... Add Constraint support to shell --- Key: HBASE-4879 URL: https://issues.apache.org/jira/browse/HBASE-4879 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Jesse Yates Follow-on ticket to HBASE-4605. Extend the shell functionality to include altering a table to add a Constraint. Discussion around this can be found at: http://search-hadoop.com/m/EeYb3dezMM -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13270) Setter for Result#getStats is #addResults; confusing!
[
https://issues.apache.org/jira/browse/HBASE-13270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370964#comment-14370964
]
Jesse Yates commented on HBASE-13270:
-
I think the original naming came from the fact that the method calling
addResults is managing multiple Results, or maybe its adding the result load as
a result of the last requestsneither is necessarily a good reason, but
that's all I can think of :(
[~larsgeorge], generally I agree with your style of naming as well;
setStatistics()/getStatistics() would certainly be better in this case, but
this was to be the foundation of a more widespread statistics mechanism, so
setRegionLoadStatistics() didn't make as much sense in the longer view. Not
sure why I got lazy this time.
Sorry about this fellas - thanks for the cleanup.
+1 on the patch.
bq. RegionLoadStats don't belong in Result
There really wasn't a better place to put it, well, without completely
re-architecting the client (again). It was expedient, got the data where it
needed to go - back into the client's rpc caller. Happy to talk about how we
can do it better :)
Setter for Result#getStats is #addResults; confusing!
-
Key: HBASE-13270
URL: https://issues.apache.org/jira/browse/HBASE-13270
Project: HBase
Issue Type: Improvement
Reporter: stack
Labels: beginner
Attachments: HBASE-13270.patch
Below is our [~larsgeorge] on a finding he made reviewing our API:
Result class having getStats() and addResults(Stats) makes little sense...
...the naming is just weird. You have a getStats() getter and an
addResults(Stats) setter???
...Especially in the Result class and addResult() is plain misleading...
This issue is about deprecating addResults and replacing it with addStats in
its place.
The getStats/addResult is recent. It came in with:
{code}
commit a411227b0ebf78b4ee8ae7179e162b54734e77de
Author: Jesse Yates jesse.k.ya...@gmail.com
Date: Tue Oct 28 16:14:16 2014 -0700
HBASE-5162 Basic client pushback mechanism
...
{code}
RegionLoadStats don't belong in Result if you ask me but better in the
enveloping on invocations... but that is another issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13270) Setter for Result#getStats is #addResults; confusing!
[
https://issues.apache.org/jira/browse/HBASE-13270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370977#comment-14370977
]
Jesse Yates commented on HBASE-13270:
-
bq. setStatistics() sounds about right?
Yup!
Setter for Result#getStats is #addResults; confusing!
-
Key: HBASE-13270
URL: https://issues.apache.org/jira/browse/HBASE-13270
Project: HBase
Issue Type: Improvement
Reporter: stack
Labels: beginner
Attachments: HBASE-13270.patch
Below is our [~larsgeorge] on a finding he made reviewing our API:
Result class having getStats() and addResults(Stats) makes little sense...
...the naming is just weird. You have a getStats() getter and an
addResults(Stats) setter???
...Especially in the Result class and addResult() is plain misleading...
This issue is about deprecating addResults and replacing it with addStats in
its place.
The getStats/addResult is recent. It came in with:
{code}
commit a411227b0ebf78b4ee8ae7179e162b54734e77de
Author: Jesse Yates jesse.k.ya...@gmail.com
Date: Tue Oct 28 16:14:16 2014 -0700
HBASE-5162 Basic client pushback mechanism
...
{code}
RegionLoadStats don't belong in Result if you ask me but better in the
enveloping on invocations... but that is another issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13031) Ability to snapshot based on a key range
[ https://issues.apache.org/jira/browse/HBASE-13031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319171#comment-14319171 ] Jesse Yates commented on HBASE-13031: - The problem I saw with that is they (flurry) don't want to keep around an entire second copy of the table if there is a compaction happening which does bring up the question of, what good is a snapshot if it doesn't capture the state of an entire table? If you have writes happening at the same time (which causes the compaction) then the partial snapshots will never be 'up to date'. Ability to snapshot based on a key range Key: HBASE-13031 URL: https://issues.apache.org/jira/browse/HBASE-13031 Project: HBase Issue Type: Improvement Reporter: churro morales Assignee: churro morales Fix For: 2.0.0, 0.94.26, 1.1.0, 0.98.11 Posted on the mailing list and seems like some people are interested. A little background for everyone. We have a very large table, we would like to snapshot and transfer the data to another cluster (compressed data is always better to ship). Our problem lies in the fact it could take many weeks to transfer all of the data and during that time with major compactions, the data stored in dfs has the potential to double which would cause us to run out of disk space. So we were thinking about allowing the ability to snapshot a specific key range. Ideally I feel the approach is that the user would specify a start and stop key, those would be associated with a region boundary. If between the time the user submits the request and the snapshot is taken the boundaries change (due to merging or splitting of regions) the snapshot should fail. We would know which regions to snapshot and if those changed between when the request was submitted and the regions locked, the snapshot could simply fail and the user would try again, instead of potentially giving the user more / less than what they had anticipated. I was planning on storing the start / stop key in the SnapshotDescription and from there it looks pretty straight forward where we just have to change the verifier code to accommodate the key ranges. If this design sounds good to anyone, or if I am overlooking anything please let me know. Once we agree on the design, I'll write and submit the patches. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-13031) Ability to snapshot based on a key range
[ https://issues.apache.org/jira/browse/HBASE-13031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319171#comment-14319171 ] Jesse Yates edited comment on HBASE-13031 at 2/12/15 10:58 PM: --- The problem I saw with that is they (flurry) don't want to keep around an entire second copy of the table if there is a compaction happening which does bring up the question of, what good is a snapshot if it doesn't capture the state of an entire table? If you have writes happening at the same time (which causes the compaction) then the partial snapshots will never be 'up to date'. or, what Andrew said :) was (Author: jesse_yates): The problem I saw with that is they (flurry) don't want to keep around an entire second copy of the table if there is a compaction happening which does bring up the question of, what good is a snapshot if it doesn't capture the state of an entire table? If you have writes happening at the same time (which causes the compaction) then the partial snapshots will never be 'up to date'. Ability to snapshot based on a key range Key: HBASE-13031 URL: https://issues.apache.org/jira/browse/HBASE-13031 Project: HBase Issue Type: Improvement Reporter: churro morales Assignee: churro morales Fix For: 2.0.0, 0.94.26, 1.1.0, 0.98.11 Posted on the mailing list and seems like some people are interested. A little background for everyone. We have a very large table, we would like to snapshot and transfer the data to another cluster (compressed data is always better to ship). Our problem lies in the fact it could take many weeks to transfer all of the data and during that time with major compactions, the data stored in dfs has the potential to double which would cause us to run out of disk space. So we were thinking about allowing the ability to snapshot a specific key range. Ideally I feel the approach is that the user would specify a start and stop key, those would be associated with a region boundary. If between the time the user submits the request and the snapshot is taken the boundaries change (due to merging or splitting of regions) the snapshot should fail. We would know which regions to snapshot and if those changed between when the request was submitted and the regions locked, the snapshot could simply fail and the user would try again, instead of potentially giving the user more / less than what they had anticipated. I was planning on storing the start / stop key in the SnapshotDescription and from there it looks pretty straight forward where we just have to change the verifier code to accommodate the key ranges. If this design sounds good to anyone, or if I am overlooking anything please let me know. Once we agree on the design, I'll write and submit the patches. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13031) Ability to snapshot based on a key range
[ https://issues.apache.org/jira/browse/HBASE-13031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318944#comment-14318944 ] Jesse Yates commented on HBASE-13031: - +1 seems like a reasonable approach. Ability to snapshot based on a key range Key: HBASE-13031 URL: https://issues.apache.org/jira/browse/HBASE-13031 Project: HBase Issue Type: Brainstorming Affects Versions: 2.0.0, 0.94.26, 1.1.0, 0.98.11 Reporter: churro morales Assignee: churro morales Priority: Critical Posted on the mailing list and seems like some people are interested. A little background for everyone. We have a very large table, we would like to snapshot and transfer the data to another cluster (compressed data is always better to ship). Our problem lies in the fact it could take many weeks to transfer all of the data and during that time with major compactions, the data stored in dfs has the potential to double which would cause us to run out of disk space. So we were thinking about allowing the ability to snapshot a specific key range. Ideally I feel the approach is that the user would specify a start and stop key, those would be associated with a region boundary. If between the time the user submits the request and the snapshot is taken the boundaries change (due to merging or splitting of regions) the snapshot should fail. We would know which regions to snapshot and if those changed between when the request was submitted and the regions locked, the snapshot could simply fail and the user would try again, instead of potentially giving the user more / less than what they had anticipated. I was planning on storing the start / stop key in the SnapshotDescription and from there it looks pretty straight forward where we just have to change the verifier code to accommodate the key ranges. If this design sounds good to anyone, or if I am overlooking anything please let me know. Once we agree on the design, I'll write and submit the patches. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12979) Use setters instead of return values for handing back statistics from HRegion methods
[ https://issues.apache.org/jira/browse/HBASE-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-12979: Resolution: Fixed Status: Resolved (was: Patch Available) Use setters instead of return values for handing back statistics from HRegion methods - Key: HBASE-12979 URL: https://issues.apache.org/jira/browse/HBASE-12979 Project: HBase Issue Type: Improvement Affects Versions: 0.98.10 Reporter: Andrew Purtell Assignee: Jesse Yates Labels: phoenix Fix For: 1.0.0, 2.0.0, 1.1.0, 0.98.11, 0.98.10.1 Attachments: hbase-12979-v0-0.98.patch, hbase-12979-v0-master.patch In HBASE-5162 (and backports such as HBASE-12729) we modified some HRegion methods to return statistics for consumption by callers. The statistics are ultimately passed back to the client as load feedback. [~lhofhansl] thinks handing back this information as return values from HRegion methods is a weird mix of concerns. This also produced a difficult to anticipate binary compatibility issue with Phoenix. There was no compile time issue because the code of course was not structured to assign from a method returning void, yet the method signature changes so the JVM cannot resolve it if older Phoenix binaries are installed into a 0.98.10 release. Let's change the HRegion methods back to returning 'void' and use setters instead. Officially we don't support use of HRegion (HBASE-12566) but we do not need to go out of our way to break things (smile) so I would also like to make a patch release containing just this change to help out our sister project. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12979) Use setters instead of return values for handing back statistics from HRegion methods
[ https://issues.apache.org/jira/browse/HBASE-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14309619#comment-14309619 ] Jesse Yates commented on HBASE-12979: - Ah, that's the doc I was looking for! All I could find was the Java3 spec. So I think the above patch is fine... I'll see if we can get a test run going then. Use setters instead of return values for handing back statistics from HRegion methods - Key: HBASE-12979 URL: https://issues.apache.org/jira/browse/HBASE-12979 Project: HBase Issue Type: Improvement Affects Versions: 0.98.10 Reporter: Andrew Purtell Assignee: Jesse Yates Labels: phoenix Fix For: 0.98.10.1 Attachments: hbase-12979-v0-master.patch In HBASE-5162 (and backports such as HBASE-12729) we modified some HRegion methods to return statistics for consumption by callers. The statistics are ultimately passed back to the client as load feedback. [~lhofhansl] thinks handing back this information as return values from HRegion methods is a weird mix of concerns. This also produced a difficult to anticipate binary compatibility issue with Phoenix. There was no compile time issue because the code of course was not structured to assign from a method returning void, yet the method signature changes so the JVM cannot resolve it if older Phoenix binaries are installed into a 0.98.10 release. Let's change the HRegion methods back to returning 'void' and use setters instead. Officially we don't support use of HRegion (HBASE-12566) but we do not need to go out of our way to break things (smile) so I would also like to make a patch release containing just this change to help out our sister project. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12979) Use setters instead of return values for handing back statistics from HRegion methods
[ https://issues.apache.org/jira/browse/HBASE-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-12979: Status: Patch Available (was: Open) Use setters instead of return values for handing back statistics from HRegion methods - Key: HBASE-12979 URL: https://issues.apache.org/jira/browse/HBASE-12979 Project: HBase Issue Type: Improvement Affects Versions: 0.98.10 Reporter: Andrew Purtell Assignee: Jesse Yates Labels: phoenix Fix For: 0.98.10.1 Attachments: hbase-12979-v0-master.patch In HBASE-5162 (and backports such as HBASE-12729) we modified some HRegion methods to return statistics for consumption by callers. The statistics are ultimately passed back to the client as load feedback. [~lhofhansl] thinks handing back this information as return values from HRegion methods is a weird mix of concerns. This also produced a difficult to anticipate binary compatibility issue with Phoenix. There was no compile time issue because the code of course was not structured to assign from a method returning void, yet the method signature changes so the JVM cannot resolve it if older Phoenix binaries are installed into a 0.98.10 release. Let's change the HRegion methods back to returning 'void' and use setters instead. Officially we don't support use of HRegion (HBASE-12566) but we do not need to go out of our way to break things (smile) so I would also like to make a patch release containing just this change to help out our sister project. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
