[jira] [Comment Edited] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471505#comment-16471505 ] stack edited comment on HBASE-20411 at 5/11/18 5:24 AM: .014 Address [~mdrob] feedback and rebase. was (Author: stack): .014 Address [~mdrob] feedback. > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > HBASE-20411.branch-2.0.011.patch, HBASE-20411.branch-2.0.012.patch, > HBASE-20411.branch-2.0.013.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471505#comment-16471505 ] stack commented on HBASE-20411: --- .014 Address [~mdrob] feedback. > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > HBASE-20411.branch-2.0.011.patch, HBASE-20411.branch-2.0.012.patch, > HBASE-20411.branch-2.0.013.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: HBASE-20411.branch-2.0.013.patch > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg, > 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg, > 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch, > HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch, > HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch, > HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch, > HBASE-20411.branch-2.0.011.patch, HBASE-20411.branch-2.0.012.patch, > HBASE-20411.branch-2.0.013.patch, jmc6.write_time_locks.png > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471503#comment-16471503
]
Nihal Jain edited comment on HBASE-20566 at 5/11/18 5:16 AM:
-
We can handle this by adding the logic to add system tables to appropriate
rsgroup in *preCreateTableAction()*
was (Author: nihaljain.cs):
We can handle this by adding the logic to add system tables to appropriate **
rsgroup ** in *preCreateTableAction()*
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471503#comment-16471503
]
Nihal Jain commented on HBASE-20566:
We can handle this by adding the logic to add system tables to appropriate **
rsgroup ** in *preCreateTableAction()*
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471482#comment-16471482
]
Appy commented on HBASE-20567:
--
I don't think we can get this into 2.0.1. Does it make sense to backport it to
2.1??
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-20567.master.001.patch
>
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-20567:
-
Status: Patch Available (was: Open)
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-20567.master.001.patch
>
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-20567:
-
Attachment: HBASE-20567.master.001.patch
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-20567.master.001.patch
>
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20565) ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4
[
https://issues.apache.org/jira/browse/HBASE-20565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471467#comment-16471467
]
Zheng Hu commented on HBASE-20565:
--
Add some log by debug.diff, and the result is shown in debug.log. I think the
bug is not from FilterList, but is from ColumnPaginationFilter#filterKeyValue.
{code}
public ReturnCode filterKeyValue(Cell v)
{
if (columnOffset != null) {
if (count >= limit) {
return ReturnCode.NEXT_ROW;
}
byte[] buffer = v.getQualifierArray();
if (buffer == null) {
return ReturnCode.SEEK_NEXT_USING_HINT;
}
int cmp = 0;
// Only compare if no KV's have been seen so far.
if (count == 0) {
cmp = Bytes.compareTo(buffer,
v.getQualifierOffset(),
v.getQualifierLength(),
this.columnOffset,
0,
this.columnOffset.length);
}
if (cmp < 0) {
return ReturnCode.SEEK_NEXT_USING_HINT;
} else {
count++;
return ReturnCode.INCLUDE_AND_NEXT_COL;
}
} else {
if (count >= offset + limit) {
return ReturnCode.NEXT_ROW;
}
ReturnCode code = count < offset ? ReturnCode.NEXT_COL :
ReturnCode.INCLUDE_AND_NEXT_COL;
count++; <--- the count increment even if return
NEXT_COL.
return code;
}
}
{code}
The count increment even if we return NEXT_COL, so after the filter checked
the column=Family:0, count=1, when filter is checking the column=Family:5,
the count is 5 now, so return a NEXT_ROW...
we should define our count as the number of included column in
ColumnPaginationFilter (just as the def in if (columnOffset != null) {} ),
rather than the index of column we encountered. ..
> ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect
> result since 1.4
> -
>
> Key: HBASE-20565
> URL: https://issues.apache.org/jira/browse/HBASE-20565
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.4
>Reporter: Jerry He
>Assignee: Zheng Hu
>Priority: Major
> Attachments: debug.diff, debug.log, test-branch-1.4.patch
>
>
> When ColumnPaginationFilter is combined with ColumnRangeFilter, we may see
> incorrect result.
> Here is a simple example.
> One row with 10 columns c0, c1, c2, .., c9. I have a ColumnRangeFilter for
> range c2 to c9. Then I have a ColumnPaginationFilter with limit 5 and offset
> 0. FileterList is FilterList(Operator.MUST_PASS_ALL, ColumnRangeFilter,
> ColumnPaginationFilter).
> We expect 5 columns being returned. But in HBase 1.4 and after, 4 columns
> are returned.
> In 1.2.x, the correct 5 columns are returned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20565) ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4
[ https://issues.apache.org/jira/browse/HBASE-20565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-20565: - Attachment: debug.diff debug.log > ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect > result since 1.4 > - > > Key: HBASE-20565 > URL: https://issues.apache.org/jira/browse/HBASE-20565 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 1.4.4 >Reporter: Jerry He >Assignee: Zheng Hu >Priority: Major > Attachments: debug.diff, debug.log, test-branch-1.4.patch > > > When ColumnPaginationFilter is combined with ColumnRangeFilter, we may see > incorrect result. > Here is a simple example. > One row with 10 columns c0, c1, c2, .., c9. I have a ColumnRangeFilter for > range c2 to c9. Then I have a ColumnPaginationFilter with limit 5 and offset > 0. FileterList is FilterList(Operator.MUST_PASS_ALL, ColumnRangeFilter, > ColumnPaginationFilter). > We expect 5 columns being returned. But in HBase 1.4 and after, 4 columns > are returned. > In 1.2.x, the correct 5 columns are returned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20230) Incorrrect log message in RSRpcService
[
https://issues.apache.org/jira/browse/HBASE-20230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471453#comment-16471453
]
maoling commented on HBASE-20230:
-
[~vishk] sorry. I can't get your idea. the log message in RSRpcService is
incorrect?
btw,
{code:java}
older than 1.3.
{code}
is is likely to lead to ambiguity
> Incorrrect log message in RSRpcService
> --
>
> Key: HBASE-20230
> URL: https://issues.apache.org/jira/browse/HBASE-20230
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1
>Reporter: Vishal Khandelwal
>Priority: Minor
>
> At RPCServices:3040, exception is thrown for version lesser 1.3 but check is
> for version 1.4
> VersionInfoUtil.hasMinimumVersion(context.getClientVersionInfo(), 1, 4)
> >> throw new UnknownScannerException("Throwing UnknownScannerException to
> >> reset the client"
> + " scanner state for clients older than 1.3.", e);
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-20567:
-
Attachment: (was: HBASE-20567.branch-2.001.patch)
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-20567:
-
Attachment: HBASE-20567.branch-2.001.patch
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-20567.branch-2.001.patch
>
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20567) Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-20567:
-
Summary: Pass both old and new descriptors to pre/post hooks of modify
operations for table and namespace (was: Give both old and new descriptors to
pre/post hooks of modify operations for table and namespace)
> Pass both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20567) Give both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-20567:
-
Summary: Give both old and new descriptors to pre/post hooks of modify
operations for table and namespace (was: Given both old and new descriptors to
pre/post hooks of modify operations for table and namespace)
> Give both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Priority: Major
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-20567) Give both old and new descriptors to pre/post hooks of modify operations for table and namespace
[
https://issues.apache.org/jira/browse/HBASE-20567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy reassigned HBASE-20567:
Assignee: Appy
> Give both old and new descriptors to pre/post hooks of modify operations for
> table and namespace
>
>
> Key: HBASE-20567
> URL: https://issues.apache.org/jira/browse/HBASE-20567
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
>
> In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's
> no way of getting the old descriptor which was there before modification
> happened.
> Having both old and new descriptors will make the hooks more useful.
> We felt the need when we wanted to audit certain events but there was no way
> of deducing them by just seeing 'after-state' of modification.
> To keep the method signatures consistent, i have modified both pre and post
> hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20567) Given both old and new descriptors to pre/post hooks of modify operations for table and namespace
Appy created HBASE-20567:
Summary: Given both old and new descriptors to pre/post hooks of
modify operations for table and namespace
Key: HBASE-20567
URL: https://issues.apache.org/jira/browse/HBASE-20567
Project: HBase
Issue Type: Improvement
Reporter: Appy
In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's no
way of getting the old descriptor which was there before modification happened.
Having both old and new descriptors will make the hooks more useful.
We felt the need when we wanted to audit certain events but there was no way of
deducing them by just seeing 'after-state' of modification.
To keep the method signatures consistent, i have modified both pre and post
hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18812) Recategorize some of classes used as tools
[ https://issues.apache.org/jira/browse/HBASE-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471430#comment-16471430 ] Guangxu Cheng commented on HBASE-18812: --- Attach 003 patch as [~chia7712] suggestions. Thanks > Recategorize some of classes used as tools > -- > > Key: HBASE-18812 > URL: https://issues.apache.org/jira/browse/HBASE-18812 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-18812.master.001.patch, > HBASE-18812.master.002.patch, HBASE-18812.master.003.patch > > > The classes used from cmd line should be made as LimitedPrivate.TOOLS. The > candidates are shown below. > # BackupDriver > # RestoreDriver > # CreateSnapshot > # SnapshotInfo > # ExportSnapshot > # Canary > # VersionInfo > # RegionMover > # CellCounter > # CopyTable > # DumpReplicationQueues > # Export > # HashTable > # Import > # ImportTsv > # LoadIncrementalHFiles > # ReplicationSyncUp > # SyncTable > # VerifyReplication > # WALPlayer > # ZkAclReset -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18812) Recategorize some of classes used as tools
[ https://issues.apache.org/jira/browse/HBASE-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guangxu Cheng updated HBASE-18812: -- Attachment: HBASE-18812.master.003.patch > Recategorize some of classes used as tools > -- > > Key: HBASE-18812 > URL: https://issues.apache.org/jira/browse/HBASE-18812 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-18812.master.001.patch, > HBASE-18812.master.002.patch, HBASE-18812.master.003.patch > > > The classes used from cmd line should be made as LimitedPrivate.TOOLS. The > candidates are shown below. > # BackupDriver > # RestoreDriver > # CreateSnapshot > # SnapshotInfo > # ExportSnapshot > # Canary > # VersionInfo > # RegionMover > # CellCounter > # CopyTable > # DumpReplicationQueues > # Export > # HashTable > # Import > # ImportTsv > # LoadIncrementalHFiles > # ReplicationSyncUp > # SyncTable > # VerifyReplication > # WALPlayer > # ZkAclReset -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18812) Recategorize some of classes used as tools
[
https://issues.apache.org/jira/browse/HBASE-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471423#comment-16471423
]
Chia-Ping Tsai commented on HBASE-18812:
{quote}Compressor is documented as "Also contains a command line tool to
compress and uncompress WALs." and "Command line tool to compress and
uncompress WALs."
{quote}
Got it. Thanks
> Recategorize some of classes used as tools
> --
>
> Key: HBASE-18812
> URL: https://issues.apache.org/jira/browse/HBASE-18812
> Project: HBase
> Issue Type: Task
>Reporter: Chia-Ping Tsai
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-18812.master.001.patch,
> HBASE-18812.master.002.patch
>
>
> The classes used from cmd line should be made as LimitedPrivate.TOOLS. The
> candidates are shown below.
> # BackupDriver
> # RestoreDriver
> # CreateSnapshot
> # SnapshotInfo
> # ExportSnapshot
> # Canary
> # VersionInfo
> # RegionMover
> # CellCounter
> # CopyTable
> # DumpReplicationQueues
> # Export
> # HashTable
> # Import
> # ImportTsv
> # LoadIncrementalHFiles
> # ReplicationSyncUp
> # SyncTable
> # VerifyReplication
> # WALPlayer
> # ZkAclReset
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20565) ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4
[ https://issues.apache.org/jira/browse/HBASE-20565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471417#comment-16471417 ] Zheng Hu commented on HBASE-20565: -- [~jinghe], Thanks for your report. Seems like it's a bug. Let me dig this . > ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect > result since 1.4 > - > > Key: HBASE-20565 > URL: https://issues.apache.org/jira/browse/HBASE-20565 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 1.4.4 >Reporter: Jerry He >Priority: Major > Attachments: test-branch-1.4.patch > > > When ColumnPaginationFilter is combined with ColumnRangeFilter, we may see > incorrect result. > Here is a simple example. > One row with 10 columns c0, c1, c2, .., c9. I have a ColumnRangeFilter for > range c2 to c9. Then I have a ColumnPaginationFilter with limit 5 and offset > 0. FileterList is FilterList(Operator.MUST_PASS_ALL, ColumnRangeFilter, > ColumnPaginationFilter). > We expect 5 columns being returned. But in HBase 1.4 and after, 4 columns > are returned. > In 1.2.x, the correct 5 columns are returned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20565) ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4
[ https://issues.apache.org/jira/browse/HBASE-20565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu reassigned HBASE-20565: Assignee: Zheng Hu > ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect > result since 1.4 > - > > Key: HBASE-20565 > URL: https://issues.apache.org/jira/browse/HBASE-20565 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 1.4.4 >Reporter: Jerry He >Assignee: Zheng Hu >Priority: Major > Attachments: test-branch-1.4.patch > > > When ColumnPaginationFilter is combined with ColumnRangeFilter, we may see > incorrect result. > Here is a simple example. > One row with 10 columns c0, c1, c2, .., c9. I have a ColumnRangeFilter for > range c2 to c9. Then I have a ColumnPaginationFilter with limit 5 and offset > 0. FileterList is FilterList(Operator.MUST_PASS_ALL, ColumnRangeFilter, > ColumnPaginationFilter). > We expect 5 columns being returned. But in HBase 1.4 and after, 4 columns > are returned. > In 1.2.x, the correct 5 columns are returned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18812) Recategorize some of classes used as tools
[ https://issues.apache.org/jira/browse/HBASE-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471408#comment-16471408 ] Guangxu Cheng commented on HBASE-18812: --- bq.BTW, is Compressor a tool class? Compressor is documented as "Also contains a command line tool to compress and uncompress WALs." and "Command line tool to compress and uncompress WALs." bq.Let the patch focus on those class. I will work on it now.Thanks > Recategorize some of classes used as tools > -- > > Key: HBASE-18812 > URL: https://issues.apache.org/jira/browse/HBASE-18812 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-18812.master.001.patch, > HBASE-18812.master.002.patch > > > The classes used from cmd line should be made as LimitedPrivate.TOOLS. The > candidates are shown below. > # BackupDriver > # RestoreDriver > # CreateSnapshot > # SnapshotInfo > # ExportSnapshot > # Canary > # VersionInfo > # RegionMover > # CellCounter > # CopyTable > # DumpReplicationQueues > # Export > # HashTable > # Import > # ImportTsv > # LoadIncrementalHFiles > # ReplicationSyncUp > # SyncTable > # VerifyReplication > # WALPlayer > # ZkAclReset -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19722) Implement a meta query statistics metrics source
[
https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471374#comment-16471374
]
Hadoop QA commented on HBASE-19722:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
50s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 5s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}111m
22s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-19722 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922933/HBASE-19722.patch.10 |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 9bceb9234076 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c60578d982 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12787/testReport/ |
| Max. process+thread count | 4099 (vs. ulimit of 1) |
|
[jira] [Commented] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471359#comment-16471359
]
Mike Drob commented on HBASE-20544:
---
Any idea why this got missed in precommit earlier? +1 for addendum.
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch, HBASE-20544.addendum.0.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
>
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471294#comment-16471294
]
Hudson commented on HBASE-20004:
Results for branch branch-1.3
[build #323 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/323/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/323//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/323//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/323//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Client is not able to execute REST queries in a secure cluster
> --
>
> Key: HBASE-20004
> URL: https://issues.apache.org/jira/browse/HBASE-20004
> Project: HBase
> Issue Type: Bug
> Components: REST, security
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20004.branch-1.patch,
> HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch,
> HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch,
> HBASE-20004.v2.patch
>
>
> Firefox browser is not able to negotiate REST queries with server in secure
> mode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20545) Improve performance of BaseLoadBalancer.retainAssignment
[
https://issues.apache.org/jira/browse/HBASE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471272#comment-16471272
]
Hudson commented on HBASE-20545:
Results for branch branch-2
[build #719 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/719/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/719//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/719//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/719//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Improve performance of BaseLoadBalancer.retainAssignment
>
>
> Key: HBASE-20545
> URL: https://issues.apache.org/jira/browse/HBASE-20545
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Affects Versions: 1.4.4, 2.0.0
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.5
>
> Attachments: HBASE-20545.branch-1.4.001.patch,
> HBASE-20545.branch-2.001.patch
>
>
> I was measuring perf at scale with a 1m region table and noticed some
> improvements can be made to BaseLoadBalancer.retainAssignment().
> retainAssignment() spends a few mins to enable a 1m regions table and also
> generates a lot of objects unnecessarily. This jira is to make the most
> common case go faster with very minimal changes. A slightly modified version
> of this patch takes about 5 seconds for a 1m region table ignoring the time
> spent in createCluster(). I think locality can be refreshed during master
> startup in different ways without taking time in retainAssignment, but will
> follow up on that in subsequent jiras. Leaving it untouched here, but wanted
> to call out the time taken without that method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471271#comment-16471271
]
Hudson commented on HBASE-20004:
Results for branch branch-2
[build #719 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/719/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/719//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/719//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/719//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Client is not able to execute REST queries in a secure cluster
> --
>
> Key: HBASE-20004
> URL: https://issues.apache.org/jira/browse/HBASE-20004
> Project: HBase
> Issue Type: Bug
> Components: REST, security
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20004.branch-1.patch,
> HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch,
> HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch,
> HBASE-20004.v2.patch
>
>
> Firefox browser is not able to negotiate REST queries with server in secure
> mode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19722) Implement a meta query statistics metrics source
[ https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-19722: Attachment: HBASE-19722.patch.10 > Implement a meta query statistics metrics source > > > Key: HBASE-19722 > URL: https://issues.apache.org/jira/browse/HBASE-19722 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-19722.patch, HBASE-19722.patch.1, > HBASE-19722.patch.10, HBASE-19722.patch.2, HBASE-19722.patch.3, > HBASE-19722.patch.4, HBASE-19722.patch.5, HBASE-19722.patch.6, > HBASE-19722.patch.7, HBASE-19722.patch.8, HBASE-19722.patch.9 > > > Implement a meta query statistics metrics source, created whenever a > regionserver starts hosting meta, removed when meta hosting moves. Provide > views on top tables by request counts, top meta rowkeys by request count, top > clients making requests by their hostname. > Can be implemented as a coprocessor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471255#comment-16471255
]
Hudson commented on HBASE-20004:
Results for branch branch-2.0
[build #283 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/283/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/283//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/283//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/283//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Client is not able to execute REST queries in a secure cluster
> --
>
> Key: HBASE-20004
> URL: https://issues.apache.org/jira/browse/HBASE-20004
> Project: HBase
> Issue Type: Bug
> Components: REST, security
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20004.branch-1.patch,
> HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch,
> HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch,
> HBASE-20004.v2.patch
>
>
> Firefox browser is not able to negotiate REST queries with server in secure
> mode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20530) Composition of backup directory containing namespace when restoring is different from the actual hfile location
[
https://issues.apache.org/jira/browse/HBASE-20530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471226#comment-16471226
]
Hadoop QA commented on HBASE-20530:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
55s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 58s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m
31s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 22m 12s{color}
| {color:red} hbase-backup in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 78m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.backup.TestBackupDescribe |
| | hadoop.hbase.backup.TestIncrementalBackupMergeWithFailures |
| | hadoop.hbase.backup.TestIncrementalBackupWithBulkLoad |
| | hadoop.hbase.backup.TestIncrementalBackupWithFailures |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20530 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922908/HBASE-20530-v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 139e267aa55f 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-19722) Implement a meta query statistics metrics source
[
https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471221#comment-16471221
]
Hadoop QA commented on HBASE-19722:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
1s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 45s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}172m 31s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}219m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.coprocessor.TestMetaTableMetrics |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-19722 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922893/HBASE-19722.patch.9 |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 45abcc9f3085 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / c60578d982 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| unit |
[jira] [Commented] (HBASE-20546) Improve perf of RegionLocationFinder.mapHostNameToServerName
[ https://issues.apache.org/jira/browse/HBASE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471212#comment-16471212 ] Andrew Purtell commented on HBASE-20546: I think we may need to make hostToServerName a concurrent hash map or otherwise protect it with synchronized access. Otherwise the load balancer could crash with CME. Map needs to be cleared first too, like [~chia7712] mentioned. How much are we saving? Seems the rebuild of the map is still pretty expensive. Can we incrementally update this state somehow instead? Or reuse state from somewhere else? Is there another place in the master we do frequent lookups of ServerNames from hostname? In the AM somewhere maybe? > Improve perf of RegionLocationFinder.mapHostNameToServerName > > > Key: HBASE-20546 > URL: https://issues.apache.org/jira/browse/HBASE-20546 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.4, 2.0.0 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 1.5.0, 2.0.1, 1.4.5 > > Attachments: HBASE-20546.branch-1.4.001.patch > > > RegionLocationFinder.getTopBlockLocations() is called multiple times during > balancer. While profiling on a large table balance, mapHostNameToServerName() > seem to take a lot of time. One of the maps is repeatedly created for each > iteration, while we can just initialize it once. > Goes into both branch-1 and branch-2, although patches differ slightly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471205#comment-16471205
]
Hudson commented on HBASE-20004:
Results for branch branch-1.2
[build #327 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/327/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/327//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/327//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/327//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Client is not able to execute REST queries in a secure cluster
> --
>
> Key: HBASE-20004
> URL: https://issues.apache.org/jira/browse/HBASE-20004
> Project: HBase
> Issue Type: Bug
> Components: REST, security
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20004.branch-1.patch,
> HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch,
> HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch,
> HBASE-20004.v2.patch
>
>
> Firefox browser is not able to negotiate REST queries with server in secure
> mode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[ https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471187#comment-16471187 ] Mike Drob commented on HBASE-20520: --- Left a minor nit on RB, otherwise patch looks good. +1 to commit it, sir. > Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize > -- > > Key: HBASE-20520 > URL: https://issues.apache.org/jira/browse/HBASE-20520 > Project: HBase > Issue Type: Sub-task > Components: conf >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20520.branch-2.0.001.patch, > HBASE-20520.branch-2.0.002.patch, HBASE-20520.branch-2.0.003.patch, > HBASE-20520.branch-2.0.004.patch, HBASE-20520.branch-2.0.005.patch > > > Good one found by our [~mdrob]. Problem in the parent issue where failed > attempt doubling default block size but halving the size at which roll -- so > we end up in roughly same place only we make less small WALs in those cases > where we are writing furiously and fail to roll before starting a new block. > From Drob: ".../we drop default log roll to 0.5, but we don't actually > increase the block size... AbstractProtobufLogWriter.java:164 still uses > default HDFS block sizing and AbstractFSWAL.java:408 block size is only used > in a log message, never actually makes it to files...". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20546) Improve perf of RegionLocationFinder.mapHostNameToServerName
[ https://issues.apache.org/jira/browse/HBASE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471173#comment-16471173 ] Thiruvel Thirumoolan commented on HBASE-20546: -- Thanks [~chia7712]. Good point. It looks like ClusterStatusChore will be updating clusterStatus. Its intention is to make StochasticBalancer better by updating regionload. Not sure why regionFinder's clusterstatus needs to be updated. I think we can have a setClusterStatus and updateClusterStatus API, so ClusterStatusChore can use the latter. I am not sure if updating clusterstatus or re-initializing the hostserver map I introduce in regionfinder in the middle of balance is a good idea. What do you think? > Improve perf of RegionLocationFinder.mapHostNameToServerName > > > Key: HBASE-20546 > URL: https://issues.apache.org/jira/browse/HBASE-20546 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.4, 2.0.0 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 1.5.0, 2.0.1, 1.4.5 > > Attachments: HBASE-20546.branch-1.4.001.patch > > > RegionLocationFinder.getTopBlockLocations() is called multiple times during > balancer. While profiling on a large table balance, mapHostNameToServerName() > seem to take a lot of time. One of the maps is repeatedly created for each > iteration, while we can just initialize it once. > Goes into both branch-1 and branch-2, although patches differ slightly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471163#comment-16471163
]
Ted Yu commented on HBASE-20566:
Interesting.
Do you want to work out a patch ?
Thanks
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20204) Add locking to RefreshFileConnections in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-20204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471101#comment-16471101
]
Hudson commented on HBASE-20204:
Results for branch branch-1.4
[build #316 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/316/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/316//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/316//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/316//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Add locking to RefreshFileConnections in BucketCache
>
>
> Key: HBASE-20204
> URL: https://issues.apache.org/jira/browse/HBASE-20204
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 1.4.3, 2.0.0
>Reporter: Zach York
>Assignee: Zach York
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.5
>
> Attachments: HBASE-20204.master.001.patch,
> HBASE-20204.master.002.patch, HBASE-20204.master.003.patch,
> HBASE-20204.master.004.patch
>
>
> This is a follow-up to HBASE-20141 where [~anoop.hbase] suggested adding
> locking for refreshing channels.
> I have also seen this become an issue when a RS has to abort and it locks on
> trying to flush out the remaining data to the cache (since cache on write was
> turned on).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20554) "WALs outstanding" message from CleanerChore is noisy
[
https://issues.apache.org/jira/browse/HBASE-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471102#comment-16471102
]
Hudson commented on HBASE-20554:
Results for branch branch-1.4
[build #316 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/316/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/316//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/316//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/316//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> "WALs outstanding" message from CleanerChore is noisy
> -
>
> Key: HBASE-20554
> URL: https://issues.apache.org/jira/browse/HBASE-20554
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 3.0.0, 2.1.0, 1.5.0, 2.0.1, 1.4.5
>
> Attachments: HBASE-20554.patch
>
>
> WARN level "WALs outstanding" from CleanerChore should be DEBUG and are not
> always correct.
> I left a cluster configured for ITBLL (retaining all WALs for post hoc
> analysis) and in the morning found the master log full of "WALs outstanding"
> warnings from CleanerChore.
> Should this really be a warning?
> {quote}
> 2018-05-09 16:42:03,893 WARN
> [node-1.cluster,16000,1525851521469_ChoreService_2] cleaner.CleanerChore:
> WALs outstanding under hdfs://node-1.cluster/hbase/oldWALs
> {quote}
> If someone has configured really long WAL retention then having WALs in
> oldWALs will be normal.
> Also, it seems the warning is sometimes incorrect.
> {quote}
> 2018-05-09 16:42:24,751 WARN
> [node-1.cluster,16000,1525851521469_ChoreService_1] cleaner.CleanerChore:
> WALs outstanding under hdfs://node-1.cluster/hbase/archive
> {quote}
> There are no WALs under archive/.
> Even at DEBUG level, if it is not correct, then it can lead an operator to be
> concerned about nothing, so better to just remove it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20447) Only fail cacheBlock if block collisions aren't related to next block metadata
[
https://issues.apache.org/jira/browse/HBASE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471091#comment-16471091
]
Hadoop QA commented on HBASE-20447:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
10s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 58s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
27s{color} | {color:red} hbase-server generated 1 new + 2 unchanged - 0 fixed =
3 total (was 2) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}116m
8s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
18s{color} | {color:green} hbase-external-blockcache in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}159m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20447 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922877/HBASE-20447.master.004.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6007bb544e92 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c60578d982 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297;
[jira] [Updated] (HBASE-20530) Composition of backup directory containing namespace when restoring is different from the actual hfile location
[
https://issues.apache.org/jira/browse/HBASE-20530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Rodionov updated HBASE-20530:
--
Attachment: HBASE-20530-v3.patch
> Composition of backup directory containing namespace when restoring is
> different from the actual hfile location
> ---
>
> Key: HBASE-20530
> URL: https://issues.apache.org/jira/browse/HBASE-20530
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Vladimir Rodionov
>Priority: Critical
> Attachments: HBASE-20530-v1.patch, HBASE-20530-v2.patch,
> HBASE-20530-v3.patch
>
>
> Here is partial listing of output from incremental backup:
> {code}
> 5306 2018-05-04 02:38
> hdfs://mycluster/user/hbase/backup_loc/backup_1525401467793/table_almphxih4u/cf1/5648501da7194783947bbf07b172f07e
> {code}
> When restoring, here is what HBackupFileSystem.getTableBackupDir returns:
> {code}
> fileBackupDir=hdfs://mycluster/user/hbase/backup_loc/backup_1525401467793/default/table_almphxih4u
> {code}
> You can see that namespace gets in the way, leading to inability of finding
> the proper hfile.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20530) Composition of backup directory containing namespace when restoring is different from the actual hfile location
[
https://issues.apache.org/jira/browse/HBASE-20530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471090#comment-16471090
]
Vladimir Rodionov commented on HBASE-20530:
---
Patch v3. Fixes javac and checkstyle warnings
> Composition of backup directory containing namespace when restoring is
> different from the actual hfile location
> ---
>
> Key: HBASE-20530
> URL: https://issues.apache.org/jira/browse/HBASE-20530
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Vladimir Rodionov
>Priority: Critical
> Attachments: HBASE-20530-v1.patch, HBASE-20530-v2.patch,
> HBASE-20530-v3.patch
>
>
> Here is partial listing of output from incremental backup:
> {code}
> 5306 2018-05-04 02:38
> hdfs://mycluster/user/hbase/backup_loc/backup_1525401467793/table_almphxih4u/cf1/5648501da7194783947bbf07b172f07e
> {code}
> When restoring, here is what HBackupFileSystem.getTableBackupDir returns:
> {code}
> fileBackupDir=hdfs://mycluster/user/hbase/backup_loc/backup_1525401467793/default/table_almphxih4u
> {code}
> You can see that namespace gets in the way, leading to inability of finding
> the proper hfile.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20556) Backport HBASE-16490 to branch-1
[
https://issues.apache.org/jira/browse/HBASE-20556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471044#comment-16471044
]
Hadoop QA commented on HBASE-20556:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
14s{color} | {color:red} hbase-server in branch-1 failed with JDK v1.7.0_181.
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
13s{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_181.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 13s{color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_181. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
7s{color} | {color:red} hbase-server: The patch generated 4 new + 279 unchanged
- 1 fixed = 283 total (was 280) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m
49s{color} | {color:red} The patch causes 44 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
34s{color} | {color:red} The patch causes 44 errors with Hadoop v2.5.2. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}104m 53s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}124m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:36a7029 |
| JIRA Issue | HBASE-20556 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922766/HBASE-20556.branch-1.002.patch
|
| Optional Tests | asflicense javac javadoc
[jira] [Updated] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Biju Nair updated HBASE-20566:
--
Description:
*Steps to reproduce*
- Enable {{rsgroup}} feature
- Enable {{quota}} feature which created {{hbase::quota}} table
- quota table region will be marked as RIT since the {{rsgroup}} for the table
is not known
{noformat}
2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
ENABLING
2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0] master.RegionStates:
Failed to open/close 89490cd5e00ea8948af413a1df65091a on null, set to
FAILED_OPEN
2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0] master.RegionStates:
Transition {89490cd5e00ea8948af413a1df65091a state=OFFLINE, ts=1525977212397,
server=null} to {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN,
ts=1525977212398, server=null}
2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING to
ENABLED
{noformat}
- Reason for this issue: Issue
- [system table
creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
doesn't move the table to the appropriate rs group to which system namespace
is assigned to. Need to execute logic similar to what is done in the
RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
*Work Around*
- Assigning the system table to ``default`` rsgroup (or to the rsgroup to
which the system namespace has been assigned).
- Manually assigning the region in RIT from the system table
was:
*Steps to reproduce*
- Enable {{rsgroup}} feature
- Enable {{quota}} feature which created {{hbase::quota}} table
- quota table region will be marked as RIT since the {{rsgroup}} for the table
is not known
{noformat}
2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
ENABLING
2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0] master.RegionStates:
Failed to open/close 89490cd5e00ea8948af413a1df65091a on null, set to
FAILED_OPEN
2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0] master.RegionStates:
Transition {89490cd5e00ea8948af413a1df65091a state=OFFLINE, ts=1525977212397,
server=null} to {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN,
ts=1525977212398, server=null}
2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING to
ENABLED
{noformat}
- Reason for this issue: Issue
- [system table
creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
doesn't move the table to the appropriate rs group to which system namespace
is assigned to. Need to execute logic similar to what is done in the
RSGroupAdminEndpoint for [post table
creation|(https://github.com/apache/hbase/blob/45586ab3006885cea29ca5e0502b1ac00dc1b916/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L377]
for user table creation.
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
>
[jira] [Created] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
Biju Nair created HBASE-20566:
-
Summary: Creating a system table after enabling rsgroup feature
puts in region into RIT
Key: HBASE-20566
URL: https://issues.apache.org/jira/browse/HBASE-20566
Project: HBase
Issue Type: Bug
Components: master
Reporter: Biju Nair
*Steps to reproduce*
- Enable {{rsgroup}} feature
- Enable {{quota}} feature which created {{hbase::quota}} table
- quota table region will be marked as RIT since the {{rsgroup}} for the table
is not known
{noformat}
2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
ENABLING
2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0] master.RegionStates:
Failed to open/close 89490cd5e00ea8948af413a1df65091a on null, set to
FAILED_OPEN
2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0] master.RegionStates:
Transition {89490cd5e00ea8948af413a1df65091a state=OFFLINE, ts=1525977212397,
server=null} to {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN,
ts=1525977212398, server=null}
2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING to
ENABLED
{noformat}
- Reason for this issue: Issue
- [system table
creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
doesn't move the table to the appropriate rs group to which system namespace
is assigned to. Need to execute logic similar to what is done in the
RSGroupAdminEndpoint for [post table
creation|(https://github.com/apache/hbase/blob/45586ab3006885cea29ca5e0502b1ac00dc1b916/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L377]
for user table creation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20565) ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4
[ https://issues.apache.org/jira/browse/HBASE-20565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471024#comment-16471024 ] Jerry He commented on HBASE-20565: -- Some research shows it is caused by HBASE-18993. FYI . [~openinx], [~apurtell] > ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect > result since 1.4 > - > > Key: HBASE-20565 > URL: https://issues.apache.org/jira/browse/HBASE-20565 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 1.4.4 >Reporter: Jerry He >Priority: Major > Attachments: test-branch-1.4.patch > > > When ColumnPaginationFilter is combined with ColumnRangeFilter, we may see > incorrect result. > Here is a simple example. > One row with 10 columns c0, c1, c2, .., c9. I have a ColumnRangeFilter for > range c2 to c9. Then I have a ColumnPaginationFilter with limit 5 and offset > 0. FileterList is FilterList(Operator.MUST_PASS_ALL, ColumnRangeFilter, > ColumnPaginationFilter). > We expect 5 columns being returned. But in HBase 1.4 and after, 4 columns > are returned. > In 1.2.x, the correct 5 columns are returned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20565) ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4
[ https://issues.apache.org/jira/browse/HBASE-20565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471010#comment-16471010 ] Jerry He commented on HBASE-20565: -- I attached a test case that can be used to re-create the problem. The test passes in branch 1.2, but fails in 1.4 and later. > ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect > result since 1.4 > - > > Key: HBASE-20565 > URL: https://issues.apache.org/jira/browse/HBASE-20565 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 1.4.4 >Reporter: Jerry He >Priority: Major > Attachments: test-branch-1.4.patch > > > When ColumnPaginationFilter is combined with ColumnRangeFilter, we may see > incorrect result. > Here is a simple example. > One row with 10 columns c0, c1, c2, .., c9. I have a ColumnRangeFilter for > range c2 to c9. Then I have a ColumnPaginationFilter with limit 5 and offset > 0. FileterList is FilterList(Operator.MUST_PASS_ALL, ColumnRangeFilter, > ColumnPaginationFilter). > We expect 5 columns being returned. But in HBase 1.4 and after, 4 columns > are returned. > In 1.2.x, the correct 5 columns are returned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471008#comment-16471008
]
Sean Busbey commented on HBASE-20544:
-
confirmed that the test in question ran during the precommit run:
https://builds.apache.org/job/PreCommit-HBASE-Build/12784/testReport/org.apache.hadoop.hbase.rsgroup/TestEnableRSGroup/
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch, HBASE-20544.addendum.0.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
>
[jira] [Updated] (HBASE-20565) ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4
[ https://issues.apache.org/jira/browse/HBASE-20565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry He updated HBASE-20565: - Attachment: test-branch-1.4.patch > ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect > result since 1.4 > - > > Key: HBASE-20565 > URL: https://issues.apache.org/jira/browse/HBASE-20565 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 1.4.4 >Reporter: Jerry He >Priority: Major > Attachments: test-branch-1.4.patch > > > When ColumnPaginationFilter is combined with ColumnRangeFilter, we may see > incorrect result. > Here is a simple example. > One row with 10 columns c0, c1, c2, .., c9. I have a ColumnRangeFilter for > range c2 to c9. Then I have a ColumnPaginationFilter with limit 5 and offset > 0. FileterList is FilterList(Operator.MUST_PASS_ALL, ColumnRangeFilter, > ColumnPaginationFilter). > We expect 5 columns being returned. But in HBase 1.4 and after, 4 columns > are returned. > In 1.2.x, the correct 5 columns are returned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20565) ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4
Jerry He created HBASE-20565: Summary: ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result since 1.4 Key: HBASE-20565 URL: https://issues.apache.org/jira/browse/HBASE-20565 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 1.4.4 Reporter: Jerry He When ColumnPaginationFilter is combined with ColumnRangeFilter, we may see incorrect result. Here is a simple example. One row with 10 columns c0, c1, c2, .., c9. I have a ColumnRangeFilter for range c2 to c9. Then I have a ColumnPaginationFilter with limit 5 and offset 0. FileterList is FilterList(Operator.MUST_PASS_ALL, ColumnRangeFilter, ColumnPaginationFilter). We expect 5 columns being returned. But in HBase 1.4 and after, 4 columns are returned. In 1.2.x, the correct 5 columns are returned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20257) hbase-spark should not depend on com.google.code.findbugs.jsr305
[
https://issues.apache.org/jira/browse/HBASE-20257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470957#comment-16470957
]
Ted Yu commented on HBASE-20257:
Ping [~busbey] for review.
> hbase-spark should not depend on com.google.code.findbugs.jsr305
>
>
> Key: HBASE-20257
> URL: https://issues.apache.org/jira/browse/HBASE-20257
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 3.0.0
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20257.v01.patch, HBASE-20257.v02.patch,
> HBASE-20257.v03.patch, HBASE-20257.v04.patch
>
>
> The following can be observed in the build output of master branch:
> {code}
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed
> with message:
> We don't allow the JSR305 jar from the Findbugs project, see HBASE-16321.
> Found Banned Dependency: com.google.code.findbugs:jsr305:jar:1.3.9
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
> {code}
> Here is related snippet from hbase-spark/pom.xml:
> {code}
>
> com.google.code.findbugs
> jsr305
> {code}
> Dependency on jsr305 should be dropped.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470949#comment-16470949
]
Hadoop QA commented on HBASE-20544:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
42s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 57s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
6s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20544 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922885/HBASE-20544.addendum.0.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e3dc1cd2d38d 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c60578d982 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12784/testReport/ |
| Max. process+thread count | 2733 (vs. ulimit of 1) |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12784/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> downstream
[jira] [Commented] (HBASE-19722) Implement a meta query statistics metrics source
[ https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470937#comment-16470937 ] Xu Cang commented on HBASE-19722: - code review: [https://reviews.apache.org/r/67062] > Implement a meta query statistics metrics source > > > Key: HBASE-19722 > URL: https://issues.apache.org/jira/browse/HBASE-19722 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-19722.patch, HBASE-19722.patch.1, > HBASE-19722.patch.2, HBASE-19722.patch.3, HBASE-19722.patch.4, > HBASE-19722.patch.5, HBASE-19722.patch.6, HBASE-19722.patch.7, > HBASE-19722.patch.8, HBASE-19722.patch.9 > > > Implement a meta query statistics metrics source, created whenever a > regionserver starts hosting meta, removed when meta hosting moves. Provide > views on top tables by request counts, top meta rowkeys by request count, top > clients making requests by their hostname. > Can be implemented as a coprocessor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19722) Implement a meta query statistics metrics source
[ https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-19722: Attachment: HBASE-19722.patch.9 > Implement a meta query statistics metrics source > > > Key: HBASE-19722 > URL: https://issues.apache.org/jira/browse/HBASE-19722 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-19722.patch, HBASE-19722.patch.1, > HBASE-19722.patch.2, HBASE-19722.patch.3, HBASE-19722.patch.4, > HBASE-19722.patch.5, HBASE-19722.patch.6, HBASE-19722.patch.7, > HBASE-19722.patch.8, HBASE-19722.patch.9 > > > Implement a meta query statistics metrics source, created whenever a > regionserver starts hosting meta, removed when meta hosting moves. Provide > views on top tables by request counts, top meta rowkeys by request count, top > clients making requests by their hostname. > Can be implemented as a coprocessor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470901#comment-16470901
]
Sean Busbey commented on HBASE-20544:
-
{quote}
bq. setConf is the wrong way to update a config
Can you elaborate under what circumstance the method cannot be used ?
{quote}
setConf is dangerous if a copy gets made and changes that TEST_UTIL uses
internally aren't reflected in the later version. For example, before the
original patch on this jira the test that's failing would copy the
Configuration from TEST_UTIL prior to any cluster startup related modifications
and then it set the config specifically on the hbase minicluster.
since the getConfiguration call returns a Configuration object that will
reflect changes back to the clusters it comes from, we can just toggle things
directly.
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch, HBASE-20544.addendum.0.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
>
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470893#comment-16470893 ] Hudson commented on HBASE-20004: SUCCESS: Integrated in Jenkins build HBase-1.2-IT # (See [https://builds.apache.org/job/HBase-1.2-IT//]) Addendum HBASE-20004 Client is not able to execute REST queries in a (ashishsinghi: rev 797a352763110413c4e806770ca13c74ef2a13ea) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/util/HttpServerUtil.java * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/HBaseRESTTestingUtility.java * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20004.branch-1.patch, > HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch, > HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch, > HBASE-20004.v2.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20544:
Status: Patch Available (was: Reopened)
addendum v0
- switch the rsgroups test to modify the hbase mini cluster config instead of
the parent one.
no idea how this didn't hit in precommit before. addendum change is pretty
simple.
great catch [~yuzhih...@gmail.com]. how's this look?
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch, HBASE-20544.addendum.0.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
>
[jira] [Commented] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470885#comment-16470885
]
Ted Yu commented on HBASE-20544:
bq. setConf is the wrong way to update a config
Can you elaborate under what circumstance the method cannot be used ?
bq. is there a profile or something?
Not sure what you mean by profile. I discovered this test failure when running
test suite locally.
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch, HBASE-20544.addendum.0.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
>
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470883#comment-16470883 ] Hudson commented on HBASE-20004: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #405 (See [https://builds.apache.org/job/HBase-1.3-IT/405/]) Addendum HBASE-20004 Client is not able to execute REST queries in a (ashishsinghi: rev 75e7714d2057917523bb66464de921f180099f71) * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/HBaseRESTTestingUtility.java > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20004.branch-1.patch, > HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch, > HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch, > HBASE-20004.v2.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20544:
Attachment: HBASE-20544.addendum.0.patch
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch, HBASE-20544.addendum.0.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
[jira] [Updated] (HBASE-20556) Backport HBASE-16490 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-20556: -- Status: Patch Available (was: Open) > Backport HBASE-16490 to branch-1 > > > Key: HBASE-20556 > URL: https://issues.apache.org/jira/browse/HBASE-20556 > Project: HBase > Issue Type: Sub-task > Components: HFile, snapshots >Affects Versions: 1.4.4, 1.4.5 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Major > Attachments: HBASE-20556.branch-1.002.patch > > > As part of HBASE-20555, HBASE-16490 is the first patch that is needed for > backporting HBASE-18083 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-20004: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.4.5 1.3.3 1.2.7 1.5.0 2.1.0 3.0.0 Release Note: Added 'hbase.rest.http.allow.options.method' configuration property to allow user to decide whether Rest Server HTTP should allow OPTIONS method or not. By default it is enabled in HBase 2.1.0+ versions and in other versions it is disabled. Similarly 'hbase.thrift.http.allow.options.method' is added HBase 1.5, 2.1.0 and 3.0.0 versions. It is disabled by default. Status: Resolved (was: Patch Available) Pushed to 1.2+ versions. > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20004.branch-1.patch, > HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch, > HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch, > HBASE-20004.v2.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470847#comment-16470847 ] Hudson commented on HBASE-20004: FAILURE: Integrated in Jenkins build HBase-1.3-IT #404 (See [https://builds.apache.org/job/HBase-1.3-IT/404/]) HBASE-20004 Client is not able to execute REST queries in a secure (ashishsinghi: rev 939ee7fd0e5dfb3ab2c72a54c9929f28bc1c8331) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/util/HttpServerUtil.java * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Fix For: 2.0.1 > > Attachments: HBASE-20004.branch-1.patch, > HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch, > HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch, > HBASE-20004.v2.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20545) Improve performance of BaseLoadBalancer.retainAssignment
[ https://issues.apache.org/jira/browse/HBASE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20545: --- Fix Version/s: 2.1.0 > Improve performance of BaseLoadBalancer.retainAssignment > > > Key: HBASE-20545 > URL: https://issues.apache.org/jira/browse/HBASE-20545 > Project: HBase > Issue Type: Improvement > Components: Balancer >Affects Versions: 1.4.4, 2.0.0 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.5 > > Attachments: HBASE-20545.branch-1.4.001.patch, > HBASE-20545.branch-2.001.patch > > > I was measuring perf at scale with a 1m region table and noticed some > improvements can be made to BaseLoadBalancer.retainAssignment(). > retainAssignment() spends a few mins to enable a 1m regions table and also > generates a lot of objects unnecessarily. This jira is to make the most > common case go faster with very minimal changes. A slightly modified version > of this patch takes about 5 seconds for a 1m region table ignoring the time > spent in createCluster(). I think locality can be refreshed during master > startup in different ways without taking time in retainAssignment, but will > follow up on that in subsequent jiras. Leaving it untouched here, but wanted > to call out the time taken without that method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20546) Improve perf of RegionLocationFinder.mapHostNameToServerName
[ https://issues.apache.org/jira/browse/HBASE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvel Thirumoolan updated HBASE-20546: - Description: RegionLocationFinder.getTopBlockLocations() is called multiple times during balancer. While profiling on a large table balance, mapHostNameToServerName() seem to take a lot of time. One of the maps is repeatedly created for each iteration, while we can just initialize it once. Goes into both branch-1 and branch-2, although patches differ slightly. was: RegionLocationFinder.getTopBlockLocations() is called multiple times during balancer. While profiling on a large table balance, mapHostNameToServerName() seem to take a lot of time. One of the maps is repeatedly consumed for each iteration, while we can just initialize it once. Goes into both branch-1 and branch-2, although patches differ slightly. > Improve perf of RegionLocationFinder.mapHostNameToServerName > > > Key: HBASE-20546 > URL: https://issues.apache.org/jira/browse/HBASE-20546 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.4, 2.0.0 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 1.5.0, 2.0.1, 1.4.5 > > Attachments: HBASE-20546.branch-1.4.001.patch > > > RegionLocationFinder.getTopBlockLocations() is called multiple times during > balancer. While profiling on a large table balance, mapHostNameToServerName() > seem to take a lot of time. One of the maps is repeatedly created for each > iteration, while we can just initialize it once. > Goes into both branch-1 and branch-2, although patches differ slightly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20545) Improve performance of BaseLoadBalancer.retainAssignment
[ https://issues.apache.org/jira/browse/HBASE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470829#comment-16470829 ] Thiruvel Thirumoolan commented on HBASE-20545: -- [~yuzhih...@gmail.com], Looks like the build you triggered passed. Since existing tests were sufficient, I didn't add any. Can we also get this into branch-2? > Improve performance of BaseLoadBalancer.retainAssignment > > > Key: HBASE-20545 > URL: https://issues.apache.org/jira/browse/HBASE-20545 > Project: HBase > Issue Type: Improvement > Components: Balancer >Affects Versions: 1.4.4, 2.0.0 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.4.5 > > Attachments: HBASE-20545.branch-1.4.001.patch, > HBASE-20545.branch-2.001.patch > > > I was measuring perf at scale with a 1m region table and noticed some > improvements can be made to BaseLoadBalancer.retainAssignment(). > retainAssignment() spends a few mins to enable a 1m regions table and also > generates a lot of objects unnecessarily. This jira is to make the most > common case go faster with very minimal changes. A slightly modified version > of this patch takes about 5 seconds for a 1m region table ignoring the time > spent in createCluster(). I think locality can be refreshed during master > startup in different ways without taking time in retainAssignment, but will > follow up on that in subsequent jiras. Leaving it untouched here, but wanted > to call out the time taken without that method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20447) Only fail cacheBlock if block collisions aren't related to next block metadata
[ https://issues.apache.org/jira/browse/HBASE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-20447: -- Attachment: HBASE-20447.master.004.patch > Only fail cacheBlock if block collisions aren't related to next block metadata > -- > > Key: HBASE-20447 > URL: https://issues.apache.org/jira/browse/HBASE-20447 > Project: HBase > Issue Type: Bug > Components: BlockCache, BucketCache >Affects Versions: 1.4.3, 2.0.0 >Reporter: Zach York >Assignee: Zach York >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 2.0.1, 1.4.5 > > Attachments: HBASE-20447.branch-1.001.patch, > HBASE-20447.branch-1.002.patch, HBASE-20447.branch-1.003.patch, > HBASE-20447.branch-1.004.patch, HBASE-20447.branch-1.005.patch, > HBASE-20447.branch-1.006.patch, HBASE-20447.master.001.patch, > HBASE-20447.master.002.patch, HBASE-20447.master.003.patch, > HBASE-20447.master.004.patch > > > This is the issue I was originally having here: > [http://mail-archives.apache.org/mod_mbox/hbase-dev/201802.mbox/%3CCAN+qs_Pav=md_aoj4xji+kcnetubg2xou2ntxv1g6m8-5vn...@mail.gmail.com%3E] > > When we pread, we don't force the read to read all of the next block header. > However, when we get into a race condition where two opener threads try to > cache the same block and one thread read all of the next block header and the > other one didn't, it will fail the open process. This is especially important > in a splitting case where it will potentially fail the split process. > Instead, in the caches, we should only fail if the required blocks are > different. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reopened HBASE-20544:
-
reopening for addendum. I thought I got through the rsgroup tests.
[~yuzhih...@gmail.com] is there a profile or something?
setConf is the wrong way to update a config. let me figure out why that conf
change isn't reflected.
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
>
[jira] [Commented] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470807#comment-16470807
]
Ted Yu commented on HBASE-20544:
Got the following test failure:
{code}
testEnableRSGroup(org.apache.hadoop.hbase.rsgroup.TestEnableRSGroup) Time
elapsed: 7.419 sec <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer cannot be cast
to org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer
at
org.apache.hadoop.hbase.rsgroup.TestEnableRSGroup.testEnableRSGroup(TestEnableRSGroup.java:85)
{code}
This seems to be due to the following change in TestEnableRSGroup.java :
{code}
conf.set(HConstants.HBASE_MASTER_LOADBALANCER_CLASS,
RSGroupBasedLoadBalancer.class.getName());
-TEST_UTIL.getMiniHBaseCluster().setConf(conf);
{code}
Without the setConf call, balancer class wouldn't pass the assertion.
Adding setConf call back makes the test pass.
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470753#comment-16470753 ] Ashish Singhi commented on HBASE-20004: --- Thanks for the review [~busbey] and [~stack]. branch-1 failures not related to the match. I will commit this patch shortly > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Fix For: 2.0.1 > > Attachments: HBASE-20004.branch-1.patch, > HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch, > HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch, > HBASE-20004.v2.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470753#comment-16470753 ] Ashish Singhi edited comment on HBASE-20004 at 5/10/18 5:07 PM: Thanks for the review [~busbey] and [~stack]. branch-1 failures are not related to the patch. I will commit this patch shortly was (Author: ashish singhi): Thanks for the review [~busbey] and [~stack]. branch-1 failures not related to the match. I will commit this patch shortly > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Fix For: 2.0.1 > > Attachments: HBASE-20004.branch-1.patch, > HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch, > HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch, > HBASE-20004.v2.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470717#comment-16470717
]
Hudson commented on HBASE-20544:
Results for branch branch-2.0
[build #282 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/282/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/282//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/282//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/282//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
[jira] [Updated] (HBASE-20545) Improve performance of BaseLoadBalancer.retainAssignment
[ https://issues.apache.org/jira/browse/HBASE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20545: --- Fix Version/s: (was: 2.0.1) 3.0.0 > Improve performance of BaseLoadBalancer.retainAssignment > > > Key: HBASE-20545 > URL: https://issues.apache.org/jira/browse/HBASE-20545 > Project: HBase > Issue Type: Improvement > Components: Balancer >Affects Versions: 1.4.4, 2.0.0 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.4.5 > > Attachments: HBASE-20545.branch-1.4.001.patch, > HBASE-20545.branch-2.001.patch > > > I was measuring perf at scale with a 1m region table and noticed some > improvements can be made to BaseLoadBalancer.retainAssignment(). > retainAssignment() spends a few mins to enable a 1m regions table and also > generates a lot of objects unnecessarily. This jira is to make the most > common case go faster with very minimal changes. A slightly modified version > of this patch takes about 5 seconds for a 1m region table ignoring the time > spent in createCluster(). I think locality can be refreshed during master > startup in different ways without taking time in retainAssignment, but will > follow up on that in subsequent jiras. Leaving it untouched here, but wanted > to call out the time taken without that method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470620#comment-16470620
]
Hudson commented on HBASE-20544:
Results for branch branch-2
[build #718 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/718/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/718//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/718//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/718//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470572#comment-16470572
]
Hadoop QA commented on HBASE-20004:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
47s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
17s{color} | {color:red} hbase-server in branch-1 failed with JDK v1.7.0_181.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
14s{color} | {color:red} hbase-thrift in branch-1 failed with JDK v1.7.0_181.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
14s{color} | {color:red} hbase-rest in branch-1 failed with JDK v1.7.0_181.
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
5s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
36s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
29s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
16s{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_181.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
15s{color} | {color:red} hbase-thrift in the patch failed with JDK v1.7.0_181.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
14s{color} | {color:red} hbase-rest in the patch failed with JDK v1.7.0_181.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 16s{color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_181. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 15s{color}
| {color:red} hbase-thrift in the patch failed with JDK v1.7.0_181. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 14s{color}
| {color:red} hbase-rest in the patch failed with JDK v1.7.0_181. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
47s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m
58s{color} | {color:red} The patch causes 44 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red}
[jira] [Commented] (HBASE-20564) Tighter ByteBufferKeyValue Cell Comparator
[
https://issues.apache.org/jira/browse/HBASE-20564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470554#comment-16470554
]
Hadoop QA commented on HBASE-20564:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
27s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 28s{color}
| {color:red} hbase-common generated 1 new + 40 unchanged - 0 fixed = 41 total
(was 40) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} hbase-common: The patch generated 5 new + 9 unchanged
- 4 fixed = 14 total (was 13) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 19s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
23s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:dba4808 |
| JIRA Issue | HBASE-20564 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922848/HBASE-20564.branch-2.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 71b5dba2bee2 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 61f96b6ffa |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/12781/artifact/patchprocess/diff-compile-javac-hbase-common.txt
|
| checkstyle |
[jira] [Commented] (HBASE-20477) Document HeapMemoryTuner in refguide
[ https://issues.apache.org/jira/browse/HBASE-20477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470486#comment-16470486 ] stack commented on HBASE-20477: --- Thanks for having a go at this [~mingdaoy]. Would be good to doc what an operator should look for to see that it is on. I just tried to enable it a day or so ago and found it tricky See HBASE-20549 > Document HeapMemoryTuner in refguide > > > Key: HBASE-20477 > URL: https://issues.apache.org/jira/browse/HBASE-20477 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: stack >Assignee: Mingdao Yang >Priority: Major > Labels: beginner > Fix For: 3.0.0 > > > This is an old facility that is in danger of disappearing because there is no > mention in refguide and because it is private interface, it does not show in > the user API. Add doc on how it works. It could probably do with a bit of > testing too to make sure it working as advertised. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20004: -- Fix Version/s: 2.0.1 > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Fix For: 2.0.1 > > Attachments: HBASE-20004.branch-1.patch, > HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch, > HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch, > HBASE-20004.v2.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470480#comment-16470480 ] stack commented on HBASE-20004: --- +1 for branch-2.0 [~ashish singhi] > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Fix For: 2.0.1 > > Attachments: HBASE-20004.branch-1.patch, > HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch, > HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch, > HBASE-20004.v2.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20564) Tighter ByteBufferKeyValue Cell Comparator
[ https://issues.apache.org/jira/browse/HBASE-20564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20564: -- Attachment: HBASE-20564.branch-2.patch > Tighter ByteBufferKeyValue Cell Comparator > -- > > Key: HBASE-20564 > URL: https://issues.apache.org/jira/browse/HBASE-20564 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20564.branch-2.patch > > > Comparing Cells in hbase2 takes almost 3x the CPU. > In hbase1, its a keyValue backed by a byte array caching a few important > values.. In hbase2, its a NoTagByteBufferChunkKeyValue(?) deserializing the > row/family/qualifier lengths repeatedly. > I tried making a purposed comparator -- one that was not generic -- and it > seemed to have a nicer profile coming close to hbase1 in percentage used > (I'll post graphs) when I ran it in my perpetual memstore filler (See scripts > attached to HBASE-20483). It doesn't work when I try to run it on cluster. > Let me run unit tests to see if it can figure what I have wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20564) Tighter ByteBufferKeyValue Cell Comparator
[ https://issues.apache.org/jira/browse/HBASE-20564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20564: -- Assignee: stack Fix Version/s: 2.0.1 Status: Patch Available (was: Open) > Tighter ByteBufferKeyValue Cell Comparator > -- > > Key: HBASE-20564 > URL: https://issues.apache.org/jira/browse/HBASE-20564 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20564.branch-2.patch > > > Comparing Cells in hbase2 takes almost 3x the CPU. > In hbase1, its a keyValue backed by a byte array caching a few important > values.. In hbase2, its a NoTagByteBufferChunkKeyValue(?) deserializing the > row/family/qualifier lengths repeatedly. > I tried making a purposed comparator -- one that was not generic -- and it > seemed to have a nicer profile coming close to hbase1 in percentage used > (I'll post graphs) when I ran it in my perpetual memstore filler (See scripts > attached to HBASE-20483). It doesn't work when I try to run it on cluster. > Let me run unit tests to see if it can figure what I have wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20564) Tighter ByteBufferKeyValue Cell Comparator
[ https://issues.apache.org/jira/browse/HBASE-20564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20564: -- Summary: Tighter ByteBufferKeyValue Cell Comparator (was: Tighter BB Cell Comparator) > Tighter ByteBufferKeyValue Cell Comparator > -- > > Key: HBASE-20564 > URL: https://issues.apache.org/jira/browse/HBASE-20564 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Priority: Major > > Comparing Cells in hbase2 takes almost 3x the CPU. > In hbase1, its a keyValue backed by a byte array caching a few important > values.. In hbase2, its a NoTagByteBufferChunkKeyValue(?) deserializing the > row/family/qualifier lengths repeatedly. > I tried making a purposed comparator -- one that was not generic -- and it > seemed to have a nicer profile coming close to hbase1 in percentage used > (I'll post graphs) when I ran it in my perpetual memstore filler (See scripts > attached to HBASE-20483). It doesn't work when I try to run it on cluster. > Let me run unit tests to see if it can figure what I have wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20564) Tighter BB Cell Comparator
stack created HBASE-20564: - Summary: Tighter BB Cell Comparator Key: HBASE-20564 URL: https://issues.apache.org/jira/browse/HBASE-20564 Project: HBase Issue Type: Sub-task Components: Performance Reporter: stack Comparing Cells in hbase2 takes almost 3x the CPU. In hbase1, its a keyValue backed by a byte array caching a few important values.. In hbase2, its a NoTagByteBufferChunkKeyValue(?) deserializing the row/family/qualifier lengths repeatedly. I tried making a purposed comparator -- one that was not generic -- and it seemed to have a nicer profile coming close to hbase1 in percentage used (I'll post graphs) when I ran it in my perpetual memstore filler (See scripts attached to HBASE-20483). It doesn't work when I try to run it on cluster. Let me run unit tests to see if it can figure what I have wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470451#comment-16470451 ] Sean Busbey commented on HBASE-20004: - +1 lgtm. thanks [~ashish singhi]! > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Attachments: HBASE-20004.branch-1.patch, > HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch, > HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch, > HBASE-20004.v2.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20563) Use parameterized logging in hbase-common
[
https://issues.apache.org/jira/browse/HBASE-20563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470439#comment-16470439
]
Hadoop QA commented on HBASE-20563:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 15 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
41s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} hbase-common: The patch generated 8 new + 337
unchanged - 31 fixed = 345 total (was 368) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 23s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
38s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20563 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12922846/HBASE-20563.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux bb64eefccc25 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8ba2a7eeb9 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/12780/artifact/patchprocess/diff-checkstyle-hbase-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12780/testReport/ |
| Max. process+thread count | 289 (vs. ulimit of 1) |
| modules | C: hbase-common U: hbase-common |
| Console output |
[jira] [Work started] (HBASE-20477) Document HeapMemoryTuner in refguide
[ https://issues.apache.org/jira/browse/HBASE-20477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-20477 started by Mingdao Yang. > Document HeapMemoryTuner in refguide > > > Key: HBASE-20477 > URL: https://issues.apache.org/jira/browse/HBASE-20477 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: stack >Assignee: Mingdao Yang >Priority: Major > Labels: beginner > Fix For: 3.0.0 > > > This is an old facility that is in danger of disappearing because there is no > mention in refguide and because it is private interface, it does not show in > the user API. Add doc on how it works. It could probably do with a bit of > testing too to make sure it working as advertised. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work stopped] (HBASE-20477) Document HeapMemoryTuner in refguide
[ https://issues.apache.org/jira/browse/HBASE-20477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-20477 stopped by Mingdao Yang. > Document HeapMemoryTuner in refguide > > > Key: HBASE-20477 > URL: https://issues.apache.org/jira/browse/HBASE-20477 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: stack >Assignee: Mingdao Yang >Priority: Major > Labels: beginner > Fix For: 3.0.0 > > > This is an old facility that is in danger of disappearing because there is no > mention in refguide and because it is private interface, it does not show in > the user API. Add doc on how it works. It could probably do with a bit of > testing too to make sure it working as advertised. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20563) Use parameterized logging in hbase-common
[ https://issues.apache.org/jira/browse/HBASE-20563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-20563: Fix Version/s: 3.0.0 Status: Patch Available (was: Open) > Use parameterized logging in hbase-common > - > > Key: HBASE-20563 > URL: https://issues.apache.org/jira/browse/HBASE-20563 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20563.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20563) Use parameterized logging in hbase-common
[ https://issues.apache.org/jira/browse/HBASE-20563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-20563: Attachment: HBASE-20563.master.001.patch > Use parameterized logging in hbase-common > - > > Key: HBASE-20563 > URL: https://issues.apache.org/jira/browse/HBASE-20563 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Attachments: HBASE-20563.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20563) Use parameterized logging in hbase-common
Balazs Meszaros created HBASE-20563: --- Summary: Use parameterized logging in hbase-common Key: HBASE-20563 URL: https://issues.apache.org/jira/browse/HBASE-20563 Project: HBase Issue Type: Sub-task Affects Versions: 3.0.0 Reporter: Balazs Meszaros Assignee: Balazs Meszaros -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20562) [UMBRELLA] Use parameterized logging
Balazs Meszaros created HBASE-20562:
---
Summary: [UMBRELLA] Use parameterized logging
Key: HBASE-20562
URL: https://issues.apache.org/jira/browse/HBASE-20562
Project: HBase
Issue Type: Umbrella
Affects Versions: 3.0.0
Reporter: Balazs Meszaros
We should use parameterized log message feature of slf4j/logback. (Use {}
instead of string concatenation.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470376#comment-16470376 ] Ashish Singhi commented on HBASE-20004: --- Reattaching for another run for branch-1 patch. Will wait for one more day for commit if QA is all ok. Waiting for for your blessings sir, [~stack]. > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Attachments: HBASE-20004.branch-1.patch, > HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch, > HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch, > HBASE-20004.v2.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20004) Client is not able to execute REST queries in a secure cluster
[ https://issues.apache.org/jira/browse/HBASE-20004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-20004: -- Attachment: HBASE-20004.branch-1.v1.patch > Client is not able to execute REST queries in a secure cluster > -- > > Key: HBASE-20004 > URL: https://issues.apache.org/jira/browse/HBASE-20004 > Project: HBase > Issue Type: Bug > Components: REST, security >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Attachments: HBASE-20004.branch-1.patch, > HBASE-20004.branch-1.v1.patch, HBASE-20004.branch-1.v1.patch, > HBASE-20004.branch-2.0.patch, HBASE-20004.patch, HBASE-20004.v1.patch, > HBASE-20004.v2.patch > > > Firefox browser is not able to negotiate REST queries with server in secure > mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19761) Fix Checkstyle errors in hbase-zookeeper
[ https://issues.apache.org/jira/browse/HBASE-19761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470341#comment-16470341 ] maoling commented on HBASE-19761: - [~busbey],[~mdrob] Could you please review this patch when you're idle > Fix Checkstyle errors in hbase-zookeeper > > > Key: HBASE-19761 > URL: https://issues.apache.org/jira/browse/HBASE-19761 > Project: HBase > Issue Type: Sub-task >Reporter: Jan Hentschel >Assignee: maoling >Priority: Minor > Attachments: HBASE-19761-master-v0.patch, > HBASE-19761-master-v1.patch, HBASE-19761-master-v2.patch, > HBASE-19761-master-v3.patch > > > Fix the remaining Checkstyle errors in the *hbase-zookeeper* module and > enable Checkstyle to fail on violations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20544) downstream HBaseTestingUtility fails with invalid port
[
https://issues.apache.org/jira/browse/HBASE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470329#comment-16470329
]
Hudson commented on HBASE-20544:
Results for branch master
[build #326 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/326/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> downstream HBaseTestingUtility fails with invalid port
> --
>
> Key: HBASE-20544
> URL: https://issues.apache.org/jira/browse/HBASE-20544
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20544.0.patch, HBASE-20544.1.patch,
> HBASE-20544.2.patch
>
>
> Attempting to update hbase-downstreamer to use our 2.0.0 release fails with
> an invalid port in the event that {{hbase.localcluster.assign.random.ports}}
> isn't set (or is set to false, specifically):
> {code}
> 2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer
> (HRegionServer.java:(631)) - Failed construction RegionServer
> java.lang.IllegalArgumentException: port out of range:-1
> at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
> at java.net.InetSocketAddress.(InetSocketAddress.java:224)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1217)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1184)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:147)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:121)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
> at
> org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
[jira] [Commented] (HBASE-20539) Disable IMC; part 2
[
https://issues.apache.org/jira/browse/HBASE-20539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470325#comment-16470325
]
Hudson commented on HBASE-20539:
Results for branch master
[build #326 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/326/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Disable IMC; part 2
> ---
>
> Key: HBASE-20539
> URL: https://issues.apache.org/jira/browse/HBASE-20539
> Project: HBase
> Issue Type: Sub-task
> Components: in-memory-compaction
>Reporter: stack
>Assignee: stack
>Priority: Minor
> Fix For: 2.0.1
>
> Attachments: HBASE-20539.branch-2.0.001.patch,
> HBASE-20539.branch-2.0.002.patch, HBASE-20539.branch-2.0.002.patch,
> HBASE-20539.branch-2.0.003.patch
>
>
> Just noticed that post-flush, we are picking up a CompactingMemStore though
> IMC is supposed to be off. We are picking up an inMemoryCompaction of BASIC
> (noticed a day or so ago by [~chia7712]) and so, we fall into the default in
> HStore#getMemstore which is set to be a CompactingMemStore
> No real perf difference going by the compares I've been running but let me
> clean this up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20554) "WALs outstanding" message from CleanerChore is noisy
[
https://issues.apache.org/jira/browse/HBASE-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470327#comment-16470327
]
Hudson commented on HBASE-20554:
Results for branch master
[build #326 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/326/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> "WALs outstanding" message from CleanerChore is noisy
> -
>
> Key: HBASE-20554
> URL: https://issues.apache.org/jira/browse/HBASE-20554
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 3.0.0, 2.1.0, 1.5.0, 2.0.1, 1.4.5
>
> Attachments: HBASE-20554.patch
>
>
> WARN level "WALs outstanding" from CleanerChore should be DEBUG and are not
> always correct.
> I left a cluster configured for ITBLL (retaining all WALs for post hoc
> analysis) and in the morning found the master log full of "WALs outstanding"
> warnings from CleanerChore.
> Should this really be a warning?
> {quote}
> 2018-05-09 16:42:03,893 WARN
> [node-1.cluster,16000,1525851521469_ChoreService_2] cleaner.CleanerChore:
> WALs outstanding under hdfs://node-1.cluster/hbase/oldWALs
> {quote}
> If someone has configured really long WAL retention then having WALs in
> oldWALs will be normal.
> Also, it seems the warning is sometimes incorrect.
> {quote}
> 2018-05-09 16:42:24,751 WARN
> [node-1.cluster,16000,1525851521469_ChoreService_1] cleaner.CleanerChore:
> WALs outstanding under hdfs://node-1.cluster/hbase/archive
> {quote}
> There are no WALs under archive/.
> Even at DEBUG level, if it is not correct, then it can lead an operator to be
> concerned about nothing, so better to just remove it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20545) Improve performance of BaseLoadBalancer.retainAssignment
[
https://issues.apache.org/jira/browse/HBASE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470328#comment-16470328
]
Hudson commented on HBASE-20545:
Results for branch master
[build #326 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/326/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Improve performance of BaseLoadBalancer.retainAssignment
>
>
> Key: HBASE-20545
> URL: https://issues.apache.org/jira/browse/HBASE-20545
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Affects Versions: 1.4.4, 2.0.0
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.5.0, 2.0.1, 1.4.5
>
> Attachments: HBASE-20545.branch-1.4.001.patch,
> HBASE-20545.branch-2.001.patch
>
>
> I was measuring perf at scale with a 1m region table and noticed some
> improvements can be made to BaseLoadBalancer.retainAssignment().
> retainAssignment() spends a few mins to enable a 1m regions table and also
> generates a lot of objects unnecessarily. This jira is to make the most
> common case go faster with very minimal changes. A slightly modified version
> of this patch takes about 5 seconds for a 1m region table ignoring the time
> spent in createCluster(). I think locality can be refreshed during master
> startup in different ways without taking time in retainAssignment, but will
> follow up on that in subsequent jiras. Leaving it untouched here, but wanted
> to call out the time taken without that method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20204) Add locking to RefreshFileConnections in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-20204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470326#comment-16470326
]
Hudson commented on HBASE-20204:
Results for branch master
[build #326 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/326/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/326//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Add locking to RefreshFileConnections in BucketCache
>
>
> Key: HBASE-20204
> URL: https://issues.apache.org/jira/browse/HBASE-20204
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 1.4.3, 2.0.0
>Reporter: Zach York
>Assignee: Zach York
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.5
>
> Attachments: HBASE-20204.master.001.patch,
> HBASE-20204.master.002.patch, HBASE-20204.master.003.patch,
> HBASE-20204.master.004.patch
>
>
> This is a follow-up to HBASE-20141 where [~anoop.hbase] suggested adding
> locking for refreshing channels.
> I have also seen this become an issue when a RS has to abort and it locks on
> trying to flush out the remaining data to the cache (since cache on write was
> turned on).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20554) "WALs outstanding" message from CleanerChore is noisy
[
https://issues.apache.org/jira/browse/HBASE-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470215#comment-16470215
]
Hudson commented on HBASE-20554:
Results for branch branch-1
[build #310 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/310/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/310//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/310//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/310//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> "WALs outstanding" message from CleanerChore is noisy
> -
>
> Key: HBASE-20554
> URL: https://issues.apache.org/jira/browse/HBASE-20554
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 3.0.0, 2.1.0, 1.5.0, 2.0.1, 1.4.5
>
> Attachments: HBASE-20554.patch
>
>
> WARN level "WALs outstanding" from CleanerChore should be DEBUG and are not
> always correct.
> I left a cluster configured for ITBLL (retaining all WALs for post hoc
> analysis) and in the morning found the master log full of "WALs outstanding"
> warnings from CleanerChore.
> Should this really be a warning?
> {quote}
> 2018-05-09 16:42:03,893 WARN
> [node-1.cluster,16000,1525851521469_ChoreService_2] cleaner.CleanerChore:
> WALs outstanding under hdfs://node-1.cluster/hbase/oldWALs
> {quote}
> If someone has configured really long WAL retention then having WALs in
> oldWALs will be normal.
> Also, it seems the warning is sometimes incorrect.
> {quote}
> 2018-05-09 16:42:24,751 WARN
> [node-1.cluster,16000,1525851521469_ChoreService_1] cleaner.CleanerChore:
> WALs outstanding under hdfs://node-1.cluster/hbase/archive
> {quote}
> There are no WALs under archive/.
> Even at DEBUG level, if it is not correct, then it can lead an operator to be
> concerned about nothing, so better to just remove it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
