[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065461#comment-15065461 ] Lars Hofhansl commented on HBASE-14822: --- pushed to master... Working on other branches now. > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15015) Checktyle plugin shouldn't check Jamon-generated Java classes
[ https://issues.apache.org/jira/browse/HBASE-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065502#comment-15065502 ] Mikhail Antonov commented on HBASE-15015: - [~stack] I ran it locally with a patch known to cause jamon warnings and compared the number of warnings reported, they were the same. > Checktyle plugin shouldn't check Jamon-generated Java classes > - > > Key: HBASE-15015 > URL: https://issues.apache.org/jira/browse/HBASE-15015 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Minor > Attachments: HBASE-15015.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[
https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065542#comment-15065542
]
Hadoop QA commented on HBASE-14976:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778714/HBASE-14976-v2.patch
against master branch at commit b68542e35a206a93262fab03fd1c2d8f481cf05e.
ATTACHMENT ID: 12778714
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16942//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16942//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16942//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16942//console
This message is automatically generated.
> Add RPC call queues to the web ui
> -
>
> Key: HBASE-14976
> URL: https://issues.apache.org/jira/browse/HBASE-14976
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Reporter: Elliott Clark
>Assignee: Pallavi Adusumilli
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch,
> HBASE-14976-v2.patch, HBASE-14976.patch
>
>
> The size of the call queue for the regionserver is a critical metric to see
> if things are going too slowly. We should add the call queue size to the ui
> under the queues tab.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065451#comment-15065451 ] Lars Hofhansl commented on HBASE-14822: --- Yeah... Lemme push to all branches. > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-14976: Status: Patch Available (was: Open) > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch, > HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15016) StoreServices facility in Region
Eshcar Hillel created HBASE-15016: - Summary: StoreServices facility in Region Key: HBASE-15016 URL: https://issues.apache.org/jira/browse/HBASE-15016 Project: HBase Issue Type: Sub-task Reporter: Eshcar Hillel The default implementation of a memstore ensures that between two flushes the memstore size increases monotonically. Supporting new memstores that store data in different formats (specifically, compressed), or that allows to eliminate data redundancies in memory (e.g., via compaction), means that the size of the data stored in memory can decrease even between two flushes. This requires memstores to have access to facilities that manipulate region counters and synchronization. This subtasks introduces a new region interface -- StoreServices, through which store components can access these facilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14974) Total number of Regions in Transition number on UI incorrect
[ https://issues.apache.org/jira/browse/HBASE-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065437#comment-15065437 ] Hudson commented on HBASE-14974: FAILURE: Integrated in HBase-0.98-matrix #275 (See [https://builds.apache.org/job/HBase-0.98-matrix/275/]) HBASE-14974 Total number of Regions in Transition number on UI incorrect (apurtell: rev 204c8d59dfd04b0632cb9ebba6c24e14e3d3e8c2) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/AssignmentManagerStatusTmpl.jamon > Total number of Regions in Transition number on UI incorrect > > > Key: HBASE-14974 > URL: https://issues.apache.org/jira/browse/HBASE-14974 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Elliott Clark >Assignee: Mikhail Antonov >Priority: Trivial > Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14974.patch, Screen Shot 2015-12-14 at 11.34.14 > AM.png > > > Total number of Regions in Transition shows 100 when there are 100 or more > regions in transition. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-14822: -- Attachment: 14822-v5-1.3.txt Patch for 1.3 for reference > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5-1.3.txt, 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065528#comment-15065528 ] Mikhail Antonov commented on HBASE-14976: - (thanks Pallavi!) > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch, > HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14968) ConcurrentModificationException in region close resulting in the region staying in closing state
[
https://issues.apache.org/jira/browse/HBASE-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065436#comment-15065436
]
Hudson commented on HBASE-14968:
FAILURE: Integrated in HBase-0.98-matrix #275 (See
[https://builds.apache.org/job/HBase-0.98-matrix/275/])
HBASE-14968 ConcurrentModificationException in region close resulting in
(apurtell: rev 07afd40024b453cd9965cf2af4bf3dceb6739106)
* hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java
> ConcurrentModificationException in region close resulting in the region
> staying in closing state
>
>
> Key: HBASE-14968
> URL: https://issues.apache.org/jira/browse/HBASE-14968
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 0.98.17, 1.0.4
>
> Attachments: hbase-14968_v1.patch
>
>
> We have seen this in our tests. The region gets closed, but the region close
> handler gets an unexpected exception causing the region to stay in closing
> state until RS is restarted.
> The Phoenix and security coprocessors were loaded in that region. Here is the
> stack trace:
> {code}
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:442)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:272)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost.java:496)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1761)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:489)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1502)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1349)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:138)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065467#comment-15065467 ] Lars Hofhansl commented on HBASE-14822: --- 1.3 and 1.2 done > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5-1.3.txt, 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-14976: Status: Open (was: Patch Available) > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch, > HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15015) Checktyle plugin shouldn't check Jamon-generated Java classes
[ https://issues.apache.org/jira/browse/HBASE-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15015: Resolution: Fixed Fix Version/s: 1.3.0 1.2.0 2.0.0 Status: Resolved (was: Patch Available) Pushed to master, branch-1 and branch-1.2. > Checktyle plugin shouldn't check Jamon-generated Java classes > - > > Key: HBASE-15015 > URL: https://issues.apache.org/jira/browse/HBASE-15015 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-15015.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14918) In-Memory MemStore Flush and Compaction
[ https://issues.apache.org/jira/browse/HBASE-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-14918: -- Description: A memstore serves as the in-memory component of a store unit, absorbing all updates to the store. From time to time these updates are flushed to a file on disk, where they are compacted (by eliminating redundancies) and compressed (i.e., written in a compressed format to reduce their storage size). We aim to speed up data access, and therefore suggest to apply in-memory memstore flush. That is to flush the active in-memory segment into an intermediate buffer where it can be accessed by the application. Data in the buffer is subject to compaction and can be stored in any format that allows it to take up smaller space in RAM. The less space the buffer consumes the longer it can reside in memory before data is flushed to disk, resulting in better performance. Specifically, the optimization is beneficial for workloads with medium-to-high key churn which incur many redundant cells, like persistent messaging. We suggest to structure the solution as 4 subtasks (respectively, patches). (1) Infrastructure - refactoring of the MemStore hierarchy, introducing segment (StoreSegment) as first-class citizen, and decoupling memstore scanner from the memstore implementation; (2) Adding StoreServices facility at the region level to allow memstores update region counters and access region level synchronization mechanism; (3) Implementation of a new memstore (CompactingMemstore) with non-optimized immutable segment representation, and (4) Memory optimization including compressed format representation and off heap allocations. This Jira continues the discussion in HBASE-13408. Design documents, evaluation results and previous patches can be found in HBASE-13408. was: A memstore serves as the in-memory component of a store unit, absorbing all updates to the store. From time to time these updates are flushed to a file on disk, where they are compacted (by eliminating redundancies) and compressed (i.e., written in a compressed format to reduce their storage size). We aim to speed up data access, and therefore suggest to apply in-memory memstore flush. That is to flush the active in-memory segment into an intermediate buffer where it can be accessed by the application. Data in the buffer is subject to compaction and can be stored in any format that allows it to take up smaller space in RAM. The less space the buffer consumes the longer it can reside in memory before data is flushed to disk, resulting in better performance. Specifically, the optimization is beneficial for workloads with medium-to-high key churn which incur many redundant cells, like persistent messaging. We suggest to structure the solution as 3 subtasks (respectively, patches). (1) Infrastructure - refactoring of the MemStore hierarchy, introducing segment (StoreSegment) as first-class citizen, and decoupling memstore scanner from the memstore implementation; (2) Implementation of a new memstore (CompactingMemstore) with non-optimized immutable segment representation, and (3) Memory optimization including compressed format representation and offheap allocations. This Jira continues the discussion in HBASE-13408. Design documents, evaluation results and previous patches can be found in HBASE-13408. > In-Memory MemStore Flush and Compaction > --- > > Key: HBASE-14918 > URL: https://issues.apache.org/jira/browse/HBASE-14918 > Project: HBase > Issue Type: Umbrella >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > > A memstore serves as the in-memory component of a store unit, absorbing all > updates to the store. From time to time these updates are flushed to a file > on disk, where they are compacted (by eliminating redundancies) and > compressed (i.e., written in a compressed format to reduce their storage > size). > We aim to speed up data access, and therefore suggest to apply in-memory > memstore flush. That is to flush the active in-memory segment into an > intermediate buffer where it can be accessed by the application. Data in the > buffer is subject to compaction and can be stored in any format that allows > it to take up smaller space in RAM. The less space the buffer consumes the > longer it can reside in memory before data is flushed to disk, resulting in > better performance. > Specifically, the optimization is beneficial for workloads with > medium-to-high key churn which incur many redundant cells, like persistent > messaging. > We suggest to structure the solution as 4 subtasks (respectively, patches). > (1) Infrastructure - refactoring of the MemStore hierarchy, introducing > segment (StoreSegment) as first-class citizen, and
[jira] [Commented] (HBASE-11290) Unlock RegionStates
[ https://issues.apache.org/jira/browse/HBASE-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065435#comment-15065435 ] Francis Liu commented on HBASE-11290: - Sorry for the delay guys. i've actually been working on a trunk rebase and just need to run the unit tests. If its alright can I upload the patch and reassign this jira? > Unlock RegionStates > --- > > Key: HBASE-11290 > URL: https://issues.apache.org/jira/browse/HBASE-11290 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-11290-0.98.patch, HBASE-11290-0.98_v2.patch, > HBASE-11290.draft.patch > > > Even though RegionStates is a highly accessed data structure in HMaster. Most > of it's methods are synchronized. Which limits concurrency. Even simply > making some of the getters non-synchronized by using concurrent data > structures has helped with region assignments. We can go as simple as this > approach or create locks per region or a bucket lock per region bucket. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[
https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065477#comment-15065477
]
Hadoop QA commented on HBASE-14822:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778712/14822-v5-1.3.txt
against master branch at commit 86a417eeade1fc1b0a7799a48694e6f1d9b4cb1c.
ATTACHMENT ID: 12778712
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16940//console
This message is automatically generated.
> Renewing leases of scanners doesn't work
>
>
> Key: HBASE-14822
> URL: https://issues.apache.org/jira/browse/HBASE-14822
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.14
>Reporter: Samarth Jain
>Assignee: Lars Hofhansl
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt,
> 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt,
> 14822-v5-1.3.txt, 14822-v5.txt, 14822.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-14822: -- Resolution: Fixed Hadoop Flags: Reviewed Release Note: And 1.1, 1.0, and 0.98. Status: Resolved (was: Patch Available) > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5-1.3.txt, 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-14976: Attachment: HBASE-14976-v2.patch reattached > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch, > HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-14976: Resolution: Fixed Status: Resolved (was: Patch Available) > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch, > HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065527#comment-15065527 ] Mikhail Antonov commented on HBASE-14976: - Pushed to master and branch-1 ([~busbey] - do you want it in 1.2?) > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch, > HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14919) Infrastructure refactoring
[ https://issues.apache.org/jira/browse/HBASE-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065538#comment-15065538 ] Eshcar Hillel commented on HBASE-14919: --- Sure [~stack] I understand. Meanwhile I will start working on the second sub-task we defined (already updated HBASE-14918). > Infrastructure refactoring > -- > > Key: HBASE-14919 > URL: https://issues.apache.org/jira/browse/HBASE-14919 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-14919-V01.patch, HBASE-14919-V01.patch, > HBASE-14919-V02.patch, HBASE-14919-V03.patch, HBASE-14919-V04.patch > > > Refactoring the MemStore hierarchy, introducing segment (StoreSegment) as > first-class citizen and decoupling memstore scanner from the memstore > implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065562#comment-15065562 ] Hudson commented on HBASE-14976: SUCCESS: Integrated in HBase-1.3-IT #392 (See [https://builds.apache.org/job/HBase-1.3-IT/392/]) HBASE-14976 Add RPC call queues to the web ui (Pallavi Adusumilli) (antonov: rev 79baaeb540f3ce575ea58294eaf2ad98ac49786e) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServer.java > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch, > HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15015) Checktyle plugin shouldn't check Jamon-generated Java classes
[ https://issues.apache.org/jira/browse/HBASE-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065563#comment-15065563 ] Hudson commented on HBASE-15015: SUCCESS: Integrated in HBase-1.3-IT #392 (See [https://builds.apache.org/job/HBase-1.3-IT/392/]) HBASE-15015 Checktyle plugin shouldn't check Jamon-generated Java (antonov: rev 6564cae667caef32713d582eca95ebf66c7dddf3) * hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml > Checktyle plugin shouldn't check Jamon-generated Java classes > - > > Key: HBASE-15015 > URL: https://issues.apache.org/jira/browse/HBASE-15015 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-15015.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14968) ConcurrentModificationException in region close resulting in the region staying in closing state
[
https://issues.apache.org/jira/browse/HBASE-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065439#comment-15065439
]
Hudson commented on HBASE-14968:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1148 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1148/])
HBASE-14968 ConcurrentModificationException in region close resulting in
(apurtell: rev 07afd40024b453cd9965cf2af4bf3dceb6739106)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
> ConcurrentModificationException in region close resulting in the region
> staying in closing state
>
>
> Key: HBASE-14968
> URL: https://issues.apache.org/jira/browse/HBASE-14968
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 0.98.17, 1.0.4
>
> Attachments: hbase-14968_v1.patch
>
>
> We have seen this in our tests. The region gets closed, but the region close
> handler gets an unexpected exception causing the region to stay in closing
> state until RS is restarted.
> The Phoenix and security coprocessors were loaded in that region. Here is the
> stack trace:
> {code}
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:442)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:272)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost.java:496)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1761)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:489)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1502)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1349)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:138)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11290) Unlock RegionStates
[ https://issues.apache.org/jira/browse/HBASE-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065492#comment-15065492 ] Mikhail Antonov commented on HBASE-11290: - Sure! (it's assigned to you know). I'll stop my rebasing-in-progress then. Will help to review once you upload. > Unlock RegionStates > --- > > Key: HBASE-11290 > URL: https://issues.apache.org/jira/browse/HBASE-11290 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-11290-0.98.patch, HBASE-11290-0.98_v2.patch, > HBASE-11290.draft.patch > > > Even though RegionStates is a highly accessed data structure in HMaster. Most > of it's methods are synchronized. Which limits concurrency. Even simply > making some of the getters non-synchronized by using concurrent data > structures has helped with region assignments. We can go as simple as this > approach or create locks per region or a bucket lock per region bucket. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065497#comment-15065497 ] Hudson commented on HBASE-14822: SUCCESS: Integrated in HBase-1.2-IT #354 (See [https://builds.apache.org/job/HBase-1.2-IT/354/]) HBASE-14822 Renewing leases of scanners doesn't work. (larsh: rev 435cc3f6b909b637a9245c2aac96113290f8bc3c) * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestLeaseRenewal.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-protocol/src/main/protobuf/Client.proto * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5-1.3.txt, 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065481#comment-15065481 ] Hudson commented on HBASE-14822: SUCCESS: Integrated in HBase-1.3-IT #391 (See [https://builds.apache.org/job/HBase-1.3-IT/391/]) HBASE-14822 Renewing leases of scanners doesn't work. (larsh: rev c27d2c9c3e2fe59906ac6ba7ac2a5d08e5928d43) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestLeaseRenewal.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * hbase-protocol/src/main/protobuf/Client.proto > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5-1.3.txt, 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[
https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065545#comment-15065545
]
Hadoop QA commented on HBASE-14976:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778714/HBASE-14976-v2.patch
against master branch at commit b68542e35a206a93262fab03fd1c2d8f481cf05e.
ATTACHMENT ID: 12778714
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16943//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16943//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16943//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16943//console
This message is automatically generated.
> Add RPC call queues to the web ui
> -
>
> Key: HBASE-14976
> URL: https://issues.apache.org/jira/browse/HBASE-14976
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Reporter: Elliott Clark
>Assignee: Pallavi Adusumilli
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch,
> HBASE-14976-v2.patch, HBASE-14976.patch
>
>
> The size of the call queue for the regionserver is a critical metric to see
> if things are going too slowly. We should add the call queue size to the ui
> under the queues tab.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14974) Total number of Regions in Transition number on UI incorrect
[ https://issues.apache.org/jira/browse/HBASE-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065440#comment-15065440 ] Hudson commented on HBASE-14974: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1148 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1148/]) HBASE-14974 Total number of Regions in Transition number on UI incorrect (apurtell: rev 204c8d59dfd04b0632cb9ebba6c24e14e3d3e8c2) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/AssignmentManagerStatusTmpl.jamon > Total number of Regions in Transition number on UI incorrect > > > Key: HBASE-14974 > URL: https://issues.apache.org/jira/browse/HBASE-14974 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Elliott Clark >Assignee: Mikhail Antonov >Priority: Trivial > Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14974.patch, Screen Shot 2015-12-14 at 11.34.14 > AM.png > > > Total number of Regions in Transition shows 100 when there are 100 or more > regions in transition. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-15016) StoreServices facility in Region
[ https://issues.apache.org/jira/browse/HBASE-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel reassigned HBASE-15016: - Assignee: Eshcar Hillel > StoreServices facility in Region > > > Key: HBASE-15016 > URL: https://issues.apache.org/jira/browse/HBASE-15016 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > > The default implementation of a memstore ensures that between two flushes the > memstore size increases monotonically. Supporting new memstores that store > data in different formats (specifically, compressed), or that allows to > eliminate data redundancies in memory (e.g., via compaction), means that the > size of the data stored in memory can decrease even between two flushes. This > requires memstores to have access to facilities that manipulate region > counters and synchronization. > This subtasks introduces a new region interface -- StoreServices, through > which store components can access these facilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-14921) Memory optimizations
[ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel reassigned HBASE-14921: - Assignee: Eshcar Hillel > Memory optimizations > > > Key: HBASE-14921 > URL: https://issues.apache.org/jira/browse/HBASE-14921 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > > Memory optimizations including compressed format representation and offheap > allocations -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-14921) Memory optimizations
[ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel reassigned HBASE-14921: - Assignee: (was: Eshcar Hillel) > Memory optimizations > > > Key: HBASE-14921 > URL: https://issues.apache.org/jira/browse/HBASE-14921 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel > > Memory optimizations including compressed format representation and offheap > allocations -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065549#comment-15065549 ] Hudson commented on HBASE-14822: FAILURE: Integrated in HBase-1.3 #454 (See [https://builds.apache.org/job/HBase-1.3/454/]) HBASE-14822 Renewing leases of scanners doesn't work. (larsh: rev c27d2c9c3e2fe59906ac6ba7ac2a5d08e5928d43) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestLeaseRenewal.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * hbase-protocol/src/main/protobuf/Client.proto * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5-1.3.txt, 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15015) Checktyle plugin shouldn't check Jamon-generated Java classes
[ https://issues.apache.org/jira/browse/HBASE-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065548#comment-15065548 ] Hudson commented on HBASE-15015: FAILURE: Integrated in HBase-1.3 #454 (See [https://builds.apache.org/job/HBase-1.3/454/]) HBASE-15015 Checktyle plugin shouldn't check Jamon-generated Java (antonov: rev 6564cae667caef32713d582eca95ebf66c7dddf3) * hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml > Checktyle plugin shouldn't check Jamon-generated Java classes > - > > Key: HBASE-15015 > URL: https://issues.apache.org/jira/browse/HBASE-15015 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-15015.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065613#comment-15065613 ] Sean Busbey commented on HBASE-14976: - +1 for 1.2. please also set fix versions and include a release note. > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch, > HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11792) Organize PerformanceEvaluation usage output
[ https://issues.apache.org/jira/browse/HBASE-11792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065534#comment-15065534 ] Nick Dimiduk commented on HBASE-11792: -- None of the options are changed, only the format of the help message, right? Should be good for all branches. +1 thanks [~misty]. Want to re-attach the patch to see about getting a build run? > Organize PerformanceEvaluation usage output > --- > > Key: HBASE-11792 > URL: https://issues.apache.org/jira/browse/HBASE-11792 > Project: HBase > Issue Type: Improvement > Components: Performance, test >Reporter: Nick Dimiduk >Assignee: Misty Stanley-Jones >Priority: Minor > Labels: beginner > Attachments: HBASE-11792-0.98.patch, HBASE-11792-branch-1.0.patch, > HBASE-11792-branch-1.1.patch, HBASE-11792-branch-1.2.patch, > HBASE-11792-branch-1.patch, HBASE-11792.patch > > > PerformanceEvaluation has enjoyed a good bit of attention recently. All the > new features are muddled together. It would be nice to organize the output of > the Options list according to some scheme. I was thinking you're group > entries by when they're used. For example > *General options* > - nomapred > - rows > - oneCon > - ... > *Table Creation/Write tests* > - compress > - flushCommits > - valueZipf > - ... > *Read tests* > - filterAll > - multiGet > - replicas > - ... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065586#comment-15065586 ] Hudson commented on HBASE-14976: FAILURE: Integrated in HBase-1.3 #455 (See [https://builds.apache.org/job/HBase-1.3/455/]) HBASE-14976 Add RPC call queues to the web ui (Pallavi Adusumilli) (antonov: rev 79baaeb540f3ce575ea58294eaf2ad98ac49786e) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServer.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch, > HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065578#comment-15065578 ] Hudson commented on HBASE-14822: FAILURE: Integrated in HBase-Trunk_matrix #571 (See [https://builds.apache.org/job/HBase-Trunk_matrix/571/]) HBASE-14822 Renewing leases of scanners doesn't work. (larsh: rev 86a417eeade1fc1b0a7799a48694e6f1d9b4cb1c) * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestLeaseRenewal.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * hbase-protocol/src/main/protobuf/Client.proto * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5-1.3.txt, 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065576#comment-15065576 ] Hudson commented on HBASE-14976: FAILURE: Integrated in HBase-Trunk_matrix #571 (See [https://builds.apache.org/job/HBase-Trunk_matrix/571/]) HBASE-14976 Add RPC call queues to the web ui (Pallavi Adusumilli) (antonov: rev 8c921ea94f330b91d4fd55aa1706fc64b1646de1) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServer.java > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2.patch, > HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15015) Checktyle plugin shouldn't check Jamon-generated Java classes
[ https://issues.apache.org/jira/browse/HBASE-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065577#comment-15065577 ] Hudson commented on HBASE-15015: FAILURE: Integrated in HBase-Trunk_matrix #571 (See [https://builds.apache.org/job/HBase-Trunk_matrix/571/]) HBASE-15015 Checktyle plugin shouldn't check Jamon-generated Java (antonov: rev b68542e35a206a93262fab03fd1c2d8f481cf05e) * hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml > Checktyle plugin shouldn't check Jamon-generated Java classes > - > > Key: HBASE-15015 > URL: https://issues.apache.org/jira/browse/HBASE-15015 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-15015.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065565#comment-15065565 ] Hudson commented on HBASE-14822: FAILURE: Integrated in HBase-1.2 #460 (See [https://builds.apache.org/job/HBase-1.2/460/]) HBASE-14822 Renewing leases of scanners doesn't work. (larsh: rev 435cc3f6b909b637a9245c2aac96113290f8bc3c) * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * hbase-protocol/src/main/protobuf/Client.proto * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestLeaseRenewal.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5-1.3.txt, 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065604#comment-15065604 ] Hudson commented on HBASE-14822: FAILURE: Integrated in HBase-1.1-JDK8 #1711 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1711/]) HBASE-14822 Renewing leases of scanners doesn't work. (larsh: rev f64cf4d8c6e09ddceda7fe737daeb648ca3fa8b4) * hbase-protocol/src/main/protobuf/Client.proto * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestLeaseRenewal.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5-1.3.txt, 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065605#comment-15065605 ] Hudson commented on HBASE-14849: FAILURE: Integrated in HBase-Trunk_matrix #572 (See [https://builds.apache.org/job/HBase-Trunk_matrix/572/]) HBASE-14849 Add option to set block cache to false on SparkSQL (tedyu: rev e75e26e3c6cfe7fd378081839d60fc711c1e095f) * hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala * hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseTestSource.scala * hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseSparkConf.scala * hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/SerializableConfiguration.scala * hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/Bound.scala * hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/DefaultSource.scala * hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Fix For: 2.0.0 > > Attachments: HBASE-14849-1.patch, HBASE-14849-2.patch, > HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14849: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) Thanks for the patch, Zhan. Thanks for the review, Ted. > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Fix For: 2.0.0 > > Attachments: HBASE-14849-1.patch, HBASE-14849-2.patch, > HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15015) Checktyle plugin shouldn't check Jamon-generated Java classes
[ https://issues.apache.org/jira/browse/HBASE-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065579#comment-15065579 ] Hudson commented on HBASE-15015: SUCCESS: Integrated in HBase-1.2-IT #355 (See [https://builds.apache.org/job/HBase-1.2-IT/355/]) HBASE-15015 Checktyle plugin shouldn't check Jamon-generated Java (antonov: rev 39e845e83e6f538d3a5861b56ae6c1d30fbe78f0) * hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml > Checktyle plugin shouldn't check Jamon-generated Java classes > - > > Key: HBASE-15015 > URL: https://issues.apache.org/jira/browse/HBASE-15015 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-15015.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14918) In-Memory MemStore Flush and Compaction
[ https://issues.apache.org/jira/browse/HBASE-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 再再 updated HBASE-14918: --- Fix Version/s: 0.98.18 > In-Memory MemStore Flush and Compaction > --- > > Key: HBASE-14918 > URL: https://issues.apache.org/jira/browse/HBASE-14918 > Project: HBase > Issue Type: Umbrella >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Fix For: 0.98.18 > > > A memstore serves as the in-memory component of a store unit, absorbing all > updates to the store. From time to time these updates are flushed to a file > on disk, where they are compacted (by eliminating redundancies) and > compressed (i.e., written in a compressed format to reduce their storage > size). > We aim to speed up data access, and therefore suggest to apply in-memory > memstore flush. That is to flush the active in-memory segment into an > intermediate buffer where it can be accessed by the application. Data in the > buffer is subject to compaction and can be stored in any format that allows > it to take up smaller space in RAM. The less space the buffer consumes the > longer it can reside in memory before data is flushed to disk, resulting in > better performance. > Specifically, the optimization is beneficial for workloads with > medium-to-high key churn which incur many redundant cells, like persistent > messaging. > We suggest to structure the solution as 4 subtasks (respectively, patches). > (1) Infrastructure - refactoring of the MemStore hierarchy, introducing > segment (StoreSegment) as first-class citizen, and decoupling memstore > scanner from the memstore implementation; > (2) Adding StoreServices facility at the region level to allow memstores > update region counters and access region level synchronization mechanism; > (3) Implementation of a new memstore (CompactingMemstore) with non-optimized > immutable segment representation, and > (4) Memory optimization including compressed format representation and off > heap allocations. > This Jira continues the discussion in HBASE-13408. > Design documents, evaluation results and previous patches can be found in > HBASE-13408. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065595#comment-15065595 ] Hudson commented on HBASE-14822: FAILURE: Integrated in HBase-1.1-JDK7 #1624 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1624/]) HBASE-14822 Renewing leases of scanners doesn't work. (larsh: rev f64cf4d8c6e09ddceda7fe737daeb648ca3fa8b4) * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-protocol/src/main/protobuf/Client.proto * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestLeaseRenewal.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5-1.3.txt, 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15015) Checktyle plugin shouldn't check Jamon-generated Java classes
[ https://issues.apache.org/jira/browse/HBASE-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065596#comment-15065596 ] Hudson commented on HBASE-15015: SUCCESS: Integrated in HBase-1.2 #461 (See [https://builds.apache.org/job/HBase-1.2/461/]) HBASE-15015 Checktyle plugin shouldn't check Jamon-generated Java (antonov: rev 39e845e83e6f538d3a5861b56ae6c1d30fbe78f0) * hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml > Checktyle plugin shouldn't check Jamon-generated Java classes > - > > Key: HBASE-15015 > URL: https://issues.apache.org/jira/browse/HBASE-15015 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-15015.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14987) HBaseFsck#mergeRegionDirs() needs to handle compaction marker in recovered edits
[
https://issues.apache.org/jira/browse/HBASE-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065626#comment-15065626
]
Ted Yu edited comment on HBASE-14987 at 12/20/15 4:59 AM:
--
Patch v2 adds a test.
Thanks to Stephen for offline discussion
was (Author: yuzhih...@gmail.com):
Patch v2 adds a test.
> HBaseFsck#mergeRegionDirs() needs to handle compaction marker in recovered
> edits
>
>
> Key: HBASE-14987
> URL: https://issues.apache.org/jira/browse/HBASE-14987
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Stephen Yuan Jiang
> Attachments: 14987-suggest.txt, 14987-v1.txt, 14987-v2.txt
>
>
> One customer encountered the following error when replaying recovered edits,
> leading to region open failure:
> {code}
> region=table1,d6b-2282-9223370590058224807-U-9856557-
> EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.,
> starting to roll back the global memstore size.
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Compaction marker
> from WAL table_name: "table1"
> encoded_region_name: "d389c70fde9ec07971d0cfd20ef8f575"
> ...
> region_name:
> "table1,d6b-2282-9223370590058224807-U-9856557-EJ452727-16313786400171,1449089609367.d389c70fde9ec07971d0cfd20ef8f575."
> targetted for region d389c70fde9ec07971d0cfd20ef8f575 does not match this
> region: {ENCODED => fa8a526f2578eb3630bb08a4b1648f5d, NAME =>
> 'table1,d6b-2282-
> 9223370590058224807-U-9856557-EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.',
> STARTKEY => 'd6b-2282-9223370590058224807-U-9856557-EJ452727-
> 16313786400171', ENDKEY =>
> 'd76-2553-9223370588576178807-U-7416904-EK875822-1766218060'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkTargetRegion(HRegion.java:4592)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayWALCompactionMarker(HRegion.java:3831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:3747)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3601)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:911)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:789)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:762)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5774)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5744)
> {code}
> This was likely caused by the following action of hbck:
> {code}
> 15/12/08 18:11:34 INFO util.HBaseFsck: [hbasefsck-pool1-t37] Moving files
> from
> hdfs://Zealand/hbase/data/default/table1/d389c70fde9ec07971d0cfd20ef8f575/recovered.edits
> into containing region
> hdfs://Zealand/hbase/data/default/table1/fa8a526f2578eb3630bb08a4b1648f5d/recovered.edits
> {code}
> The recovered.edits for d389c70fde9ec07971d0cfd20ef8f575 contained compaction
> marker which couldn't be replayed against fa8a526f2578eb3630bb08a4b1648f5d
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14987) HBaseFsck#mergeRegionDirs() needs to handle compaction marker in recovered edits
[
https://issues.apache.org/jira/browse/HBASE-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14987:
---
Attachment: 14987-v2.txt
Patch v2 adds a test.
> HBaseFsck#mergeRegionDirs() needs to handle compaction marker in recovered
> edits
>
>
> Key: HBASE-14987
> URL: https://issues.apache.org/jira/browse/HBASE-14987
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Stephen Yuan Jiang
> Attachments: 14987-suggest.txt, 14987-v1.txt, 14987-v2.txt
>
>
> One customer encountered the following error when replaying recovered edits,
> leading to region open failure:
> {code}
> region=table1,d6b-2282-9223370590058224807-U-9856557-
> EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.,
> starting to roll back the global memstore size.
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Compaction marker
> from WAL table_name: "table1"
> encoded_region_name: "d389c70fde9ec07971d0cfd20ef8f575"
> ...
> region_name:
> "table1,d6b-2282-9223370590058224807-U-9856557-EJ452727-16313786400171,1449089609367.d389c70fde9ec07971d0cfd20ef8f575."
> targetted for region d389c70fde9ec07971d0cfd20ef8f575 does not match this
> region: {ENCODED => fa8a526f2578eb3630bb08a4b1648f5d, NAME =>
> 'table1,d6b-2282-
> 9223370590058224807-U-9856557-EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.',
> STARTKEY => 'd6b-2282-9223370590058224807-U-9856557-EJ452727-
> 16313786400171', ENDKEY =>
> 'd76-2553-9223370588576178807-U-7416904-EK875822-1766218060'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkTargetRegion(HRegion.java:4592)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayWALCompactionMarker(HRegion.java:3831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:3747)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3601)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:911)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:789)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:762)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5774)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5744)
> {code}
> This was likely caused by the following action of hbck:
> {code}
> 15/12/08 18:11:34 INFO util.HBaseFsck: [hbasefsck-pool1-t37] Moving files
> from
> hdfs://Zealand/hbase/data/default/table1/d389c70fde9ec07971d0cfd20ef8f575/recovered.edits
> into containing region
> hdfs://Zealand/hbase/data/default/table1/fa8a526f2578eb3630bb08a4b1648f5d/recovered.edits
> {code}
> The recovered.edits for d389c70fde9ec07971d0cfd20ef8f575 contained compaction
> marker which couldn't be replayed against fa8a526f2578eb3630bb08a4b1648f5d
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14940) Make our unsafe based ops more safe
[ https://issues.apache.org/jira/browse/HBASE-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065310#comment-15065310 ] stack commented on HBASE-14940: --- Not from me. > Make our unsafe based ops more safe > --- > > Key: HBASE-14940 > URL: https://issues.apache.org/jira/browse/HBASE-14940 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Attachments: HBASE-14940.patch, HBASE-14940_branch-1.patch, > HBASE-14940_branch-1.patch, HBASE-14940_branch-1.patch > > > Thanks for the nice findings [~ikeda] > This jira solves 3 issues with Unsafe operations and ByteBufferUtils > 1. We can do sun unsafe based reads and writes iff unsafe package is > available and underlying platform is having unaligned-access capability. But > we were missing the second check > 2. Java NIO is doing a chunk based copy while doing Unsafe copyMemory. The > max chunk size is 1 MB. This is done for "A limit is imposed to allow for > safepoint polling during a large copy" as mentioned in comments in Bits.java. > We are also going to do same way > 3. In ByteBufferUtils, when Unsafe is not available and ByteBuffers are off > heap, we were doing byte by byte operation (read/copy). We can avoid this and > do better way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[ https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065306#comment-15065306 ] stack commented on HBASE-13158: --- Looks like we got the report working again. This is good to commit I'd say. > When client supports CellBlock, return the result Cells as controller payload > for get(Get) API also > --- > > Key: HBASE-13158 > URL: https://issues.apache.org/jira/browse/HBASE-13158 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 1.3.0 > > Attachments: 13158v4.suggestion.txt, HBASE-13158.patch, > HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch, > HBASE-13158_V8.patch, HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15009) Update test-patch.sh on branches; to fix curtailed build report
[ https://issues.apache.org/jira/browse/HBASE-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15009: -- Resolution: Fixed Status: Resolved (was: Patch Available) Looks like reports on branches are full again rather than one liners showing overall only. > Update test-patch.sh on branches; to fix curtailed build report > --- > > Key: HBASE-15009 > URL: https://issues.apache.org/jira/browse/HBASE-15009 > Project: HBase > Issue Type: Bug > Components: build >Reporter: stack >Assignee: stack > Fix For: 1.2.0, 1.3.0 > > Attachments: 15009.branch-1.2.patch, do_nothing-branch-1.patch, > do_nothing-branch-1.patch > > > Backport changes to test-patch.sh to branches. Looks like test-patch got some > changes already but not all so currently falis trying to run the missing > zombie detector script. Try backporting current state of test-patch.sh. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[ https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065362#comment-15065362 ] Hudson commented on HBASE-13158: FAILURE: Integrated in HBase-1.3 #453 (See [https://builds.apache.org/job/HBase-1.3/453/]) HBASE-13158 When client supports CellBlock, return the result Cells as (anoopsamjohn: rev 574774d8b103b958880ce2bb7f605568511eabda) * hbase-protocol/src/main/protobuf/HBase.proto * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/client/VersionInfoUtil.java > When client supports CellBlock, return the result Cells as controller payload > for get(Get) API also > --- > > Key: HBASE-13158 > URL: https://issues.apache.org/jira/browse/HBASE-13158 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 1.3.0 > > Attachments: 13158v4.suggestion.txt, HBASE-13158.patch, > HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch, > HBASE-13158_V8.patch, HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-11290) Unlock RegionStates
[ https://issues.apache.org/jira/browse/HBASE-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov reassigned HBASE-11290: --- Assignee: Mikhail Antonov (was: Francis Liu) > Unlock RegionStates > --- > > Key: HBASE-11290 > URL: https://issues.apache.org/jira/browse/HBASE-11290 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-11290-0.98.patch, HBASE-11290-0.98_v2.patch, > HBASE-11290.draft.patch > > > Even though RegionStates is a highly accessed data structure in HMaster. Most > of it's methods are synchronized. Which limits concurrency. Even simply > making some of the getters non-synchronized by using concurrent data > structures has helped with region assignments. We can go as simple as this > approach or create locks per region or a bucket lock per region bucket. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[ https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-13158: --- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to 1.3 and trunk. Thanks for the reviews. > When client supports CellBlock, return the result Cells as controller payload > for get(Get) API also > --- > > Key: HBASE-13158 > URL: https://issues.apache.org/jira/browse/HBASE-13158 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 1.3.0 > > Attachments: 13158v4.suggestion.txt, HBASE-13158.patch, > HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch, > HBASE-13158_V8.patch, HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11290) Unlock RegionStates
[ https://issues.apache.org/jira/browse/HBASE-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-11290: Assignee: Francis Liu (was: Mikhail Antonov) > Unlock RegionStates > --- > > Key: HBASE-11290 > URL: https://issues.apache.org/jira/browse/HBASE-11290 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: HBASE-11290-0.98.patch, HBASE-11290-0.98_v2.patch, > HBASE-11290.draft.patch > > > Even though RegionStates is a highly accessed data structure in HMaster. Most > of it's methods are synchronized. Which limits concurrency. Even simply > making some of the getters non-synchronized by using concurrent data > structures has helped with region assignments. We can go as simple as this > approach or create locks per region or a bucket lock per region bucket. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14420) Zombie Stomping Session
[ https://issues.apache.org/jira/browse/HBASE-14420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065404#comment-15065404 ] Jonathan Hsieh commented on HBASE-14420: Great stuff Stack. > Zombie Stomping Session > --- > > Key: HBASE-14420 > URL: https://issues.apache.org/jira/browse/HBASE-14420 > Project: HBase > Issue Type: Umbrella > Components: test >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: hangers.txt, none_fix (1).txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, > none_fix.txt > > > Patch build are now failing most of the time because we are dropping zombies. > I confirm we are doing this on non-apache build boxes too. > Left-over zombies consume resources on build boxes (OOME cannot create native > threads). Having to do multiple test runs in the hope that we can get a > non-zombie-making build or making (arbitrary) rulings that the zombies are > 'not related' is a productivity sink. And so on... > This is an umbrella issue for a zombie stomping session that started earlier > this week. Will hang sub-issues of this one. Am running builds back-to-back > on little cluster to turn out the monsters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[ https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065410#comment-15065410 ] Hudson commented on HBASE-13158: FAILURE: Integrated in HBase-1.3-IT #390 (See [https://builds.apache.org/job/HBase-1.3-IT/390/]) HBASE-13158 When client supports CellBlock, return the result Cells as (anoopsamjohn: rev 574774d8b103b958880ce2bb7f605568511eabda) * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * hbase-server/src/main/java/org/apache/hadoop/hbase/client/VersionInfoUtil.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-protocol/src/main/protobuf/HBase.proto * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java > When client supports CellBlock, return the result Cells as controller payload > for get(Get) API also > --- > > Key: HBASE-13158 > URL: https://issues.apache.org/jira/browse/HBASE-13158 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 1.3.0 > > Attachments: 13158v4.suggestion.txt, HBASE-13158.patch, > HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch, > HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch, > HBASE-13158_V8.patch, HBASE-13158_branch-1.patch, HBASE-13158_branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
