[jira] [Assigned] (HBASE-16807) RegionServer will fail to report new active Hmaster until HMaster/RegionServer failover
[
https://issues.apache.org/jira/browse/HBASE-16807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar reassigned HBASE-16807:
Assignee: Pankaj Kumar
> RegionServer will fail to report new active Hmaster until
> HMaster/RegionServer failover
> ---
>
> Key: HBASE-16807
> URL: https://issues.apache.org/jira/browse/HBASE-16807
> Project: HBase
> Issue Type: Bug
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>
> It's little weird, but it happened in the product environment that few
> RegionServer missed master znode create notification on master failover. In
> that case ZooKeeperNodeTracker will not refresh the cached data and
> MasterAddressTracker
> will always return old active HM detail to Region server on ServiceException.
> Though We create region server stub on failure but without refreshing the
> MasterAddressTracker data.
> In HRegionServer.createRegionServerStatusStub()
> {code}
> boolean refresh = false; // for the first time, use cached data
> RegionServerStatusService.BlockingInterface intf = null;
> boolean interrupted = false;
> try {
> while (keepLooping()) {
> sn = this.masterAddressTracker.getMasterAddress(refresh);
> if (sn == null) {
> if (!keepLooping()) {
> // give up with no connection.
> LOG.debug("No master found and cluster is stopped; bailing out");
> return null;
> }
> if (System.currentTimeMillis() > (previousLogTime + 1000)) {
> LOG.debug("No master found; retry");
> previousLogTime = System.currentTimeMillis();
> }
> refresh = true; // let's try pull it from ZK directly
> if (sleep(200)) {
> interrupted = true;
> }
> continue;
> }
> {code}
> Here we refresh node only when 'sn' is NULL otherwise it will use same cached
> data.
> So in above case RegionServer will never report active HMaster successfully
> until HMaster failover or RegionServer restart.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16505) Pass deadline to HRegion operations
[ https://issues.apache.org/jira/browse/HBASE-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-16505: -- Attachment: HBASE-16505-branch-1-v3.patch > Pass deadline to HRegion operations > --- > > Key: HBASE-16505 > URL: https://issues.apache.org/jira/browse/HBASE-16505 > Project: HBase > Issue Type: Sub-task >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0 > > Attachments: HBASE-16505-branch-1-v1.patch, > HBASE-16505-branch-1-v2.patch, HBASE-16505-branch-1-v3.patch, > HBASE-16505-v1.patch, HBASE-16505-v10.patch, HBASE-16505-v10.patch, > HBASE-16505-v11.patch, HBASE-16505-v11.patch, HBASE-16505-v12.patch, > HBASE-16505-v13.patch, HBASE-16505-v2.patch, HBASE-16505-v3.patch, > HBASE-16505-v4.patch, HBASE-16505-v5.patch, HBASE-16505-v6.patch, > HBASE-16505-v7.patch, HBASE-16505-v8.patch, HBASE-16505-v9.patch > > > If we want to know the correct setting of timeout in read/write path, we need > add a new parameter in operation-methods of Region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
[
https://issues.apache.org/jira/browse/HBASE-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16806:
--
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
Resolving as won't fix; wrong direction. We don't checkin .classpath files
anywhere else in other modules (as argued by [~Apache9]). Seems to be a real
issue here with the new hbase-protocol-shaded module and IDEs on a clean setup.
Will open new issue when figure root problem.
> Eclipse can't find checked-in shaded protobuf files
> ---
>
> Key: HBASE-16806
> URL: https://issues.apache.org/jira/browse/HBASE-16806
> Project: HBase
> Issue Type: Bug
> Components: Protobufs
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-16806.master.001.patch
>
>
> In a new project checkout, Eclipse complains it cant find shaded protobufs
> though they src is checked in (FYI [~syuanjiang]) This issue came in with the
> recent commit:
> {code}
> commit 95c1dc93fb6780f66fd8ebb57914a050b75b9b11
> Author: stack
> Date: Mon Oct 3 21:37:32 2016 -0700
> HBASE-15638 Shade protobuf
> Which includes
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
[
https://issues.apache.org/jira/browse/HBASE-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564569#comment-15564569
]
stack commented on HBASE-16806:
---
Makes sense. Let me just resolve this as 'wont fix' so as not to confuse. Will
open new issue when figure why IDE setup against clean build hiccups on the new
module.
> Eclipse can't find checked-in shaded protobuf files
> ---
>
> Key: HBASE-16806
> URL: https://issues.apache.org/jira/browse/HBASE-16806
> Project: HBase
> Issue Type: Bug
> Components: Protobufs
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-16806.master.001.patch
>
>
> In a new project checkout, Eclipse complains it cant find shaded protobufs
> though they src is checked in (FYI [~syuanjiang]) This issue came in with the
> recent commit:
> {code}
> commit 95c1dc93fb6780f66fd8ebb57914a050b75b9b11
> Author: stack
> Date: Mon Oct 3 21:37:32 2016 -0700
> HBASE-15638 Shade protobuf
> Which includes
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16808) Included the generated, shaded java files from "HBASE-15638 Shade protobuf" in our src assembly
stack created HBASE-16808: - Summary: Included the generated, shaded java files from "HBASE-15638 Shade protobuf" in our src assembly Key: HBASE-16808 URL: https://issues.apache.org/jira/browse/HBASE-16808 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack [~Apache9] found that I forgot to include the generated, shaded files in our src tgz. Let me fix. This is a follow-on to HBASE-16793 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16793) Exclude shaded protobuf files from rat check
[ https://issues.apache.org/jira/browse/HBASE-16793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564566#comment-15564566 ] stack commented on HBASE-16793: --- +1 on patch. HBASE-16808 is to fix the assembly. > Exclude shaded protobuf files from rat check > > > Key: HBASE-16793 > URL: https://issues.apache.org/jira/browse/HBASE-16793 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-16793.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16808) Included the generated, shaded java files from "HBASE-15638 Shade protobuf" in our src assembly
[ https://issues.apache.org/jira/browse/HBASE-16808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16808: -- Affects Version/s: 2.0.0 Priority: Blocker (was: Major) > Included the generated, shaded java files from "HBASE-15638 Shade protobuf" > in our src assembly > --- > > Key: HBASE-16808 > URL: https://issues.apache.org/jira/browse/HBASE-16808 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Blocker > > [~Apache9] found that I forgot to include the generated, shaded files in our > src tgz. Let me fix. This is a follow-on to HBASE-16793 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16505) Pass deadline to HRegion operations
[ https://issues.apache.org/jira/browse/HBASE-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-16505: -- Attachment: HBASE-16505-v13.patch > Pass deadline to HRegion operations > --- > > Key: HBASE-16505 > URL: https://issues.apache.org/jira/browse/HBASE-16505 > Project: HBase > Issue Type: Sub-task >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0 > > Attachments: HBASE-16505-branch-1-v1.patch, > HBASE-16505-branch-1-v2.patch, HBASE-16505-v1.patch, HBASE-16505-v10.patch, > HBASE-16505-v10.patch, HBASE-16505-v11.patch, HBASE-16505-v11.patch, > HBASE-16505-v12.patch, HBASE-16505-v13.patch, HBASE-16505-v2.patch, > HBASE-16505-v3.patch, HBASE-16505-v4.patch, HBASE-16505-v5.patch, > HBASE-16505-v6.patch, HBASE-16505-v7.patch, HBASE-16505-v8.patch, > HBASE-16505-v9.patch > > > If we want to know the correct setting of timeout in read/write path, we need > add a new parameter in operation-methods of Region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16505) Pass deadline to HRegion operations
[ https://issues.apache.org/jira/browse/HBASE-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564561#comment-15564561 ] Phil Yang commented on HBASE-16505: --- Sounds reasonable > Pass deadline to HRegion operations > --- > > Key: HBASE-16505 > URL: https://issues.apache.org/jira/browse/HBASE-16505 > Project: HBase > Issue Type: Sub-task >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0 > > Attachments: HBASE-16505-branch-1-v1.patch, > HBASE-16505-branch-1-v2.patch, HBASE-16505-v1.patch, HBASE-16505-v10.patch, > HBASE-16505-v10.patch, HBASE-16505-v11.patch, HBASE-16505-v11.patch, > HBASE-16505-v12.patch, HBASE-16505-v2.patch, HBASE-16505-v3.patch, > HBASE-16505-v4.patch, HBASE-16505-v5.patch, HBASE-16505-v6.patch, > HBASE-16505-v7.patch, HBASE-16505-v8.patch, HBASE-16505-v9.patch > > > If we want to know the correct setting of timeout in read/write path, we need > add a new parameter in operation-methods of Region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
[
https://issues.apache.org/jira/browse/HBASE-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564549#comment-15564549
]
Duo Zhang commented on HBASE-16806:
---
This is content of our gitignore file. There is also some entries for idea. I
think this means we do not want any build related files to be committed to the
git repo?
{noformat}
/.externalToolBuilders
.project
*.settings/
.classpath
/build
/.idea/
/logs
*target/
*.iml
*.orig
*~
hbase-*/test
*.iws
*.iml
*.ipr
patchprocess/
dependency-reduced-pom.xml
{noformat}
> Eclipse can't find checked-in shaded protobuf files
> ---
>
> Key: HBASE-16806
> URL: https://issues.apache.org/jira/browse/HBASE-16806
> Project: HBase
> Issue Type: Bug
> Components: Protobufs
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-16806.master.001.patch
>
>
> In a new project checkout, Eclipse complains it cant find shaded protobufs
> though they src is checked in (FYI [~syuanjiang]) This issue came in with the
> recent commit:
> {code}
> commit 95c1dc93fb6780f66fd8ebb57914a050b75b9b11
> Author: stack
> Date: Mon Oct 3 21:37:32 2016 -0700
> HBASE-15638 Shade protobuf
> Which includes
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
[
https://issues.apache.org/jira/browse/HBASE-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564535#comment-15564535
]
stack commented on HBASE-16806:
---
Ok. Let me let this patch hang out here another while in case others complain.
Looking more -- prompted by the [~Apache9] comment below -- this patch is
probably wrong since we don't check in .classpath in other modules ...
seemingly.
I need then to get to the root of why for me on fresh checkout, the
hbase-protocol-shaded can be wonky. I'll be back.
> Eclipse can't find checked-in shaded protobuf files
> ---
>
> Key: HBASE-16806
> URL: https://issues.apache.org/jira/browse/HBASE-16806
> Project: HBase
> Issue Type: Bug
> Components: Protobufs
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-16806.master.001.patch
>
>
> In a new project checkout, Eclipse complains it cant find shaded protobufs
> though they src is checked in (FYI [~syuanjiang]) This issue came in with the
> recent commit:
> {code}
> commit 95c1dc93fb6780f66fd8ebb57914a050b75b9b11
> Author: stack
> Date: Mon Oct 3 21:37:32 2016 -0700
> HBASE-15638 Shade protobuf
> Which includes
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15561) See how G1GC works with MSLAB and chunk pool
[ https://issues.apache.org/jira/browse/HBASE-15561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564510#comment-15564510 ] stack commented on HBASE-15561: --- Were we doing any i/o? Or all was cached? Client was on same machine as the RS. How many cores? Nice writeup. Lets flip the default for chunk pool to be on -- it has to be for the offheap and these basic tests show it generally better. There'll be more perf testing. If a mistake, we can undo then ( FYI [~vrodionov]) > See how G1GC works with MSLAB and chunk pool > > > Key: HBASE-15561 > URL: https://issues.apache.org/jira/browse/HBASE-15561 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > > Based on the discussion in https://issues.apache.org/jira/browse/HBASE-14613, > we need to identify specifically how MSLAB and G1GC work. This sub-task is > mainly to identify how things work with G1GC and note down the observations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15561) See how G1GC works with MSLAB and chunk pool
[ https://issues.apache.org/jira/browse/HBASE-15561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564497#comment-15564497 ] stack commented on HBASE-15561: --- Was the server under load when only one client pushing load? > See how G1GC works with MSLAB and chunk pool > > > Key: HBASE-15561 > URL: https://issues.apache.org/jira/browse/HBASE-15561 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > > Based on the discussion in https://issues.apache.org/jira/browse/HBASE-14613, > we need to identify specifically how MSLAB and G1GC work. This sub-task is > mainly to identify how things work with G1GC and note down the observations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
[
https://issues.apache.org/jira/browse/HBASE-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564440#comment-15564440
]
Hadoop QA commented on HBASE-16806:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 40s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 19s
{color} | {color:red} The patch generated 140 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 3s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12832597/HBASE-16806.master.001.patch
|
| JIRA Issue | HBASE-16806 |
| Optional Tests | asflicense |
| uname | Linux 2b5c6dba6cf3 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 83fc59d |
| asflicense |
https://builds.apache.org/job/PreCommit-HBASE-Build/3930/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: hbase-protocol-shaded hbase-endpoint U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3930/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Eclipse can't find checked-in shaded protobuf files
> ---
>
> Key: HBASE-16806
> URL: https://issues.apache.org/jira/browse/HBASE-16806
> Project: HBase
> Issue Type: Bug
> Components: Protobufs
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-16806.master.001.patch
>
>
> In a new project checkout, Eclipse complains it cant find shaded protobufs
> though they src is checked in (FYI [~syuanjiang]) This issue came in with the
> recent commit:
> {code}
> commit 95c1dc93fb6780f66fd8ebb57914a050b75b9b11
> Author: stack
> Date: Mon Oct 3 21:37:32 2016 -0700
> HBASE-15638 Shade protobuf
> Which includes
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16788) Race in compacted file deletion between HStore close() and closeAndArchiveCompactedFiles()
[
https://issues.apache.org/jira/browse/HBASE-16788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564430#comment-15564430
]
Hudson commented on HBASE-16788:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1764 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1764/])
HBASE-16788 Guard HFile archiving under a separate lock (garyh: rev
7493e79f15e0a1217dc50ca4431d6ded07df479f)
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionArchiveConcurrentClose.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> Race in compacted file deletion between HStore close() and
> closeAndArchiveCompactedFiles()
> --
>
> Key: HBASE-16788
> URL: https://issues.apache.org/jira/browse/HBASE-16788
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.3.0
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: 16788-suggest.v2, HBASE-16788.001.patch,
> HBASE-16788.002.patch, HBASE-16788_1.patch
>
>
> HBASE-13082 changed the way that compacted files are archived from being done
> inline on compaction completion to an async cleanup by the
> CompactedHFilesDischarger chore. It looks like the changes to HStore to
> support this introduced a race condition in the compacted HFile archiving.
> In the following sequence, we can wind up with two separate threads trying to
> archive the same HFiles, causing a regionserver abort:
> # compaction completes normally and the compacted files are added to
> {{compactedfiles}} in HStore's DefaultStoreFileManager
> # *threadA*: CompactedHFilesDischargeHandler runs in a RS executor service,
> calling closeAndArchiveCompactedFiles()
> ## obtains HStore readlock
> ## gets a copy of compactedfiles
> ## releases readlock
> # *threadB*: calls HStore.close() as part of region close
> ## obtains HStore writelock
> ## calls DefaultStoreFileManager.clearCompactedfiles(), getting a copy of
> same compactedfiles
> # *threadA*: calls HStore.removeCompactedfiles(compactedfiles)
> ## archives files in {compactedfiles} in HRegionFileSystem.removeStoreFiles()
> ## call HStore.clearCompactedFiles()
> ## waits on write lock
> # *threadB*: continues with close()
> ## calls removeCompactedfiles(compactedfiles)
> ## calls HRegionFIleSystem.removeStoreFiles() ->
> HFileArchiver.archiveStoreFiles()
> ## receives FileNotFoundException because the files have already been
> archived by threadA
> ## throws IOException
> # RS aborts
> I think the combination of fetching the compactedfiles list and removing the
> files needs to be covered by locking. Options I see are:
> * Modify HStore.closeAndArchiveCompactedFiles(): use writelock instead of
> readlock and move the call to removeCompactedfiles() inside the lock. This
> means the read operations will be blocked while the files are being archived,
> which is bad.
> * Synchronize closeAndArchiveCompactedFiles() and modify close() to call it
> instead of calling removeCompactedfiles() directly
> * Add a separate lock for compacted files removal and use in
> closeAndArchiveCompactedFiles() and close()
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16661) Add last major compaction age to per-region metrics
[ https://issues.apache.org/jira/browse/HBASE-16661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564432#comment-15564432 ] Hudson commented on HBASE-16661: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1764 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1764/]) HBASE-16661 Add last major compaction age to per-region metrics (garyh: rev fcef2c02c99a0fd62135aa86e95a03edf1fef3ce) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperImpl.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSourceImpl.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapper.java * (edit) hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionSourceImpl.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSource.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperStub.java > Add last major compaction age to per-region metrics > --- > > Key: HBASE-16661 > URL: https://issues.apache.org/jira/browse/HBASE-16661 > Project: HBase > Issue Type: Improvement >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16661.001.patch, HBASE-16661.002.patch, > HBASE-16661.003.patch > > > After HBASE-12859, we can now track the last major compaction timestamp for > each region. However, this is only exposed through cluster status reporting > and the admin API. > We have similar per-region metrics around storefile age, but none that > filters on major compaction specifically. > Let's add a metric for last major compaction age. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16699) Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval()
[ https://issues.apache.org/jira/browse/HBASE-16699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564431#comment-15564431 ] Hudson commented on HBASE-16699: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1764 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1764/]) HBASE-16699 Overflows in AverageIntervalRateLimiter's refill() and (matteo.bertozzi: rev 341f049e7734be0677ce38018dcb125b991469e7) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/NoopQuotaLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/NoopOperationQuota.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/DefaultOperationQuota.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/TimeBasedLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/OperationQuota.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java > Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval() > > > Key: HBASE-16699 > URL: https://issues.apache.org/jira/browse/HBASE-16699 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.4.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.5, 1.1.8 > > Attachments: HBASE-16699.branch-1.001.patch, > HBASE-16699.master.001.patch, HBASE-16699.master.002.patch > > > It seems that there are more overflow in other places, and a concurrent issue. > I will post a patch within one or 2 days after I figure out adding new > unittest cases. > Please see the following two lines. Once it overflows, it will cause wrong > behavior. For unconfigured RateLimiters, we should have simpler logic to > byPass the check. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L37 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L51 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
[
https://issues.apache.org/jira/browse/HBASE-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564417#comment-15564417
]
Duo Zhang commented on HBASE-16806:
---
I always need to fix the eclipse project configs manually after running mvn
eclipse:eclipse...
And I do not think it is a good idea to commit the eclipse config files to git
because the config files maybe changed between eclipse versions and developers
may use different version of eclipse?
Thanks.
> Eclipse can't find checked-in shaded protobuf files
> ---
>
> Key: HBASE-16806
> URL: https://issues.apache.org/jira/browse/HBASE-16806
> Project: HBase
> Issue Type: Bug
> Components: Protobufs
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-16806.master.001.patch
>
>
> In a new project checkout, Eclipse complains it cant find shaded protobufs
> though they src is checked in (FYI [~syuanjiang]) This issue came in with the
> recent commit:
> {code}
> commit 95c1dc93fb6780f66fd8ebb57914a050b75b9b11
> Author: stack
> Date: Mon Oct 3 21:37:32 2016 -0700
> HBASE-15638 Shade protobuf
> Which includes
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
[
https://issues.apache.org/jira/browse/HBASE-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564399#comment-15564399
]
Stephen Yuan Jiang commented on HBASE-16806:
[~stack], thanks for looking at this. Actually I made a mistake by just add
the project to eclipse. I should do {{'mvn eclipse:eclipse'}} after I built
and before I import the module in eclipse. Once I follow the right step, my
problem went away.
> Eclipse can't find checked-in shaded protobuf files
> ---
>
> Key: HBASE-16806
> URL: https://issues.apache.org/jira/browse/HBASE-16806
> Project: HBase
> Issue Type: Bug
> Components: Protobufs
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-16806.master.001.patch
>
>
> In a new project checkout, Eclipse complains it cant find shaded protobufs
> though they src is checked in (FYI [~syuanjiang]) This issue came in with the
> recent commit:
> {code}
> commit 95c1dc93fb6780f66fd8ebb57914a050b75b9b11
> Author: stack
> Date: Mon Oct 3 21:37:32 2016 -0700
> HBASE-15638 Shade protobuf
> Which includes
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
[
https://issues.apache.org/jira/browse/HBASE-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16806:
--
Description:
In a new project checkout, Eclipse complains it cant find shaded protobufs
though they src is checked in (FYI [~syuanjiang]) This issue came in with the
recent commit:
{code}
commit 95c1dc93fb6780f66fd8ebb57914a050b75b9b11
Author: stack
Date: Mon Oct 3 21:37:32 2016 -0700
HBASE-15638 Shade protobuf
Which includes
...
{code}
was:In a new project checkout, Eclipse complains it cant find shaded
protobufs though they src is checked in (FYI [~syuanjiang])
> Eclipse can't find checked-in shaded protobuf files
> ---
>
> Key: HBASE-16806
> URL: https://issues.apache.org/jira/browse/HBASE-16806
> Project: HBase
> Issue Type: Bug
> Components: Protobufs
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-16806.master.001.patch
>
>
> In a new project checkout, Eclipse complains it cant find shaded protobufs
> though they src is checked in (FYI [~syuanjiang]) This issue came in with the
> recent commit:
> {code}
> commit 95c1dc93fb6780f66fd8ebb57914a050b75b9b11
> Author: stack
> Date: Mon Oct 3 21:37:32 2016 -0700
> HBASE-15638 Shade protobuf
> Which includes
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
[ https://issues.apache.org/jira/browse/HBASE-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564383#comment-15564383 ] stack commented on HBASE-16806: --- Does this fix the issue you were seeing [~syuanjiang]? It seems to fix it for me (I saw it on fresh checkout). > Eclipse can't find checked-in shaded protobuf files > --- > > Key: HBASE-16806 > URL: https://issues.apache.org/jira/browse/HBASE-16806 > Project: HBase > Issue Type: Bug > Components: Protobufs >Reporter: stack >Assignee: stack > Attachments: HBASE-16806.master.001.patch > > > In a new project checkout, Eclipse complains it cant find shaded protobufs > though they src is checked in (FYI [~syuanjiang]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
[ https://issues.apache.org/jira/browse/HBASE-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16806: -- Status: Patch Available (was: Open) > Eclipse can't find checked-in shaded protobuf files > --- > > Key: HBASE-16806 > URL: https://issues.apache.org/jira/browse/HBASE-16806 > Project: HBase > Issue Type: Bug > Components: Protobufs >Reporter: stack >Assignee: stack > Attachments: HBASE-16806.master.001.patch > > > In a new project checkout, Eclipse complains it cant find shaded protobufs > though they src is checked in (FYI [~syuanjiang]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
[ https://issues.apache.org/jira/browse/HBASE-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16806: -- Attachment: HBASE-16806.master.001.patch > Eclipse can't find checked-in shaded protobuf files > --- > > Key: HBASE-16806 > URL: https://issues.apache.org/jira/browse/HBASE-16806 > Project: HBase > Issue Type: Bug > Components: Protobufs >Reporter: stack >Assignee: stack > Attachments: HBASE-16806.master.001.patch > > > In a new project checkout, Eclipse complains it cant find shaded protobufs > though they src is checked in (FYI [~syuanjiang]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16807) RegionServer will fail to report new active Hmaster until HMaster/RegionServer failover
Pankaj Kumar created HBASE-16807:
Summary: RegionServer will fail to report new active Hmaster until
HMaster/RegionServer failover
Key: HBASE-16807
URL: https://issues.apache.org/jira/browse/HBASE-16807
Project: HBase
Issue Type: Bug
Reporter: Pankaj Kumar
It's little weird, but it happened in the product environment that few
RegionServer missed master znode create notification on master failover. In
that case ZooKeeperNodeTracker will not refresh the cached data and
MasterAddressTracker
will always return old active HM detail to Region server on ServiceException.
Though We create region server stub on failure but without refreshing the
MasterAddressTracker data.
In HRegionServer.createRegionServerStatusStub()
{code}
boolean refresh = false; // for the first time, use cached data
RegionServerStatusService.BlockingInterface intf = null;
boolean interrupted = false;
try {
while (keepLooping()) {
sn = this.masterAddressTracker.getMasterAddress(refresh);
if (sn == null) {
if (!keepLooping()) {
// give up with no connection.
LOG.debug("No master found and cluster is stopped; bailing out");
return null;
}
if (System.currentTimeMillis() > (previousLogTime + 1000)) {
LOG.debug("No master found; retry");
previousLogTime = System.currentTimeMillis();
}
refresh = true; // let's try pull it from ZK directly
if (sleep(200)) {
interrupted = true;
}
continue;
}
{code}
Here we refresh node only when 'sn' is NULL otherwise it will use same cached
data.
So in above case RegionServer will never report active HMaster successfully
until HMaster failover or RegionServer restart.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16806) Eclipse can't find checked-in shaded protobuf files
stack created HBASE-16806: - Summary: Eclipse can't find checked-in shaded protobuf files Key: HBASE-16806 URL: https://issues.apache.org/jira/browse/HBASE-16806 Project: HBase Issue Type: Bug Components: Protobufs Reporter: stack Assignee: stack In a new project checkout, Eclipse complains it cant find shaded protobufs though they src is checked in (FYI [~syuanjiang]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16716) OfflineMetaRepair leave empty directory inside /hbase/WALs which remains forever
[
https://issues.apache.org/jira/browse/HBASE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564357#comment-15564357
]
Pankaj Kumar commented on HBASE-16716:
--
Oh my bad. I just took it from WALFactory, will update this in another patch
once QA execution is over.
> OfflineMetaRepair leave empty directory inside /hbase/WALs which remains
> forever
>
>
> Key: HBASE-16716
> URL: https://issues.apache.org/jira/browse/HBASE-16716
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 2.0.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16716.patch
>
>
> OfflineMetaRepair rebuild Meta table, while creating meta region it creates
> it's own WAL (inside /hbase/WALs/hbck-meta-recovery-) which wll
> be closed and archived after rebuilding Meta.
> {noformat}
> hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair
> >> /hbase/WALs/hbck-meta-recovery-
> {noformat}
> It doesn't clear the empty dir, empty directory should be removed after
> success.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16716) OfflineMetaRepair leave empty directory inside /hbase/WALs which remains forever
[
https://issues.apache.org/jira/browse/HBASE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564347#comment-15564347
]
Ted Yu commented on HBASE-16716:
lgtm
{code}
1391 * @param walFactoryID A unique identifier for WAL factory. used i.e.
by filesystem
{code}
i.e. means 'that is'
Did you intend to use 'e.g.' ?
> OfflineMetaRepair leave empty directory inside /hbase/WALs which remains
> forever
>
>
> Key: HBASE-16716
> URL: https://issues.apache.org/jira/browse/HBASE-16716
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 2.0.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16716.patch
>
>
> OfflineMetaRepair rebuild Meta table, while creating meta region it creates
> it's own WAL (inside /hbase/WALs/hbck-meta-recovery-) which wll
> be closed and archived after rebuilding Meta.
> {noformat}
> hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair
> >> /hbase/WALs/hbck-meta-recovery-
> {noformat}
> It doesn't clear the empty dir, empty directory should be removed after
> success.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16716) OfflineMetaRepair leave empty directory inside /hbase/WALs which remains forever
[
https://issues.apache.org/jira/browse/HBASE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564346#comment-15564346
]
Pankaj Kumar commented on HBASE-16716:
--
Thanks [~tedyu] :)
> OfflineMetaRepair leave empty directory inside /hbase/WALs which remains
> forever
>
>
> Key: HBASE-16716
> URL: https://issues.apache.org/jira/browse/HBASE-16716
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 2.0.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16716.patch
>
>
> OfflineMetaRepair rebuild Meta table, while creating meta region it creates
> it's own WAL (inside /hbase/WALs/hbck-meta-recovery-) which wll
> be closed and archived after rebuilding Meta.
> {noformat}
> hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair
> >> /hbase/WALs/hbck-meta-recovery-
> {noformat}
> It doesn't clear the empty dir, empty directory should be removed after
> success.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16716) OfflineMetaRepair leave empty directory inside /hbase/WALs which remains forever
[
https://issues.apache.org/jira/browse/HBASE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564341#comment-15564341
]
Ted Yu commented on HBASE-16716:
I manually triggered one QA run.
> OfflineMetaRepair leave empty directory inside /hbase/WALs which remains
> forever
>
>
> Key: HBASE-16716
> URL: https://issues.apache.org/jira/browse/HBASE-16716
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 2.0.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16716.patch
>
>
> OfflineMetaRepair rebuild Meta table, while creating meta region it creates
> it's own WAL (inside /hbase/WALs/hbck-meta-recovery-) which wll
> be closed and archived after rebuilding Meta.
> {noformat}
> hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair
> >> /hbase/WALs/hbck-meta-recovery-
> {noformat}
> It doesn't clear the empty dir, empty directory should be removed after
> success.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16622) Fix some issues with the HBase reference guide
[
https://issues.apache.org/jira/browse/HBASE-16622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564328#comment-15564328
]
Hadoop QA commented on HBASE-16622:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 47s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 37s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 47s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 111m 1s {color}
| {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 21s
{color} | {color:red} The patch generated 140 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 151m 33s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.TestMultiVersions |
| |
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService
|
| |
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithCustomVisLabService
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12832581/HBASE-16622-v2.diff |
| JIRA Issue | HBASE-16622 |
| Optional Tests | asflicense javac javadoc unit |
| uname | Linux 8d5e5b4498c2 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7493e79 |
| Default Java | 1.8.0_101 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/3927/artifact/patchprocess/patch-unit-root.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/3927/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3927/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-HBASE-Build/3927/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3927/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Fix some issues with the HBase reference guide
> --
>
> Key: HBASE-16622
> URL: https://issues.apache.org/jira/browse/HBASE-16622
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: alexxiyang
>Assignee:
[jira] [Updated] (HBASE-16805) HMaster may send reportForDuty himself while shutting down
[
https://issues.apache.org/jira/browse/HBASE-16805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-16805:
-
Description:
We met an interesting scenario where HMaster had sent reportForDuty to himself
during shutting down.
Initially HMaster had registered himself as active master, but couldn't finish
its initialization as Namespace table was not assigned due to some reason
within the specified time,
{noformat}
2016-07-30 19:36:52,161 | FATAL | hadoopc1h2:21300.activeMasterManager | Failed
to become active master |
org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1610)
java.io.IOException: Timedout 30ms waiting for namespace table to be
assigned
at
org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:102)
at
org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:977)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:171)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1606)
at java.lang.Thread.run(Thread.java:745)
2016-07-30 19:36:52,162 | FATAL | hadoopc1h2:21300.activeMasterManager | Master
server abort: loaded coprocessors are:
[org.apache.hadoop.hbase.security.access.AccessController,
org.apache.hadoop.hbase.index.coprocessor.master.IndexMasterObserver,
org.apache.hadoop.hbase.JMXListener] |
org.apache.hadoop.hbase.master.HMaster.abort(HMaster.java:1981)
2016-07-30 19:36:52,162 | FATAL | hadoopc1h2:21300.activeMasterManager |
Unhandled exception. Starting shutdown. |
org.apache.hadoop.hbase.master.HMaster.abort(HMaster.java:1984)
java.io.IOException: Timedout 30ms waiting for namespace table to be
assigned
at
org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:102)
at
org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:977)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:171)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1606)
at java.lang.Thread.run(Thread.java:745)
2016-07-30 19:36:52,187 | INFO | master/hadoopc1h2/machine-ip:21300 |
reportForDuty to master=hadoopc1h2,21300,1469877905979 with port=21300,
startcode=1469877905979 |
org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2271)
2016-07-30 19:36:52,198 | INFO | hadoopc1h2:21300.activeMasterManager |
ConnectorServer stopped! |
org.apache.hadoop.hbase.JMXListener.stopConnectorServer(JMXListener.java:160)
{noformat}
Above in the second last line, HMaster sent reportForDuty to himself.
Background:
1) During master startup HMasterCommandLine constructs the HMaster which starts
another thread which is waiting to become active,
{code}
startActiveMasterManager(infoPort);
{code}
2) Same time after constructing HMaster, HMasterCommandLine started the HMaster
thread,
{code}
HMaster master = HMaster.constructMaster(masterClass, conf, csm);
if (master.isStopped()) {
LOG.info("Won't bring the Master up as a shutdown is requested");
return 1;
}
master.start();
master.join();
{code}
which will be waiting at below code flow,
{noformat}
HRegionServer
run()
preRegistrationInitialization()
initializeZooKeeper()
waitForMasterActive()
{noformat}
3) In HMaster,
{code}
protected void waitForMasterActive(){
boolean tablesOnMaster = BaseLoadBalancer.tablesOnMaster(conf);
while (!(tablesOnMaster && isActiveMaster)
&& !isStopped() && !isAborted()) {
sleeper.sleep();
}
}
{code}
HMaster will wait here until it is stopped/aborted as
"hbase.balancer.tablesOnMaster" is not configured.
When HMaster failed to complete its initialization (as Namespace table was not
assigned) then it will be abort,
{noformat}
abort("Unhandled exception. Starting shutdown.", t);
{noformat}
So step-2 thread will not wait anymore on HMaster abort and while processing
further it will send send report to active master.
{code}
// Try and register with the Master; tell it we are here. Break if
// server is stopped or the clusterup flag is down or hdfs went wacky.
while (keepLooping()) {
RegionServerStartupResponse w = reportForDuty();
if (w == null) {
LOG.warn("reportForDuty failed; sleeping and then retrying.");
this.sleeper.sleep();
} else {
handleReportForDutyResponse(w);
break;
}
}
{code}
was:
We met an interesting scenario where HMaster had sent reportForDuty to himself
during shutting down.
[jira] [Created] (HBASE-16805) HMaster may send reportForDuty himself while shutting down
Pankaj Kumar created HBASE-16805:
Summary: HMaster may send reportForDuty himself while shutting down
Key: HBASE-16805
URL: https://issues.apache.org/jira/browse/HBASE-16805
Project: HBase
Issue Type: Bug
Components: master
Reporter: Pankaj Kumar
Assignee: Pankaj Kumar
Priority: Minor
We met an interesting scenario where HMaster had sent reportForDuty to himself
during shutting down.
Initially HMaster had registered himself as active master, but couldn't finish
its initialization as Namespace table was not assigned due to some reason
within the specified time,
{noformat}
2016-07-30 19:36:52,161 | FATAL | hadoopc1h2:21300.activeMasterManager | Failed
to become active master |
org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1610)
java.io.IOException: Timedout 30ms waiting for namespace table to be
assigned
at
org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:102)
at
org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:977)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:171)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1606)
at java.lang.Thread.run(Thread.java:745)
2016-07-30 19:36:52,162 | FATAL | hadoopc1h2:21300.activeMasterManager | Master
server abort: loaded coprocessors are:
[org.apache.hadoop.hbase.security.access.AccessController,
org.apache.hadoop.hbase.index.coprocessor.master.IndexMasterObserver,
org.apache.hadoop.hbase.JMXListener] |
org.apache.hadoop.hbase.master.HMaster.abort(HMaster.java:1981)
2016-07-30 19:36:52,162 | FATAL | hadoopc1h2:21300.activeMasterManager |
Unhandled exception. Starting shutdown. |
org.apache.hadoop.hbase.master.HMaster.abort(HMaster.java:1984)
java.io.IOException: Timedout 30ms waiting for namespace table to be
assigned
at
org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:102)
at
org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:977)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:171)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1606)
at java.lang.Thread.run(Thread.java:745)
2016-07-30 19:36:52,187 | INFO | master/hadoopc1h2/172.16.19.51:21300 |
reportForDuty to master=hadoopc1h2,21300,1469877905979 with port=21300,
startcode=1469877905979 |
org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2271)
2016-07-30 19:36:52,198 | INFO | hadoopc1h2:21300.activeMasterManager |
ConnectorServer stopped! |
org.apache.hadoop.hbase.JMXListener.stopConnectorServer(JMXListener.java:160)
{noformat}
Above in the second last line, HMaster sent reportForDuty to himself.
Background:
1) During master startup HMasterCommandLine constructs the HMaster which starts
another thread which is waiting to become active,
{code}
startActiveMasterManager(infoPort);
{code}
2) Same time after constructing HMaster, HMasterCommandLine started the HMaster
thread,
{code}
HMaster master = HMaster.constructMaster(masterClass, conf, csm);
if (master.isStopped()) {
LOG.info("Won't bring the Master up as a shutdown is requested");
return 1;
}
master.start();
master.join();
{code}
which will be waiting at below code flow,
{noformat}
HRegionServer
run()
preRegistrationInitialization()
initializeZooKeeper()
waitForMasterActive()
{noformat}
3) In HMaster,
{code}
protected void waitForMasterActive(){
boolean tablesOnMaster = BaseLoadBalancer.tablesOnMaster(conf);
while (!(tablesOnMaster && isActiveMaster)
&& !isStopped() && !isAborted()) {
sleeper.sleep();
}
}
{code}
HMaster will wait here until it is stopped/aborted as
"hbase.balancer.tablesOnMaster" is not configured.
When HMaster failed to complete its initialization (as Namespace table was not
assigned) then it will be abort,
{noformat}
abort("Unhandled exception. Starting shutdown.", t);
{noformat}
So step-2 thread will not wait anymore on HMaster abort and while processing
further it will send send report to active master.
{code}
// Try and register with the Master; tell it we are here. Break if
// server is stopped or the clusterup flag is down or hdfs went wacky.
while (keepLooping()) {
RegionServerStartupResponse w = reportForDuty();
if (w == null) {
LOG.warn("reportForDuty failed; sleeping and then retrying.");
[jira] [Commented] (HBASE-14417) Incremental backup and bulk loading
[ https://issues.apache.org/jira/browse/HBASE-14417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564243#comment-15564243 ] Ted Yu commented on HBASE-14417: Constructive suggestion is welcome. > Incremental backup and bulk loading > --- > > Key: HBASE-14417 > URL: https://issues.apache.org/jira/browse/HBASE-14417 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Ted Yu >Priority: Critical > Labels: backup > Fix For: 2.0.0 > > Attachments: 14417.v1.txt, 14417.v2.txt, 14417.v6.txt > > > Currently, incremental backup is based on WAL files. Bulk data loading > bypasses WALs for obvious reasons, breaking incremental backups. The only way > to continue backups after bulk loading is to create new full backup of a > table. This may not be feasible for customers who do bulk loading regularly > (say, every day). > Google doc for design: > https://docs.google.com/document/d/1ACCLsecHDvzVSasORgqqRNrloGx4mNYIbvAU7lq5lJE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16800) Support for smart querying of our published jmx per server over http+json
[
https://issues.apache.org/jira/browse/HBASE-16800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564241#comment-15564241
]
stack commented on HBASE-16800:
---
>From a server @appy put up w/ replication running:
{code}
{
"beans" : [ {
"name" : "Hadoop:service=HBase,name=RegionServer,sub=Replication",
"modelerType" : "RegionServer,sub=Replication",
"tag.Context" : "regionserver",
"tag.Hostname" : "ve0206.halxg.cloudera.com",
"source.1.shippedOps" : 3,
"source.shippedKBs" : 0,
"source.1.shippedBatches" : 3,
"source.sizeOfLogQueue" : 3,
"source.logEditsFiltered" : 2,
"source.1.shippedKBs" : 0,
"source.1.ageOfLastShippedOp" : 1657226,
"source.1.logReadInBytes" : 969,
"source.logReadInBytes" : 969,
"source.1.logEditsRead" : 6,
"sink.appliedOps" : 0,
"source.1.sizeOfLogQueue" : 3,
"source.shippedOps" : 3,
"source.logEditsRead" : 6,
"source.shippedBatches" : 3,
"sink.ageOfLastAppliedOp" : 0,
"source.ageOfLastShippedOp" : 1657226,
"source.1.logEditsFiltered" : 2,
"sink.appliedBatches" : 0
} ]
}
{code}
Seems like we want the whole bean to tell the replication story so in this case
the qry by mbean as @appy says is good enough.
If we need more fine-grained querying, searching, listing, compression, batch,
etc., then we can come back here to this patch that adds jolokai. Meantime I've
made the issue minor.
Also, we were going to bring in jolokai -- another dependency -- it'd be good
to replace our current jmx servlet with it instead.
> Support for smart querying of our published jmx per server over http+json
> -
>
> Key: HBASE-16800
> URL: https://issues.apache.org/jira/browse/HBASE-16800
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: stack
>Assignee: stack
>Priority: Minor
> Attachments: HBASE-16800.master.001.patch,
> HBASE-16800.master.002.patch
>
>
> We currently expose our jmx metrics via a json dump in a servlet at /jmx. You
> get all or nothing (You can add a '?description=true' to get a description
> for each metric published but that is it).
> A means of being able to query by metric or by a set of metrics would make
> graphing easier to do, especially over http dumping json. In particular, our
> [~appy] is trying to put up pages that give insight into current state of
> replication.
> Related, HBASE-11747 ClusterStatus is too bulky, is about how whenever we
> need a metric 'exposed', our only means is by bulking out the heartbeat
> adding payload on each message we send the server (In the issue, w/ 1M
> regions, each RS is sending 100MB of 'status' every second). We need to undo
> metrics collection from clusterstatus function
> Bonus if able to compress/cache payloads, security, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16804) JavaHBaseContext.streamBulkGet is void but should be JavaDStream
[
https://issues.apache.org/jira/browse/HBASE-16804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16804:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
Status: Resolved (was: Patch Available)
Thanks for the patch, Igor.
> JavaHBaseContext.streamBulkGet is void but should be JavaDStream
> -
>
> Key: HBASE-16804
> URL: https://issues.apache.org/jira/browse/HBASE-16804
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.0.0
>Reporter: Igor Yurinok
>Assignee: Igor Yurinok
> Labels: beginner, spark
> Fix For: 2.0.0
>
> Attachments: HBASE-16804.patch
>
>
> This is current implementation in JavaHBaseContext.scala:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U])
> {code}
> Should be:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U]): JavaDStream[U]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-16804) JavaHBaseContext.streamBulkGet is void but should be JavaDStream
[
https://issues.apache.org/jira/browse/HBASE-16804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reassigned HBASE-16804:
--
Assignee: Igor Yurinok
> JavaHBaseContext.streamBulkGet is void but should be JavaDStream
> -
>
> Key: HBASE-16804
> URL: https://issues.apache.org/jira/browse/HBASE-16804
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.0.0
>Reporter: Igor Yurinok
>Assignee: Igor Yurinok
> Labels: beginner, spark
> Fix For: 2.0.0
>
> Attachments: HBASE-16804.patch
>
>
> This is current implementation in JavaHBaseContext.scala:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U])
> {code}
> Should be:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U]): JavaDStream[U]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16800) Support for smart querying of our published jmx per server over http+json
[ https://issues.apache.org/jira/browse/HBASE-16800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16800: -- Priority: Minor (was: Major) > Support for smart querying of our published jmx per server over http+json > - > > Key: HBASE-16800 > URL: https://issues.apache.org/jira/browse/HBASE-16800 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: stack >Assignee: stack >Priority: Minor > Attachments: HBASE-16800.master.001.patch, > HBASE-16800.master.002.patch > > > We currently expose our jmx metrics via a json dump in a servlet at /jmx. You > get all or nothing (You can add a '?description=true' to get a description > for each metric published but that is it). > A means of being able to query by metric or by a set of metrics would make > graphing easier to do, especially over http dumping json. In particular, our > [~appy] is trying to put up pages that give insight into current state of > replication. > Related, HBASE-11747 ClusterStatus is too bulky, is about how whenever we > need a metric 'exposed', our only means is by bulking out the heartbeat > adding payload on each message we send the server (In the issue, w/ 1M > regions, each RS is sending 100MB of 'status' every second). We need to undo > metrics collection from clusterstatus function > Bonus if able to compress/cache payloads, security, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16804) JavaHBaseContext.streamBulkGet is void but should be JavaDStream
[
https://issues.apache.org/jira/browse/HBASE-16804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16804:
---
Summary: JavaHBaseContext.streamBulkGet is void but should be JavaDStream
(was: JavaHBaseContext.streamBulkGet is void but must be JavaDStream )
> JavaHBaseContext.streamBulkGet is void but should be JavaDStream
> -
>
> Key: HBASE-16804
> URL: https://issues.apache.org/jira/browse/HBASE-16804
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.0.0
>Reporter: Igor Yurinok
> Labels: beginner, spark
> Attachments: HBASE-16804.patch
>
>
> This is current implementation in JavaHBaseContext.scala:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U])
> {code}
> Should be:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U]): JavaDStream[U]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16804) JavaHBaseContext.streamBulkGet is void but must be JavaDStream
[
https://issues.apache.org/jira/browse/HBASE-16804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564200#comment-15564200
]
Hadoop QA commented on HBASE-16804:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 16s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 48s
{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 25s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12832589/HBASE-16804.patch |
| JIRA Issue | HBASE-16804 |
| Optional Tests | asflicense scalac scaladoc unit compile |
| uname | Linux 25c35e2e071a 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0f21c41 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3928/testReport/ |
| modules | C: hbase-spark U: hbase-spark |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3928/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> JavaHBaseContext.streamBulkGet is void but must be JavaDStream
> ---
>
> Key: HBASE-16804
> URL: https://issues.apache.org/jira/browse/HBASE-16804
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.0.0
>Reporter: Igor Yurinok
> Labels: beginner, spark
> Attachments: HBASE-16804.patch
>
>
> This is current implementation in JavaHBaseContext.scala:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U])
> {code}
> Should be:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U]): JavaDStream[U]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16373) precommit needs a dockerfile with hbase prereqs
[ https://issues.apache.org/jira/browse/HBASE-16373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564150#comment-15564150 ] Duo Zhang commented on HBASE-16373: --- It should be /usr/lib/jvm/java-6-oracle. > precommit needs a dockerfile with hbase prereqs > --- > > Key: HBASE-16373 > URL: https://issues.apache.org/jira/browse/HBASE-16373 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 1.3.0, 1.4.0, 1.1.6, 1.2.3, 0.98.22 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Critical > Fix For: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16373-0.98.patch, HBASE-16373-branch-1.patch > > > specifically, we need protoc. starting with the dockerfile used by default in > yetus and adding it will probably suffice. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16788) Race in compacted file deletion between HStore close() and closeAndArchiveCompactedFiles()
[
https://issues.apache.org/jira/browse/HBASE-16788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564128#comment-15564128
]
Hudson commented on HBASE-16788:
FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #35 (See
[https://builds.apache.org/job/HBase-1.3-JDK7/35/])
HBASE-16788 Guard HFile archiving under a separate lock (garyh: rev
8eea3a5777a25907dcf6486bfeafd8482a072b80)
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionArchiveConcurrentClose.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> Race in compacted file deletion between HStore close() and
> closeAndArchiveCompactedFiles()
> --
>
> Key: HBASE-16788
> URL: https://issues.apache.org/jira/browse/HBASE-16788
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.3.0
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: 16788-suggest.v2, HBASE-16788.001.patch,
> HBASE-16788.002.patch, HBASE-16788_1.patch
>
>
> HBASE-13082 changed the way that compacted files are archived from being done
> inline on compaction completion to an async cleanup by the
> CompactedHFilesDischarger chore. It looks like the changes to HStore to
> support this introduced a race condition in the compacted HFile archiving.
> In the following sequence, we can wind up with two separate threads trying to
> archive the same HFiles, causing a regionserver abort:
> # compaction completes normally and the compacted files are added to
> {{compactedfiles}} in HStore's DefaultStoreFileManager
> # *threadA*: CompactedHFilesDischargeHandler runs in a RS executor service,
> calling closeAndArchiveCompactedFiles()
> ## obtains HStore readlock
> ## gets a copy of compactedfiles
> ## releases readlock
> # *threadB*: calls HStore.close() as part of region close
> ## obtains HStore writelock
> ## calls DefaultStoreFileManager.clearCompactedfiles(), getting a copy of
> same compactedfiles
> # *threadA*: calls HStore.removeCompactedfiles(compactedfiles)
> ## archives files in {compactedfiles} in HRegionFileSystem.removeStoreFiles()
> ## call HStore.clearCompactedFiles()
> ## waits on write lock
> # *threadB*: continues with close()
> ## calls removeCompactedfiles(compactedfiles)
> ## calls HRegionFIleSystem.removeStoreFiles() ->
> HFileArchiver.archiveStoreFiles()
> ## receives FileNotFoundException because the files have already been
> archived by threadA
> ## throws IOException
> # RS aborts
> I think the combination of fetching the compactedfiles list and removing the
> files needs to be covered by locking. Options I see are:
> * Modify HStore.closeAndArchiveCompactedFiles(): use writelock instead of
> readlock and move the call to removeCompactedfiles() inside the lock. This
> means the read operations will be blocked while the files are being archived,
> which is bad.
> * Synchronize closeAndArchiveCompactedFiles() and modify close() to call it
> instead of calling removeCompactedfiles() directly
> * Add a separate lock for compacted files removal and use in
> closeAndArchiveCompactedFiles() and close()
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16699) Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval()
[ https://issues.apache.org/jira/browse/HBASE-16699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564129#comment-15564129 ] Hudson commented on HBASE-16699: FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #35 (See [https://builds.apache.org/job/HBase-1.3-JDK7/35/]) HBASE-16699 Overflows in AverageIntervalRateLimiter's refill() and (matteo.bertozzi: rev 387a08657c06e350e3d912e98486c8ae3a1fe025) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/NoopOperationQuota.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/OperationQuota.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/DefaultOperationQuota.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/NoopQuotaLimiter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/TimeBasedLimiter.java > Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval() > > > Key: HBASE-16699 > URL: https://issues.apache.org/jira/browse/HBASE-16699 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.4.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.5, 1.1.8 > > Attachments: HBASE-16699.branch-1.001.patch, > HBASE-16699.master.001.patch, HBASE-16699.master.002.patch > > > It seems that there are more overflow in other places, and a concurrent issue. > I will post a patch within one or 2 days after I figure out adding new > unittest cases. > Please see the following two lines. Once it overflows, it will cause wrong > behavior. For unconfigured RateLimiters, we should have simpler logic to > byPass the check. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L37 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L51 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16661) Add last major compaction age to per-region metrics
[ https://issues.apache.org/jira/browse/HBASE-16661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564130#comment-15564130 ] Hudson commented on HBASE-16661: FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #35 (See [https://builds.apache.org/job/HBase-1.3-JDK7/35/]) HBASE-16661 Add last major compaction age to per-region metrics (garyh: rev 3e32164e44245b4d9e2ffadcb4ae233184f0f9ea) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSource.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperImpl.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperStub.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapper.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSourceImpl.java * (edit) hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionSourceImpl.java > Add last major compaction age to per-region metrics > --- > > Key: HBASE-16661 > URL: https://issues.apache.org/jira/browse/HBASE-16661 > Project: HBase > Issue Type: Improvement >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16661.001.patch, HBASE-16661.002.patch, > HBASE-16661.003.patch > > > After HBASE-12859, we can now track the last major compaction timestamp for > each region. However, this is only exposed through cluster status reporting > and the admin API. > We have similar per-region metrics around storefile age, but none that > filters on major compaction specifically. > Let's add a metric for last major compaction age. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16804) JavaHBaseContext.streamBulkGet is void but must be JavaDStream
[
https://issues.apache.org/jira/browse/HBASE-16804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16804:
---
Status: Patch Available (was: Open)
> JavaHBaseContext.streamBulkGet is void but must be JavaDStream
> ---
>
> Key: HBASE-16804
> URL: https://issues.apache.org/jira/browse/HBASE-16804
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.0.0
>Reporter: Igor Yurinok
> Labels: beginner, spark
> Attachments: HBASE-16804.patch
>
>
> This is current implementation in JavaHBaseContext.scala:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U])
> {code}
> Should be:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U]): JavaDStream[U]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16804) JavaHBaseContext.streamBulkGet is void but must be JavaDStream
[
https://issues.apache.org/jira/browse/HBASE-16804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564123#comment-15564123
]
Igor Yurinok commented on HBASE-16804:
--
Added
> JavaHBaseContext.streamBulkGet is void but must be JavaDStream
> ---
>
> Key: HBASE-16804
> URL: https://issues.apache.org/jira/browse/HBASE-16804
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.0.0
>Reporter: Igor Yurinok
> Labels: beginner, spark
> Attachments: HBASE-16804.patch
>
>
> This is current implementation in JavaHBaseContext.scala:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U])
> {code}
> Should be:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U]): JavaDStream[U]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16804) JavaHBaseContext.streamBulkGet is void but must be JavaDStream
[
https://issues.apache.org/jira/browse/HBASE-16804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Yurinok updated HBASE-16804:
-
Attachment: HBASE-16804.patch
> JavaHBaseContext.streamBulkGet is void but must be JavaDStream
> ---
>
> Key: HBASE-16804
> URL: https://issues.apache.org/jira/browse/HBASE-16804
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.0.0
>Reporter: Igor Yurinok
> Labels: beginner, spark
> Attachments: HBASE-16804.patch
>
>
> This is current implementation in JavaHBaseContext.scala:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U])
> {code}
> Should be:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U]): JavaDStream[U]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16801) The Append/Increment may return the data from future
[ https://issues.apache.org/jira/browse/HBASE-16801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16801: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the patch, Chiaping. > The Append/Increment may return the data from future > > > Key: HBASE-16801 > URL: https://issues.apache.org/jira/browse/HBASE-16801 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ChiaPing Tsai >Assignee: ChiaPing Tsai >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16801.v0.patch > > > OperationContext maintains the mvcc as a static member, so any Append’s and > Increment’s read point will be changed by others. That is, a retrying > Append/Increment may “see” the future data. > This is a master only issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16804) JavaHBaseContext.streamBulkGet is void but must be JavaDStream
[
https://issues.apache.org/jira/browse/HBASE-16804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Yurinok updated HBASE-16804:
-
Labels: beginner spark (was: )
> JavaHBaseContext.streamBulkGet is void but must be JavaDStream
> ---
>
> Key: HBASE-16804
> URL: https://issues.apache.org/jira/browse/HBASE-16804
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.0.0
>Reporter: Igor Yurinok
> Labels: beginner, spark
>
> This is current implementation in JavaHBaseContext.scala:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U])
> {code}
> Should be:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U]): JavaDStream[U]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16801) The Append/Increment may return the data from future
[ https://issues.apache.org/jira/browse/HBASE-16801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16801: --- Summary: The Append/Increment may return the data from future (was: The Append/Increment may return the data came from future) > The Append/Increment may return the data from future > > > Key: HBASE-16801 > URL: https://issues.apache.org/jira/browse/HBASE-16801 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ChiaPing Tsai >Assignee: ChiaPing Tsai >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16801.v0.patch > > > OperationContext maintains the mvcc as a static member, so any Append’s and > Increment’s read point will be changed by others. That is, a retrying > Append/Increment may “see” the future data. > This is a master only issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16804) JavaHBaseContext.streamBulkGet is void but must be JavaDStream
[
https://issues.apache.org/jira/browse/HBASE-16804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564107#comment-15564107
]
Ted Yu commented on HBASE-16804:
Mind providing a patch ?
> JavaHBaseContext.streamBulkGet is void but must be JavaDStream
> ---
>
> Key: HBASE-16804
> URL: https://issues.apache.org/jira/browse/HBASE-16804
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.0.0
>Reporter: Igor Yurinok
>
> This is current implementation in JavaHBaseContext.scala:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U])
> {code}
> Should be:
> {code}
> def streamBulkGet[T, U](tableName: TableName,
> batchSize: Integer,
> javaDStream: JavaDStream[T],
> makeGet: Function[T, Get],
> convertResult: Function[Result, U]): JavaDStream[U]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16804) JavaHBaseContext.streamBulkGet is void but must be JavaDStream
Igor Yurinok created HBASE-16804:
Summary: JavaHBaseContext.streamBulkGet is void but must be
JavaDStream
Key: HBASE-16804
URL: https://issues.apache.org/jira/browse/HBASE-16804
Project: HBase
Issue Type: Bug
Components: spark
Affects Versions: 2.0.0
Reporter: Igor Yurinok
This is current implementation in JavaHBaseContext.scala:
{code}
def streamBulkGet[T, U](tableName: TableName,
batchSize: Integer,
javaDStream: JavaDStream[T],
makeGet: Function[T, Get],
convertResult: Function[Result, U])
{code}
Should be:
{code}
def streamBulkGet[T, U](tableName: TableName,
batchSize: Integer,
javaDStream: JavaDStream[T],
makeGet: Function[T, Get],
convertResult: Function[Result, U]): JavaDStream[U]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16622) Fix some issues with the HBase reference guide
[
https://issues.apache.org/jira/browse/HBASE-16622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
alexxiyang updated HBASE-16622:
---
Attachment: HBASE-16622-v2.diff
> Fix some issues with the HBase reference guide
> --
>
> Key: HBASE-16622
> URL: https://issues.apache.org/jira/browse/HBASE-16622
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: alexxiyang
>Assignee: alexxiyang
> Attachments: HBASE-16622-v0.diff, HBASE-16622-v1.diff,
> HBASE-16622-v2.diff
>
>
> 1.
> {code}
> if (admin.tableExists(tableName)) {
> System.out.println("Table does not exist.");
> System.exit(-1);
> }
> {code}
> This should be
> {code}
> if (!admin.tableExists(tableName)) {
> {code}
> 2.
> SNAPPY is not suitable for begginer. They may get exceptions like
> {code}
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException):
> org.apache.hadoop.hbase.DoNotRetryIOException: Compression algorithm
> 'snappy' previously failed test. Set hbase.table.sanity.checks to false at
> conf or table descriptor if you want to bypass sanity checks
> at
> org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1701)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1569)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1491)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:462)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:55682)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2178)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> So the code below
> {code}
> table.addFamily(new
> HColumnDescriptor(CF_DEFAULT).setCompressionType(Algorithm.SNAPPY));
> {code}
> it better to change into
> {code}
> table.addFamily(new
> HColumnDescriptor(CF_DEFAULT).setCompressionType(Algorithm.NONE));
> {code}
> 3.
> Before modify column family , get the table from connection
> Change
> {code}
> HTableDescriptor table = new HTableDescriptor(tableName);
> {code}
> into
> {code}
> Table table = connection.getTable(TableName.valueOf(tablename));
> {code}
> 4.
> In 143.1.1. Code Formatting
> it just said
> {code}
> Still in Preferences, click . Be sure the following options are
> selected:Apache HBase ™ Reference Guide
> {code}
> But nothing after click. It should be Java->Editor->Save Actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16803) Make hbase:acl table unsplittable
[
https://issues.apache.org/jira/browse/HBASE-16803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564061#comment-15564061
]
Hadoop QA commented on HBASE-16803:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 2s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 8s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 58s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 6s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 36s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 125m 34s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes |
| |
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsReplication |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12832557/16803.v1.txt |
| JIRA Issue | HBASE-16803 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 97d6652ad6bd 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-16767) Mob compaction needs to clean up files in /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload when running into IO exceptions
[ https://issues.apache.org/jira/browse/HBASE-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564015#comment-15564015 ] Hudson commented on HBASE-16767: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #1763 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1763/]) HBASE-16767 Mob compaction needs to clean up temporary files in face of (busbey: rev c930bc92f4f2beac4fe1f9a4f6364648317e1eee) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestPartitionedMobCompactor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java > Mob compaction needs to clean up files in /hbase/mobdir/.tmp and > /hbase/mobdir/.tmp/.bulkload when running into IO exceptions > -- > > Key: HBASE-16767 > URL: https://issues.apache.org/jira/browse/HBASE-16767 > Project: HBase > Issue Type: Bug > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0 > > Attachments: HBASE-16767.master.001.patch, > HBASE-16767.master.002.patch, HBASE-16767.master.003.patch, > HBASE-16767.master.004.patch, HBASE-16767.master.004.patch, > HBASE-16767.master.005.patch > > > For the following lines, when it runs into IOException, it does not clean up > the files under /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload. > It could be due to creating of new mob file or new reference file under > .bulkload directory fails. Or when mob files from ./tmp to the mob region > directory fails. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16699) Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval()
[ https://issues.apache.org/jira/browse/HBASE-16699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564002#comment-15564002 ] Hudson commented on HBASE-16699: FAILURE: Integrated in Jenkins build HBase-1.1-JDK7 #1794 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1794/]) HBASE-16699 Overflows in AverageIntervalRateLimiter's refill() and (matteo.bertozzi: rev 47c8659d2ac6fd5bb715e30e317aa44ee574d8b0) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/NoopQuotaLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/DefaultOperationQuota.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/TimeBasedLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/OperationQuota.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/NoopOperationQuota.java > Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval() > > > Key: HBASE-16699 > URL: https://issues.apache.org/jira/browse/HBASE-16699 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.4.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.5, 1.1.8 > > Attachments: HBASE-16699.branch-1.001.patch, > HBASE-16699.master.001.patch, HBASE-16699.master.002.patch > > > It seems that there are more overflow in other places, and a concurrent issue. > I will post a patch within one or 2 days after I figure out adding new > unittest cases. > Please see the following two lines. Once it overflows, it will cause wrong > behavior. For unconfigured RateLimiters, we should have simpler logic to > byPass the check. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L37 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L51 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564001#comment-15564001
]
Hadoop QA commented on HBASE-14123:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 3s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 7s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 48 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
14s {color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 47s
{color} | {color:red} hbase-protocol-shaded in master has 22 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 4s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 1s
{color} | {color:red} The patch has 544 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 16s
{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 3s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 1s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 57s
{color} | {color:red} hbase-server generated 13 new + 0 unchanged - 0 fixed =
13 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 27s
{color} | {color:red} hbase-server generated 14 new + 1 unchanged - 0 fixed =
15 total (was 1) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 51s
{color} | {color:red} root generated 14 new + 19 unchanged - 0 fixed = 33 total
(was 19) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 40s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} |
[jira] [Commented] (HBASE-16716) OfflineMetaRepair leave empty directory inside /hbase/WALs which remains forever
[
https://issues.apache.org/jira/browse/HBASE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563978#comment-15563978
]
Pankaj Kumar commented on HBASE-16716:
--
[~tedyu] Can you please help here, why QA didn't run?
> OfflineMetaRepair leave empty directory inside /hbase/WALs which remains
> forever
>
>
> Key: HBASE-16716
> URL: https://issues.apache.org/jira/browse/HBASE-16716
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 2.0.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16716.patch
>
>
> OfflineMetaRepair rebuild Meta table, while creating meta region it creates
> it's own WAL (inside /hbase/WALs/hbck-meta-recovery-) which wll
> be closed and archived after rebuilding Meta.
> {noformat}
> hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair
> >> /hbase/WALs/hbck-meta-recovery-
> {noformat}
> It doesn't clear the empty dir, empty directory should be removed after
> success.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15561) See how G1GC works with MSLAB and chunk pool
[ https://issues.apache.org/jira/browse/HBASE-15561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563893#comment-15563893 ] Vladimir Rodionov commented on HBASE-15561: --- Yes. We have a JIRA: HBASE-15513 > See how G1GC works with MSLAB and chunk pool > > > Key: HBASE-15561 > URL: https://issues.apache.org/jira/browse/HBASE-15561 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > > Based on the discussion in https://issues.apache.org/jira/browse/HBASE-14613, > we need to identify specifically how MSLAB and G1GC work. This sub-task is > mainly to identify how things work with G1GC and note down the observations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16699) Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval()
[ https://issues.apache.org/jira/browse/HBASE-16699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563895#comment-15563895 ] Hudson commented on HBASE-16699: FAILURE: Integrated in Jenkins build HBase-1.1-JDK8 #1878 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1878/]) HBASE-16699 Overflows in AverageIntervalRateLimiter's refill() and (matteo.bertozzi: rev 47c8659d2ac6fd5bb715e30e317aa44ee574d8b0) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/NoopQuotaLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaLimiter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/TimeBasedLimiter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/OperationQuota.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/DefaultOperationQuota.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/NoopOperationQuota.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java > Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval() > > > Key: HBASE-16699 > URL: https://issues.apache.org/jira/browse/HBASE-16699 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.4.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.5, 1.1.8 > > Attachments: HBASE-16699.branch-1.001.patch, > HBASE-16699.master.001.patch, HBASE-16699.master.002.patch > > > It seems that there are more overflow in other places, and a concurrent issue. > I will post a patch within one or 2 days after I figure out adding new > unittest cases. > Please see the following two lines. Once it overflows, it will cause wrong > behavior. For unconfigured RateLimiters, we should have simpler logic to > byPass the check. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L37 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L51 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16699) Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval()
[
https://issues.apache.org/jira/browse/HBASE-16699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563878#comment-15563878
]
Hadoop QA commented on HBASE-16699:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m 40s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
18s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} branch-1 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 57s
{color} | {color:red} hbase-server in branch-1 has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} branch-1 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 44s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 95m 49s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 146m 13s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.client.TestFromClientSide |
| | org.apache.hadoop.hbase.util.TestCoprocessorScanPolicy |
| | org.apache.hadoop.hbase.util.TestMiniClusterLoadEncoded |
| | org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:b2c5d84 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12832537/HBASE-16699.branch-1.001.patch
|
| JIRA Issue | HBASE-16699 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux ba4702c1ae36
[jira] [Updated] (HBASE-16788) Race in compacted file deletion between HStore close() and closeAndArchiveCompactedFiles()
[
https://issues.apache.org/jira/browse/HBASE-16788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-16788:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 1.4.0
1.3.0
2.0.0
Status: Resolved (was: Patch Available)
Committed to branch-1.3+. Thanks to all for reviews.
> Race in compacted file deletion between HStore close() and
> closeAndArchiveCompactedFiles()
> --
>
> Key: HBASE-16788
> URL: https://issues.apache.org/jira/browse/HBASE-16788
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.3.0
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: 16788-suggest.v2, HBASE-16788.001.patch,
> HBASE-16788.002.patch, HBASE-16788_1.patch
>
>
> HBASE-13082 changed the way that compacted files are archived from being done
> inline on compaction completion to an async cleanup by the
> CompactedHFilesDischarger chore. It looks like the changes to HStore to
> support this introduced a race condition in the compacted HFile archiving.
> In the following sequence, we can wind up with two separate threads trying to
> archive the same HFiles, causing a regionserver abort:
> # compaction completes normally and the compacted files are added to
> {{compactedfiles}} in HStore's DefaultStoreFileManager
> # *threadA*: CompactedHFilesDischargeHandler runs in a RS executor service,
> calling closeAndArchiveCompactedFiles()
> ## obtains HStore readlock
> ## gets a copy of compactedfiles
> ## releases readlock
> # *threadB*: calls HStore.close() as part of region close
> ## obtains HStore writelock
> ## calls DefaultStoreFileManager.clearCompactedfiles(), getting a copy of
> same compactedfiles
> # *threadA*: calls HStore.removeCompactedfiles(compactedfiles)
> ## archives files in {compactedfiles} in HRegionFileSystem.removeStoreFiles()
> ## call HStore.clearCompactedFiles()
> ## waits on write lock
> # *threadB*: continues with close()
> ## calls removeCompactedfiles(compactedfiles)
> ## calls HRegionFIleSystem.removeStoreFiles() ->
> HFileArchiver.archiveStoreFiles()
> ## receives FileNotFoundException because the files have already been
> archived by threadA
> ## throws IOException
> # RS aborts
> I think the combination of fetching the compactedfiles list and removing the
> files needs to be covered by locking. Options I see are:
> * Modify HStore.closeAndArchiveCompactedFiles(): use writelock instead of
> readlock and move the call to removeCompactedfiles() inside the lock. This
> means the read operations will be blocked while the files are being archived,
> which is bad.
> * Synchronize closeAndArchiveCompactedFiles() and modify close() to call it
> instead of calling removeCompactedfiles() directly
> * Add a separate lock for compacted files removal and use in
> closeAndArchiveCompactedFiles() and close()
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15561) See how G1GC works with MSLAB and chunk pool
[ https://issues.apache.org/jira/browse/HBASE-15561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563851#comment-15563851 ] Enis Soztutar commented on HBASE-15561: --- So as a result of this, we will enable chunk pool by default? From the results, it seems that we should even with G1. > See how G1GC works with MSLAB and chunk pool > > > Key: HBASE-15561 > URL: https://issues.apache.org/jira/browse/HBASE-15561 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > > Based on the discussion in https://issues.apache.org/jira/browse/HBASE-14613, > we need to identify specifically how MSLAB and G1GC work. This sub-task is > mainly to identify how things work with G1GC and note down the observations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16653) Backport HBASE-11393 to all branches which support namespace
[ https://issues.apache.org/jira/browse/HBASE-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563768#comment-15563768 ] Guanghao Zhang commented on HBASE-16653: The v3 patch didn't remove the deprecated methods in ReplicationAdmin. But ReplicationPeerConfig added new methods and ReplicationPeers changed some methods. Did the InterfaceAudience.Private means we can change the methods in these class? Thanks > Backport HBASE-11393 to all branches which support namespace > > > Key: HBASE-16653 > URL: https://issues.apache.org/jira/browse/HBASE-16653 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0, 1.0.5, 1.3.1, 0.98.22, 1.1.7, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 1.4.0 > > Attachments: HBASE-16653-branch-1-v1.patch, > HBASE-16653-branch-1-v2.patch, HBASE-16653-branch-1-v3.patch > > > As HBASE-11386 mentioned, the parse code about replication table-cfs config > will be wrong when table name contains namespace and we can only config the > default namespace's tables in the peer. It is a bug for all branches which > support namespace. HBASE-11393 resolved this by use a pb object but it was > only merged to master branch. Other branches still have this problem. I > thought we should fix this bug in all branches which support namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16661) Add last major compaction age to per-region metrics
[ https://issues.apache.org/jira/browse/HBASE-16661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Helmling updated HBASE-16661: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.4.0 1.3.0 2.0.0 Release Note: This adds a new per-region metric named "lastMajorCompactionAge" for tracking time since the last major compaction ran on a given region. If a major compaction has never run, the age will be equal to the current timestamp. Status: Resolved (was: Patch Available) Pushed to branch-1.3+. Thanks for the patch, Dustin! > Add last major compaction age to per-region metrics > --- > > Key: HBASE-16661 > URL: https://issues.apache.org/jira/browse/HBASE-16661 > Project: HBase > Issue Type: Improvement >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16661.001.patch, HBASE-16661.002.patch, > HBASE-16661.003.patch > > > After HBASE-12859, we can now track the last major compaction timestamp for > each region. However, this is only exposed through cluster status reporting > and the admin API. > We have similar per-region metrics around storefile age, but none that > filters on major compaction specifically. > Let's add a metric for last major compaction age. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16801) The Append/Increment may return the data came from future
[
https://issues.apache.org/jira/browse/HBASE-16801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563716#comment-15563716
]
Hadoop QA commented on HBASE-16801:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 11s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 101m 59s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 143m 17s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestMobRestoreFlushSnapshotFromClient |
| | org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient |
| | org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12832531/HBASE-16801.v0.patch |
| JIRA Issue | HBASE-16801 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux d00a89651533 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3c35a72 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/3921/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/3921/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3921/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Created] (HBASE-16803) Make hbase:acl table unsplittable
Ted Yu created HBASE-16803: -- Summary: Make hbase:acl table unsplittable Key: HBASE-16803 URL: https://issues.apache.org/jira/browse/HBASE-16803 Project: HBase Issue Type: Improvement Reporter: Ted Yu Assignee: Ted Yu Attachments: 16803.v1.txt HBASE-16773 fixed a case where PriorityRpcServer handler threads are all occupied accessing hbase:acl table. However, the fix relies on the fact that there is single region in hbase:acl table so that the access can be local. As discussed at the end of HBASE-16773, we should disable split of hbase:acl table as well. hbase:meta is normally much larger than hbase:acl table and it has only one region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16803) Make hbase:acl table unsplittable
[ https://issues.apache.org/jira/browse/HBASE-16803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16803: --- Attachment: 16803.v1.txt > Make hbase:acl table unsplittable > - > > Key: HBASE-16803 > URL: https://issues.apache.org/jira/browse/HBASE-16803 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 16803.v1.txt > > > HBASE-16773 fixed a case where PriorityRpcServer handler threads are all > occupied accessing hbase:acl table. > However, the fix relies on the fact that there is single region in hbase:acl > table so that the access can be local. > As discussed at the end of HBASE-16773, we should disable split of hbase:acl > table as well. > hbase:meta is normally much larger than hbase:acl table and it has only one > region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16803) Make hbase:acl table unsplittable
[ https://issues.apache.org/jira/browse/HBASE-16803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16803: --- Status: Patch Available (was: Open) > Make hbase:acl table unsplittable > - > > Key: HBASE-16803 > URL: https://issues.apache.org/jira/browse/HBASE-16803 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 16803.v1.txt > > > HBASE-16773 fixed a case where PriorityRpcServer handler threads are all > occupied accessing hbase:acl table. > However, the fix relies on the fact that there is single region in hbase:acl > table so that the access can be local. > As discussed at the end of HBASE-16773, we should disable split of hbase:acl > table as well. > hbase:meta is normally much larger than hbase:acl table and it has only one > region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16773) AccessController should access local region if possible
[
https://issues.apache.org/jira/browse/HBASE-16773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563679#comment-15563679
]
Andrew Purtell commented on HBASE-16773:
> It should be fine to restrict this, given that even meta is single region,
> although meta is much bigger.
As long as meta is not splittable doing this for the sake of expediency
elsewhere is not without precedent but it does compound the original sin.
We should allow meta to split.
We should have an answer for this class of problem.
Getting back to expedient solutions, I have to concede that it's unlikely we
will reach a scalability limit by preventing the ACL table from splitting given
how we cache its complete contents in each RS. That will be a problem first.
> AccessController should access local region if possible
> ---
>
> Key: HBASE-16773
> URL: https://issues.apache.org/jira/browse/HBASE-16773
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16773.branch-1.txt, 16773.v2.txt, 16773.v3.txt,
> 16773.v4.txt, 16773.v5.txt, 16773.v6.txt, 16773.v7.txt
>
>
> We observed the following in the stack trace of region server on a 1.1.2
> cluster:
> {code}
> "PriorityRpcServer.handler=19,queue=1,port=60200" #225 daemon prio=5
> os_prio=0 tid=0x7fb562296000 nid=0x81c0 runnable [0x7fb509a27000]
>java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x0003d4dfd770> (a sun.nio.ch.Util$2)
> - locked <0x0003d4dfd760> (a java.util.Collections$UnmodifiableSet)
> - locked <0x0003d4dfd648> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
> - locked <0x0003d7dae180> (a java.io.BufferedInputStream)
> at java.io.DataInputStream.readInt(DataInputStream.java:387)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:151)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:611)
> - locked <0x0003d5c7edc0> (a
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:156)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:737)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:734)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734)
> - locked <0x0003d5c7edc0> (a
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:887)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:856)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1199)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:32627)
> at org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:854)
> at org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:845)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:862)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:828)
> at
>
[jira] [Commented] (HBASE-16802) Procedure v2 - group procedure cleaning
[
https://issues.apache.org/jira/browse/HBASE-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563654#comment-15563654
]
Hadoop QA commented on HBASE-16802:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 10s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 31s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 58s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
6s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 46s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12832541/HBASE-16802-v0.patch |
| JIRA Issue | HBASE-16802 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux b58f2d962b41 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c930bc9 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3924/testReport/ |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3924/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Procedure v2 - group procedure cleaning
> ---
>
> Key: HBASE-16802
> URL: https://issues.apache.org/jira/browse/HBASE-16802
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Affects Versions: 2.0.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-16802-v0.patch
>
>
> group
[jira] [Commented] (HBASE-16788) Race in compacted file deletion between HStore close() and closeAndArchiveCompactedFiles()
[
https://issues.apache.org/jira/browse/HBASE-16788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563638#comment-15563638
]
Mikhail Antonov commented on HBASE-16788:
-
When I first looked at the patch, my only concern was about if it makes RS
stopping significantly slower. Looks like, based on the notes and comments
above, it does not.
+1, LGTM.
> Race in compacted file deletion between HStore close() and
> closeAndArchiveCompactedFiles()
> --
>
> Key: HBASE-16788
> URL: https://issues.apache.org/jira/browse/HBASE-16788
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.3.0
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Blocker
> Attachments: 16788-suggest.v2, HBASE-16788.001.patch,
> HBASE-16788.002.patch, HBASE-16788_1.patch
>
>
> HBASE-13082 changed the way that compacted files are archived from being done
> inline on compaction completion to an async cleanup by the
> CompactedHFilesDischarger chore. It looks like the changes to HStore to
> support this introduced a race condition in the compacted HFile archiving.
> In the following sequence, we can wind up with two separate threads trying to
> archive the same HFiles, causing a regionserver abort:
> # compaction completes normally and the compacted files are added to
> {{compactedfiles}} in HStore's DefaultStoreFileManager
> # *threadA*: CompactedHFilesDischargeHandler runs in a RS executor service,
> calling closeAndArchiveCompactedFiles()
> ## obtains HStore readlock
> ## gets a copy of compactedfiles
> ## releases readlock
> # *threadB*: calls HStore.close() as part of region close
> ## obtains HStore writelock
> ## calls DefaultStoreFileManager.clearCompactedfiles(), getting a copy of
> same compactedfiles
> # *threadA*: calls HStore.removeCompactedfiles(compactedfiles)
> ## archives files in {compactedfiles} in HRegionFileSystem.removeStoreFiles()
> ## call HStore.clearCompactedFiles()
> ## waits on write lock
> # *threadB*: continues with close()
> ## calls removeCompactedfiles(compactedfiles)
> ## calls HRegionFIleSystem.removeStoreFiles() ->
> HFileArchiver.archiveStoreFiles()
> ## receives FileNotFoundException because the files have already been
> archived by threadA
> ## throws IOException
> # RS aborts
> I think the combination of fetching the compactedfiles list and removing the
> files needs to be covered by locking. Options I see are:
> * Modify HStore.closeAndArchiveCompactedFiles(): use writelock instead of
> readlock and move the call to removeCompactedfiles() inside the lock. This
> means the read operations will be blocked while the files are being archived,
> which is bad.
> * Synchronize closeAndArchiveCompactedFiles() and modify close() to call it
> instead of calling removeCompactedfiles() directly
> * Add a separate lock for compacted files removal and use in
> closeAndArchiveCompactedFiles() and close()
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15437) Response size calculated in RPCServer for warning tooLarge responses does NOT count CellScanner payload
[
https://issues.apache.org/jira/browse/HBASE-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563635#comment-15563635
]
Jerry He commented on HBASE-15437:
--
Looks like requestSize has similar problem too?
> Response size calculated in RPCServer for warning tooLarge responses does NOT
> count CellScanner payload
> ---
>
> Key: HBASE-15437
> URL: https://issues.apache.org/jira/browse/HBASE-15437
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: deepankar
>Assignee: deepankar
> Attachments: HBASE-15437-v1.patch, HBASE-15437.patch
>
>
> After HBASE-13158 where we respond back to RPCs with cells in the payload ,
> the protobuf response will just have the count the cells to read from
> payload, but there are set of features where we log warn in RPCServer
> whenever the response is tooLarge, but this size now is not considering the
> sizes of the cells in the PayloadCellScanner. Code form RPCServer
> {code}
> long responseSize = result.getSerializedSize();
> // log any RPC responses that are slower than the configured warn
> // response time or larger than configured warning size
> boolean tooSlow = (processingTime > warnResponseTime &&
> warnResponseTime > -1);
> boolean tooLarge = (responseSize > warnResponseSize && warnResponseSize
> > -1);
> if (tooSlow || tooLarge) {
> // when tagging, we let TooLarge trump TooSmall to keep output simple
> // note that large responses will often also be slow.
> logResponse(new Object[]{param},
> md.getName(), md.getName() + "(" + param.getClass().getName() +
> ")",
> (tooLarge ? "TooLarge" : "TooSlow"),
> status.getClient(), startTime, processingTime, qTime,
> responseSize);
> }
> {code}
> Should this feature be not supported any more or should we add a method to
> CellScanner or a new interface which returns the serialized size (but this
> might not include the compression codecs which might be used during response
> ?) Any other Idea this could be fixed ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15437) Response size calculated in RPCServer for warning tooLarge responses does NOT count CellScanner payload
[
https://issues.apache.org/jira/browse/HBASE-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563628#comment-15563628
]
Jerry He commented on HBASE-15437:
--
The RPC responseSize should be defined as the raw Results/Cells size, or the
size after encode?
> Response size calculated in RPCServer for warning tooLarge responses does NOT
> count CellScanner payload
> ---
>
> Key: HBASE-15437
> URL: https://issues.apache.org/jira/browse/HBASE-15437
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: deepankar
>Assignee: deepankar
> Attachments: HBASE-15437-v1.patch, HBASE-15437.patch
>
>
> After HBASE-13158 where we respond back to RPCs with cells in the payload ,
> the protobuf response will just have the count the cells to read from
> payload, but there are set of features where we log warn in RPCServer
> whenever the response is tooLarge, but this size now is not considering the
> sizes of the cells in the PayloadCellScanner. Code form RPCServer
> {code}
> long responseSize = result.getSerializedSize();
> // log any RPC responses that are slower than the configured warn
> // response time or larger than configured warning size
> boolean tooSlow = (processingTime > warnResponseTime &&
> warnResponseTime > -1);
> boolean tooLarge = (responseSize > warnResponseSize && warnResponseSize
> > -1);
> if (tooSlow || tooLarge) {
> // when tagging, we let TooLarge trump TooSmall to keep output simple
> // note that large responses will often also be slow.
> logResponse(new Object[]{param},
> md.getName(), md.getName() + "(" + param.getClass().getName() +
> ")",
> (tooLarge ? "TooLarge" : "TooSlow"),
> status.getClient(), startTime, processingTime, qTime,
> responseSize);
> }
> {code}
> Should this feature be not supported any more or should we add a method to
> CellScanner or a new interface which returns the serialized size (but this
> might not include the compression codecs which might be used during response
> ?) Any other Idea this could be fixed ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15437) Response size calculated in RPCServer for warning tooLarge responses does NOT count CellScanner payload
[
https://issues.apache.org/jira/browse/HBASE-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563600#comment-15563600
]
Hadoop QA commented on HBASE-15437:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} HBASE-15437 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12796789/HBASE-15437-v1.patch |
| JIRA Issue | HBASE-15437 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3925/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Response size calculated in RPCServer for warning tooLarge responses does NOT
> count CellScanner payload
> ---
>
> Key: HBASE-15437
> URL: https://issues.apache.org/jira/browse/HBASE-15437
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: deepankar
>Assignee: deepankar
> Attachments: HBASE-15437-v1.patch, HBASE-15437.patch
>
>
> After HBASE-13158 where we respond back to RPCs with cells in the payload ,
> the protobuf response will just have the count the cells to read from
> payload, but there are set of features where we log warn in RPCServer
> whenever the response is tooLarge, but this size now is not considering the
> sizes of the cells in the PayloadCellScanner. Code form RPCServer
> {code}
> long responseSize = result.getSerializedSize();
> // log any RPC responses that are slower than the configured warn
> // response time or larger than configured warning size
> boolean tooSlow = (processingTime > warnResponseTime &&
> warnResponseTime > -1);
> boolean tooLarge = (responseSize > warnResponseSize && warnResponseSize
> > -1);
> if (tooSlow || tooLarge) {
> // when tagging, we let TooLarge trump TooSmall to keep output simple
> // note that large responses will often also be slow.
> logResponse(new Object[]{param},
> md.getName(), md.getName() + "(" + param.getClass().getName() +
> ")",
> (tooLarge ? "TooLarge" : "TooSlow"),
> status.getClient(), startTime, processingTime, qTime,
> responseSize);
> }
> {code}
> Should this feature be not supported any more or should we add a method to
> CellScanner or a new interface which returns the serialized size (but this
> might not include the compression codecs which might be used during response
> ?) Any other Idea this could be fixed ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16773) AccessController should access local region if possible
[
https://issues.apache.org/jira/browse/HBASE-16773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563596#comment-15563596
]
Enis Soztutar commented on HBASE-16773:
---
bq. One approach is to disable splitting of hbase:acl table.
I thought that we already do that. It should be fine to restrict this, given
that even meta is single region, although meta is much bigger.
> AccessController should access local region if possible
> ---
>
> Key: HBASE-16773
> URL: https://issues.apache.org/jira/browse/HBASE-16773
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16773.branch-1.txt, 16773.v2.txt, 16773.v3.txt,
> 16773.v4.txt, 16773.v5.txt, 16773.v6.txt, 16773.v7.txt
>
>
> We observed the following in the stack trace of region server on a 1.1.2
> cluster:
> {code}
> "PriorityRpcServer.handler=19,queue=1,port=60200" #225 daemon prio=5
> os_prio=0 tid=0x7fb562296000 nid=0x81c0 runnable [0x7fb509a27000]
>java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x0003d4dfd770> (a sun.nio.ch.Util$2)
> - locked <0x0003d4dfd760> (a java.util.Collections$UnmodifiableSet)
> - locked <0x0003d4dfd648> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
> - locked <0x0003d7dae180> (a java.io.BufferedInputStream)
> at java.io.DataInputStream.readInt(DataInputStream.java:387)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:151)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:611)
> - locked <0x0003d5c7edc0> (a
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:156)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:737)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:734)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734)
> - locked <0x0003d5c7edc0> (a
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:887)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:856)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1199)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:32627)
> at org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:854)
> at org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:845)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:862)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:828)
> at
> org.apache.hadoop.hbase.security.access.AccessControlLists.getPermissions(AccessControlLists.java:461)
> at
> org.apache.hadoop.hbase.security.access.AccessController.updateACL(AccessController.java:260)
> at
> org.apache.hadoop.hbase.security.access.AccessController.postPut(AccessController.java:1661)
> at
>
[jira] [Commented] (HBASE-16653) Backport HBASE-11393 to all branches which support namespace
[ https://issues.apache.org/jira/browse/HBASE-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563590#comment-15563590 ] Enis Soztutar commented on HBASE-16653: --- bq. And HBASE-11393 has some compatibility issues, if we do backport to branch-1, we should take care of this Do you mind reminding us what were the issues? It has been some time that I've reviewed the patch. > Backport HBASE-11393 to all branches which support namespace > > > Key: HBASE-16653 > URL: https://issues.apache.org/jira/browse/HBASE-16653 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0, 1.0.5, 1.3.1, 0.98.22, 1.1.7, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 1.4.0 > > Attachments: HBASE-16653-branch-1-v1.patch, > HBASE-16653-branch-1-v2.patch, HBASE-16653-branch-1-v3.patch > > > As HBASE-11386 mentioned, the parse code about replication table-cfs config > will be wrong when table name contains namespace and we can only config the > default namespace's tables in the peer. It is a bug for all branches which > support namespace. HBASE-11393 resolved this by use a pb object but it was > only merged to master branch. Other branches still have this problem. I > thought we should fix this bug in all branches which support namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16699) Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval()
[ https://issues.apache.org/jira/browse/HBASE-16699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-16699: Resolution: Fixed Fix Version/s: 1.1.8 1.2.5 1.4.0 1.3.0 2.0.0 Status: Resolved (was: Patch Available) > Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval() > > > Key: HBASE-16699 > URL: https://issues.apache.org/jira/browse/HBASE-16699 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.4.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.5, 1.1.8 > > Attachments: HBASE-16699.branch-1.001.patch, > HBASE-16699.master.001.patch, HBASE-16699.master.002.patch > > > It seems that there are more overflow in other places, and a concurrent issue. > I will post a patch within one or 2 days after I figure out adding new > unittest cases. > Please see the following two lines. Once it overflows, it will cause wrong > behavior. For unconfigured RateLimiters, we should have simpler logic to > byPass the check. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L37 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L51 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16788) Race in compacted file deletion between HStore close() and closeAndArchiveCompactedFiles()
[
https://issues.apache.org/jira/browse/HBASE-16788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563579#comment-15563579
]
Ted Yu commented on HBASE-16788:
Since this is a blocker, I am fine with Gary's patch going in.
In my opinion, not introducing additional lock would facilitate future
development. e.g. a feature as complex as HBASE-13082 comes along, the freedom
would be greater for the respective developer.
Thanks
> Race in compacted file deletion between HStore close() and
> closeAndArchiveCompactedFiles()
> --
>
> Key: HBASE-16788
> URL: https://issues.apache.org/jira/browse/HBASE-16788
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.3.0
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Blocker
> Attachments: 16788-suggest.v2, HBASE-16788.001.patch,
> HBASE-16788.002.patch, HBASE-16788_1.patch
>
>
> HBASE-13082 changed the way that compacted files are archived from being done
> inline on compaction completion to an async cleanup by the
> CompactedHFilesDischarger chore. It looks like the changes to HStore to
> support this introduced a race condition in the compacted HFile archiving.
> In the following sequence, we can wind up with two separate threads trying to
> archive the same HFiles, causing a regionserver abort:
> # compaction completes normally and the compacted files are added to
> {{compactedfiles}} in HStore's DefaultStoreFileManager
> # *threadA*: CompactedHFilesDischargeHandler runs in a RS executor service,
> calling closeAndArchiveCompactedFiles()
> ## obtains HStore readlock
> ## gets a copy of compactedfiles
> ## releases readlock
> # *threadB*: calls HStore.close() as part of region close
> ## obtains HStore writelock
> ## calls DefaultStoreFileManager.clearCompactedfiles(), getting a copy of
> same compactedfiles
> # *threadA*: calls HStore.removeCompactedfiles(compactedfiles)
> ## archives files in {compactedfiles} in HRegionFileSystem.removeStoreFiles()
> ## call HStore.clearCompactedFiles()
> ## waits on write lock
> # *threadB*: continues with close()
> ## calls removeCompactedfiles(compactedfiles)
> ## calls HRegionFIleSystem.removeStoreFiles() ->
> HFileArchiver.archiveStoreFiles()
> ## receives FileNotFoundException because the files have already been
> archived by threadA
> ## throws IOException
> # RS aborts
> I think the combination of fetching the compactedfiles list and removing the
> files needs to be covered by locking. Options I see are:
> * Modify HStore.closeAndArchiveCompactedFiles(): use writelock instead of
> readlock and move the call to removeCompactedfiles() inside the lock. This
> means the read operations will be blocked while the files are being archived,
> which is bad.
> * Synchronize closeAndArchiveCompactedFiles() and modify close() to call it
> instead of calling removeCompactedfiles() directly
> * Add a separate lock for compacted files removal and use in
> closeAndArchiveCompactedFiles() and close()
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16796) Correct RpcServer responseSize and requestSize to account for cellScanner payload
[ https://issues.apache.org/jira/browse/HBASE-16796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563574#comment-15563574 ] Jerry He commented on HBASE-16796: -- Thanks [~anoop.hbase]. Found this one HBASE-15437, which seems to be a recent attempt to fix the issue. Let me link them. We can discuss on HBASE-15437. > Correct RpcServer responseSize and requestSize to account for cellScanner > payload > - > > Key: HBASE-16796 > URL: https://issues.apache.org/jira/browse/HBASE-16796 > Project: HBase > Issue Type: Bug >Reporter: Jerry He >Assignee: Jerry He > > The reponseSize and requestSize on the RpcServer don't seem to account for > the cellScanner payload. We should correct them. > The metrics numbers and the logging for reponseTooLarge and TooSlow won't > show up correctly otherwise. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16505) Pass deadline to HRegion operations
[
https://issues.apache.org/jira/browse/HBASE-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563571#comment-15563571
]
Ted Yu commented on HBASE-16505:
{code}
126 } catch (TimeoutIOException e){
127 RpcServer.LOG.info("Can not complete this request in time, drop
it: " + call);
{code}
Should the log be at WARN level ?
> Pass deadline to HRegion operations
> ---
>
> Key: HBASE-16505
> URL: https://issues.apache.org/jira/browse/HBASE-16505
> Project: HBase
> Issue Type: Sub-task
>Reporter: Phil Yang
>Assignee: Phil Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-16505-branch-1-v1.patch,
> HBASE-16505-branch-1-v2.patch, HBASE-16505-v1.patch, HBASE-16505-v10.patch,
> HBASE-16505-v10.patch, HBASE-16505-v11.patch, HBASE-16505-v11.patch,
> HBASE-16505-v12.patch, HBASE-16505-v2.patch, HBASE-16505-v3.patch,
> HBASE-16505-v4.patch, HBASE-16505-v5.patch, HBASE-16505-v6.patch,
> HBASE-16505-v7.patch, HBASE-16505-v8.patch, HBASE-16505-v9.patch
>
>
> If we want to know the correct setting of timeout in read/write path, we need
> add a new parameter in operation-methods of Region.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16788) Race in compacted file deletion between HStore close() and closeAndArchiveCompactedFiles()
[
https://issues.apache.org/jira/browse/HBASE-16788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563563#comment-15563563
]
Gary Helmling commented on HBASE-16788:
---
{quote}
If I understand the code correctly, it would take longer for the close() to
complete when concurrent CompactedHFilesDischargeHandler operation gets the
archiveLock first.
If this is not a concern, I am fine with your patch.
{quote}
It's true that close() may be blocked by the discharge chore thread if it is
holding the archiveLock. But whether the work for archiving compacted HFiles
is being done by the discharge thread or by close(), the same work needs to be
done before close() can complete. So I don't expect this to appreciably change
the time taken by close(). It just means that if close() is blocked by the
discharger, it should be able to skip over the archive step once it gets to run.
> Race in compacted file deletion between HStore close() and
> closeAndArchiveCompactedFiles()
> --
>
> Key: HBASE-16788
> URL: https://issues.apache.org/jira/browse/HBASE-16788
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.3.0
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Blocker
> Attachments: 16788-suggest.v2, HBASE-16788.001.patch,
> HBASE-16788.002.patch, HBASE-16788_1.patch
>
>
> HBASE-13082 changed the way that compacted files are archived from being done
> inline on compaction completion to an async cleanup by the
> CompactedHFilesDischarger chore. It looks like the changes to HStore to
> support this introduced a race condition in the compacted HFile archiving.
> In the following sequence, we can wind up with two separate threads trying to
> archive the same HFiles, causing a regionserver abort:
> # compaction completes normally and the compacted files are added to
> {{compactedfiles}} in HStore's DefaultStoreFileManager
> # *threadA*: CompactedHFilesDischargeHandler runs in a RS executor service,
> calling closeAndArchiveCompactedFiles()
> ## obtains HStore readlock
> ## gets a copy of compactedfiles
> ## releases readlock
> # *threadB*: calls HStore.close() as part of region close
> ## obtains HStore writelock
> ## calls DefaultStoreFileManager.clearCompactedfiles(), getting a copy of
> same compactedfiles
> # *threadA*: calls HStore.removeCompactedfiles(compactedfiles)
> ## archives files in {compactedfiles} in HRegionFileSystem.removeStoreFiles()
> ## call HStore.clearCompactedFiles()
> ## waits on write lock
> # *threadB*: continues with close()
> ## calls removeCompactedfiles(compactedfiles)
> ## calls HRegionFIleSystem.removeStoreFiles() ->
> HFileArchiver.archiveStoreFiles()
> ## receives FileNotFoundException because the files have already been
> archived by threadA
> ## throws IOException
> # RS aborts
> I think the combination of fetching the compactedfiles list and removing the
> files needs to be covered by locking. Options I see are:
> * Modify HStore.closeAndArchiveCompactedFiles(): use writelock instead of
> readlock and move the call to removeCompactedfiles() inside the lock. This
> means the read operations will be blocked while the files are being archived,
> which is bad.
> * Synchronize closeAndArchiveCompactedFiles() and modify close() to call it
> instead of calling removeCompactedfiles() directly
> * Add a separate lock for compacted files removal and use in
> closeAndArchiveCompactedFiles() and close()
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15109) HM/RS failed to start when "fs.hdfs.impl.disable.cache" is set to true
[
https://issues.apache.org/jira/browse/HBASE-15109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563555#comment-15563555
]
Ted Yu commented on HBASE-15109:
+1
> HM/RS failed to start when "fs.hdfs.impl.disable.cache" is set to true
> --
>
> Key: HBASE-15109
> URL: https://issues.apache.org/jira/browse/HBASE-15109
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Attachments: HBASE-15109-V2.patch, HBASE-15109-branch-1.patch,
> HBASE-15109.patch, HBASE-15109.patch
>
>
> HMaster failed to start during installing ShutdownHook when
> "fs.hdfs.impl.disable.cache" is set to true.
> {noformat}
> 2016-10-10 10:33:28,367 FATAL [master/HOST-10-18-106-146/10.18.106.146:16000]
> master.HMaster: Unhandled: Failed suppression of fs shutdown hook:
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@19a003b4
> java.lang.RuntimeException: Failed suppression of fs shutdown hook:
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@19a003b4
> at
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:204)
> at
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:84)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:950)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> And RegionServer is in blocking state until a master is available,
> {noformat}
> "regionserver/HOST-10-18-106-146/10.18.106.146:16020" #17 prio=5 os_prio=0
> tid=0x00e4e800 nid=0xdc38 in Object.wait() [0x7f329dac6000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:159)
> - locked <0xae25c0c8> (a
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:870)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:809)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:783)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:943)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> During installation, first time it will try to suppress the HDFS shutdownhook
> by removing the hdfsclientfinalizer from HDFS. Since
> fs.hdfs.impl.disable.cache is enabled, so removal will be unsuccessful from
> HDFS (FS is not cached) and RuntimeException will be thrown with "Failed
> suppression of fs shutdown hook" message.
> In ShutdownHook,
> {code}
> if (!fsShutdownHooks.containsKey(hdfsClientFinalizer) &&
> !ShutdownHookManager.deleteShutdownHook(hdfsClientFinalizer)) {
> throw new RuntimeException("Failed suppression of fs shutdown hook: " +
> hdfsClientFinalizer);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16802) Procedure v2 - group procedure cleaning
[ https://issues.apache.org/jira/browse/HBASE-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-16802: Status: Patch Available (was: Open) > Procedure v2 - group procedure cleaning > --- > > Key: HBASE-16802 > URL: https://issues.apache.org/jira/browse/HBASE-16802 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-16802-v0.patch > > > group the cleaning of the evicted procedures -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16802) Procedure v2 - group procedure cleaning
[ https://issues.apache.org/jira/browse/HBASE-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-16802: Attachment: HBASE-16802-v0.patch > Procedure v2 - group procedure cleaning > --- > > Key: HBASE-16802 > URL: https://issues.apache.org/jira/browse/HBASE-16802 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-16802-v0.patch > > > group the cleaning of the evicted procedures -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16802) Procedure v2 - group procedure cleaning
Matteo Bertozzi created HBASE-16802: --- Summary: Procedure v2 - group procedure cleaning Key: HBASE-16802 URL: https://issues.apache.org/jira/browse/HBASE-16802 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 2.0.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Trivial Fix For: 2.0.0 group the cleaning of the evicted procedures -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16770) ZKCanary does not return exit code as non-zero incase of failure
[ https://issues.apache.org/jira/browse/HBASE-16770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563521#comment-15563521 ] churro morales commented on HBASE-16770: Do you get this issue when the entire ZK cluster is down? Or only when you hit certain hosts? > ZKCanary does not return exit code as non-zero incase of failure > > > Key: HBASE-16770 > URL: https://issues.apache.org/jira/browse/HBASE-16770 > Project: HBase > Issue Type: Bug > Components: canary >Reporter: Vishal Khandelwal > > 2016-10-05 13:30:54,255 WARN [Thread-550] client.ZooKeeperRegistry(88): > Can't retrieve clusterId from Zookeeper > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss for /hbase/hbaseid > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102) > at > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:220) > at > org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:417) > at > org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) > at > org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:83) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:926) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.(HConnectionManager.java:711) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at > org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:463) > at > org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:358) > at org.apache.hadoop.hbase.tool.Canary.run(Canary.java:672) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > com.salesforce.hbase.synthtests.tests.ZKCanaryTests.testZKCanaryRead(ZKCanaryTests.java:50) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at > org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16800) Support for smart querying of our published jmx per server over http+json
[ https://issues.apache.org/jira/browse/HBASE-16800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563522#comment-15563522 ] Appy commented on HBASE-16800: -- To give more idea about numbers: RS /jmx response is ~350kb (~200-300ms). Without server-side filtering, collecting metrics for dashboard for a 1000node cluster will be 350M. Won't be possible to refresh dashboard every 30sec. With filtering (?qry=Hadoop:service=HBase,name=RegionServer,sub=Replication), it's just ~1kb which totally makes it possible to have a dashboard which can be refreshed with a good cadence. Thank [~stack] for figuring this out. > Support for smart querying of our published jmx per server over http+json > - > > Key: HBASE-16800 > URL: https://issues.apache.org/jira/browse/HBASE-16800 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: stack >Assignee: stack > Attachments: HBASE-16800.master.001.patch, > HBASE-16800.master.002.patch > > > We currently expose our jmx metrics via a json dump in a servlet at /jmx. You > get all or nothing (You can add a '?description=true' to get a description > for each metric published but that is it). > A means of being able to query by metric or by a set of metrics would make > graphing easier to do, especially over http dumping json. In particular, our > [~appy] is trying to put up pages that give insight into current state of > replication. > Related, HBASE-11747 ClusterStatus is too bulky, is about how whenever we > need a metric 'exposed', our only means is by bulking out the heartbeat > adding payload on each message we send the server (In the issue, w/ 1M > regions, each RS is sending 100MB of 'status' every second). We need to undo > metrics collection from clusterstatus function > Bonus if able to compress/cache payloads, security, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14417) Incremental backup and bulk loading
[ https://issues.apache.org/jira/browse/HBASE-14417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563507#comment-15563507 ] Vladimir Rodionov commented on HBASE-14417: --- >> One potential approach is to store hfile information in zookeeper. -1. No Zk, please. > Incremental backup and bulk loading > --- > > Key: HBASE-14417 > URL: https://issues.apache.org/jira/browse/HBASE-14417 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Ted Yu >Priority: Critical > Labels: backup > Fix For: 2.0.0 > > Attachments: 14417.v1.txt, 14417.v2.txt, 14417.v6.txt > > > Currently, incremental backup is based on WAL files. Bulk data loading > bypasses WALs for obvious reasons, breaking incremental backups. The only way > to continue backups after bulk loading is to create new full backup of a > table. This may not be feasible for customers who do bulk loading regularly > (say, every day). > Google doc for design: > https://docs.google.com/document/d/1ACCLsecHDvzVSasORgqqRNrloGx4mNYIbvAU7lq5lJE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16767) Mob compaction needs to clean up files in /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload when running into IO exceptions
[ https://issues.apache.org/jira/browse/HBASE-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563477#comment-15563477 ] Jonathan Hsieh commented on HBASE-16767: [~tedyu], thanks for catching this. [~busbey] thanks for fixing it. My bad. > Mob compaction needs to clean up files in /hbase/mobdir/.tmp and > /hbase/mobdir/.tmp/.bulkload when running into IO exceptions > -- > > Key: HBASE-16767 > URL: https://issues.apache.org/jira/browse/HBASE-16767 > Project: HBase > Issue Type: Bug > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0 > > Attachments: HBASE-16767.master.001.patch, > HBASE-16767.master.002.patch, HBASE-16767.master.003.patch, > HBASE-16767.master.004.patch, HBASE-16767.master.004.patch, > HBASE-16767.master.005.patch > > > For the following lines, when it runs into IOException, it does not clean up > the files under /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload. > It could be due to creating of new mob file or new reference file under > .bulkload directory fails. Or when mob files from ./tmp to the mob region > directory fails. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16699) Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval()
[ https://issues.apache.org/jira/browse/HBASE-16699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-16699: - Attachment: HBASE-16699.branch-1.001.patch > Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval() > > > Key: HBASE-16699 > URL: https://issues.apache.org/jira/browse/HBASE-16699 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.4.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-16699.branch-1.001.patch, > HBASE-16699.master.001.patch, HBASE-16699.master.002.patch > > > It seems that there are more overflow in other places, and a concurrent issue. > I will post a patch within one or 2 days after I figure out adding new > unittest cases. > Please see the following two lines. Once it overflows, it will cause wrong > behavior. For unconfigured RateLimiters, we should have simpler logic to > byPass the check. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L37 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/AverageIntervalRateLimiter.java#L51 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16663) JMX ConnectorServer stopped when unauthorized user try to stop HM/RS/cluster
[
https://issues.apache.org/jira/browse/HBASE-16663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563461#comment-15563461
]
Andrew Purtell commented on HBASE-16663:
Thanks, will try again today
> JMX ConnectorServer stopped when unauthorized user try to stop HM/RS/cluster
>
>
> Key: HBASE-16663
> URL: https://issues.apache.org/jira/browse/HBASE-16663
> Project: HBase
> Issue Type: Bug
> Components: metrics, security
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-16663-0.98-V4.patch, HBASE-16663-0.98.patch,
> HBASE-16663-V2.patch, HBASE-16663-V3.patch, HBASE-16663-V4.patch,
> HBASE-16663-branch-1.patch, HBASE-16663.patch
>
>
> After HBASE-16284, unauthorized user will not able allowed to stop
> HM/RS/cluster, but while executing "cpHost.preStopMaster()", ConnectorServer
> will be stopped before AccessController validation.
> hbase-site.xml,
> {noformat}
>
> hbase.coprocessor.master.classes
>
> org.apache.hadoop.hbase.JMXListener,org.apache.hadoop.hbase.security.access.AccessController
>
>
> hbase.coprocessor.regionserver.classes
>
> org.apache.hadoop.hbase.JMXListener,org.apache.hadoop.hbase.security.access.AccessController
>
> {noformat}
> HBaseAdmin.stopMaster(),
> {noformat}
> 2016-09-20 21:12:26,796 INFO
> [RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=16000]
> hbase.JMXListener: ConnectorServer stopped!
> 2016-09-20 21:13:55,380 WARN
> [RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=16000]
> security.ShellBasedUnixGroupsMapping: got exception trying to get groups for
> user P72981
> ExitCodeException exitCode=1: id: P72981: No such user
> 2016-09-20 21:14:00,495 ERROR
> [RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=16000]
> master.MasterRpcServices: Exception occurred while stopping master
> org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
> permissions for user 'P72981' (global, action=ADMIN)
> at
> org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:546)
> at
> org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522)
> at
> org.apache.hadoop.hbase.security.access.AccessController.preStopMaster(AccessController.java:1297)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost$68.call(MasterCoprocessorHost.java:821)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.execOperation(MasterCoprocessorHost.java:1188)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.preStopMaster(MasterCoprocessorHost.java:817)
> at org.apache.hadoop.hbase.master.HMaster.stopMaster(HMaster.java:2352)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.stopMaster(MasterRpcServices.java:1364)
> {noformat}
> HBaseAdmin.stopRegionServer(rs-host-port),
> {noformat}
> 2016-09-20 20:59:01,234 INFO
> [RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=16020]
> hbase.JMXListener: ConnectorServer stopped!
> 2016-09-20 20:59:01,250 WARN
> [RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=16020]
> security.ShellBasedUnixGroupsMapping: got exception trying to get groups for
> user P72981
> ExitCodeException exitCode=1: id: P72981: No such user
> 2016-09-20 20:59:01,253 WARN
> [RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=16020]
> regionserver.HRegionServer: The region server did not stop
> org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
> permissions for user 'P72981' (global, action=ADMIN)
> at
> org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:546)
> at
> org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522)
> at
> org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501)
> at
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:84)
> at
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execOperation(RegionServerCoprocessorHost.java:256)
> at
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:80)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.stopServer(RSRpcServices.java:1961)
> {noformat}
> HBaseAdmin.shutdown(),
> {noformat}
> 2016-09-21 12:09:08,259 INFO
> [RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=16000]
> master.MasterRpcServices:
[jira] [Commented] (HBASE-14417) Incremental backup and bulk loading
[ https://issues.apache.org/jira/browse/HBASE-14417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563463#comment-15563463 ] Ted Yu commented on HBASE-14417: While working on BackupHFileCleaner, the counterpart to ReplicationHFileCleaner, I notice the potential impact on the server hosting hbase:backup because we need to have up-to-date information on the hfiles which are still referenced by the incremental backup. One potential approach is to store hfile information in zookeeper. This would also alleviate the issue mentioned above about reducing writing BulkLoadDescriptor's to hbase:backup table. Any suggestions, [~mbertozzi] [~vrodionov] ? > Incremental backup and bulk loading > --- > > Key: HBASE-14417 > URL: https://issues.apache.org/jira/browse/HBASE-14417 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Ted Yu >Priority: Critical > Labels: backup > Fix For: 2.0.0 > > Attachments: 14417.v1.txt, 14417.v2.txt, 14417.v6.txt > > > Currently, incremental backup is based on WAL files. Bulk data loading > bypasses WALs for obvious reasons, breaking incremental backups. The only way > to continue backups after bulk loading is to create new full backup of a > table. This may not be feasible for customers who do bulk loading regularly > (say, every day). > Google doc for design: > https://docs.google.com/document/d/1ACCLsecHDvzVSasORgqqRNrloGx4mNYIbvAU7lq5lJE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16750) hbase compilation failed on power system
[
https://issues.apache.org/jira/browse/HBASE-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563338#comment-15563338
]
Hudson commented on HBASE-16750:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #1762 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1762/])
Revert "HBASE-16750 hbase compilation failed on power system (Saravanan
(dimaspivak: rev 3c35a722d9c1c77826d7c86ee204274bfdaae65f)
* (edit) pom.xml
> hbase compilation failed on power system
>
>
> Key: HBASE-16750
> URL: https://issues.apache.org/jira/browse/HBASE-16750
> Project: HBase
> Issue Type: Bug
> Components: build, documentation
>Affects Versions: 1.1.2
>Reporter: Saravanan Krishnamoorthy
>Assignee: Saravanan Krishnamoorthy
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-16750.branch-1.patch, HBASE-16750.master.patch,
> apache_hbase_reference_guide.pdf, apache_hbase_reference_guide.pdfmarks,
> book.pdf, book.pdfmarks
>
>
> Hi,
> hbase compilation failed on IBM power system ppc64le architecture with below
> error:
> {code}
> Hbase Failure:
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 04:33 min
> [INFO] Finished at: 2016-09-30T08:58:47-04:00
> [INFO] Final Memory: 215M/843M
> [INFO]
>
> [ERROR] Failed to execute goal
> org.asciidoctor:asciidoctor-maven-plugin:1.5.2.1:process-asciidoc
> (output-pdf) on project hbase: Execution output-pdf of goal
> org.asciidoctor:asciidoctor-maven-plugin:1.5.2.1:process-asciidoc failed:
> (NotImplementedError) fstat unimplemented unsupported or native support
> failed to load -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.asciidoctor:asciidoctor-maven-plugin:1.5.2.1:process-asciidoc
> (output-pdf) on project hbase: Execution output-pdf of goal
> org.asciidoctor:asciidoctor-maven-plugin:1.5.2.1:process-asciidoc failed:
> (NotImplementedError) fstat unimplemented unsupported or native support
> failed to load
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
> output-pdf of goal
> org.asciidoctor:asciidoctor-maven-plugin:1.5.2.1:process-asciidoc failed:
> (NotImplementedError) fstat unimplemented unsupported or native support
> failed to load
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
> ... 20 more
> Caused by: org.jruby.exceptions.RaiseException: (NotImplementedError) fstat
> unimplemented unsupported or native support failed to load
> at org.jruby.RubyFile.size(org/jruby/RubyFile.java:1108)
> at
>
[jira] [Commented] (HBASE-10070) HBase read high-availability using timeline-consistent region replicas
[ https://issues.apache.org/jira/browse/HBASE-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563339#comment-15563339 ] Santosh Banerjee commented on HBASE-10070: -- So, that means the EndPoint co-processors could still fail if the primary replica is down.Right? In other words, they are not benefitted by read HA, which probably makes sense as long as the endpoint co-processors are used for write operations. > HBase read high-availability using timeline-consistent region replicas > -- > > Key: HBASE-10070 > URL: https://issues.apache.org/jira/browse/HBASE-10070 > Project: HBase > Issue Type: New Feature > Components: Admin, API, LatencyResilience >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Labels: needs_releasenote > Fix For: 2.0.0, 1.2.0 > > Attachments: HighAvailabilityDesignforreadsApachedoc.pdf > > > In the present HBase architecture, it is hard, probably impossible, to > satisfy constraints like 99th percentile of the reads will be served under 10 > ms. One of the major factors that affects this is the MTTR for regions. There > are three phases in the MTTR process - detection, assignment, and recovery. > Of these, the detection is usually the longest and is presently in the order > of 20-30 seconds. During this time, the clients would not be able to read the > region data. > However, some clients will be better served if regions will be available for > reads during recovery for doing eventually consistent reads. This will help > with satisfying low latency guarantees for some class of applications which > can work with stale reads. > For improving read availability, we propose a replicated read-only region > serving design, also referred as secondary regions, or region shadows. > Extending current model of a region being opened for reads and writes in a > single region server, the region will be also opened for reading in region > servers. The region server which hosts the region for reads and writes (as in > current case) will be declared as PRIMARY, while 0 or more region servers > might be hosting the region as SECONDARY. There may be more than one > secondary (replica count > 2). > Will attach a design doc shortly which contains most of the details and some > thoughts about development approaches. Reviews are more than welcome. > We also have a proof of concept patch, which includes the master and regions > server side of changes. Client side changes will be coming soon as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16752) Upgrading from 1.2 to 1.3 can lead to replication failures due to difference in RPC size limit
[ https://issues.apache.org/jira/browse/HBASE-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563323#comment-15563323 ] Ashu Pachauri commented on HBASE-16752: --- The current plan I have is to return a RequestTooBigException to the client. The behavior of the server currently when it encounters a giant request is to locally log the message and drop the connection after reading the data length, without having to read the entire rpc off the wire. I plan to maintain the current behavior. However, to return the response I need to read the request off the wire to get the RPC Call ID. I can think of two ways to tackle this: 1. Read only first few bytes of the rpc to try to decode the request header from them. This makes assumptions on request header format and size, and will need to be kept in sync with the changes to the request format. 2. Instead of returning response with correct Call ID, return with a special call ID (negative) for responses that are to be following by dropping the connection and modify the rpc connection implementation to handle there responses in a special way. This breaks the fundamental client server model, because server is sending special instructions to the client which the client did not ask for. Personally, I like the second approach because it just sets a special contract and does not require any funky stuff to decode the header, but I am open to suggestions. > Upgrading from 1.2 to 1.3 can lead to replication failures due to difference > in RPC size limit > -- > > Key: HBASE-16752 > URL: https://issues.apache.org/jira/browse/HBASE-16752 > Project: HBase > Issue Type: Bug > Components: Replication, rpc >Affects Versions: 1.3.0 >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > > In HBase 1.2, we don't limit size of a single RPC but in 1.3 we limit it by > default to 256 MB. This means that during upgrade scenarios (or when source > is 1.2 peer is already on 1.3), it's possible to encounter a situation where > we try to send an rpc with size greater than 256 MB because we never unroll a > WALEdit while sending replication traffic. > RpcServer throws the underlying exception locally, but closes the connection > with returning the underlying error to the client, and client only sees a > "Broken pipe" error. > I am not sure what is the proper fix here (or if one is needed) to make sure > this does not happen, but we should return the underlying exception to the > RpcClient, because without it, it can be difficult to diagnose the problem, > especially for someone new to HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10070) HBase read high-availability using timeline-consistent region replicas
[ https://issues.apache.org/jira/browse/HBASE-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563309#comment-15563309 ] Enis Soztutar commented on HBASE-10070: --- bq. Is there any documentation about how this feature affect co-processors, specially endpoint co-processors? No effect to endpoint coprocessors. Only the get requests or scan requests are performed with backup RPC requests to replicas. The endpoint RPCs will only go to the primary replica. The coprocessors which inject into preGet() etc will get called though, whenever the client does an RPC to the primary replica or secondary replicas. > HBase read high-availability using timeline-consistent region replicas > -- > > Key: HBASE-10070 > URL: https://issues.apache.org/jira/browse/HBASE-10070 > Project: HBase > Issue Type: New Feature > Components: Admin, API, LatencyResilience >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Labels: needs_releasenote > Fix For: 2.0.0, 1.2.0 > > Attachments: HighAvailabilityDesignforreadsApachedoc.pdf > > > In the present HBase architecture, it is hard, probably impossible, to > satisfy constraints like 99th percentile of the reads will be served under 10 > ms. One of the major factors that affects this is the MTTR for regions. There > are three phases in the MTTR process - detection, assignment, and recovery. > Of these, the detection is usually the longest and is presently in the order > of 20-30 seconds. During this time, the clients would not be able to read the > region data. > However, some clients will be better served if regions will be available for > reads during recovery for doing eventually consistent reads. This will help > with satisfying low latency guarantees for some class of applications which > can work with stale reads. > For improving read availability, we propose a replicated read-only region > serving design, also referred as secondary regions, or region shadows. > Extending current model of a region being opened for reads and writes in a > single region server, the region will be also opened for reading in region > servers. The region server which hosts the region for reads and writes (as in > current case) will be declared as PRIMARY, while 0 or more region servers > might be hosting the region as SECONDARY. There may be more than one > secondary (replica count > 2). > Will attach a design doc shortly which contains most of the details and some > thoughts about development approaches. Reviews are more than welcome. > We also have a proof of concept patch, which includes the master and regions > server side of changes. Client side changes will be coming soon as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16721) Concurrency issue in WAL unflushed seqId tracking
[
https://issues.apache.org/jira/browse/HBASE-16721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-16721:
--
Release Note: Fixed a bug in sequenceId tracking for the WALs that caused
WAL files to accumulate without being deleted due to a rare race condition.
> Concurrency issue in WAL unflushed seqId tracking
> -
>
> Key: HBASE-16721
> URL: https://issues.apache.org/jira/browse/HBASE-16721
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.0.0, 1.1.0, 1.2.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.4, 1.1.8
>
> Attachments: hbase-16721_addendum.patch,
> hbase-16721_v1.branch-1.patch, hbase-16721_v2.branch-1.patch,
> hbase-16721_v2.master.patch
>
>
> I'm inspecting an interesting case where in a production cluster, some
> regionservers ends up accumulating hundreds of WAL files, even with force
> flushes going on due to max logs. This happened multiple times on the
> cluster, but not on other clusters. The cluster has periodic memstore flusher
> disabled, however, this still does not explain why the force flush of regions
> due to max limit is not working. I think the periodic memstore flusher just
> masks the underlying problem, which is why we do not see this in other
> clusters.
> The problem starts like this:
> {code}
> 2016-09-21 17:49:18,272 INFO [regionserver//10.2.0.55:16020.logRoller]
> wal.FSHLog: Too many wals: logs=33, maxlogs=32; forcing flush of 1
> regions(s): d4cf39dc40ea79f5da4d0cf66d03cb1f
> 2016-09-21 17:49:18,273 WARN [regionserver//10.2.0.55:16020.logRoller]
> regionserver.LogRoller: Failed to schedule flush of
> d4cf39dc40ea79f5da4d0cf66d03cb1f, region=null, requester=null
> {code}
> then, it continues until the RS is restarted:
> {code}
> 2016-09-23 17:43:49,356 INFO [regionserver//10.2.0.55:16020.logRoller]
> wal.FSHLog: Too many wals: logs=721, maxlogs=32; forcing flush of 1
> regions(s): d4cf39dc40ea79f5da4d0cf66d03cb1f
> 2016-09-23 17:43:49,357 WARN [regionserver//10.2.0.55:16020.logRoller]
> regionserver.LogRoller: Failed to schedule flush of
> d4cf39dc40ea79f5da4d0cf66d03cb1f, region=null, requester=null
> {code}
> The problem is that region {{d4cf39dc40ea79f5da4d0cf66d03cb1f}} is already
> split some time ago, and was able to flush its data and split without any
> problems. However, the FSHLog still thinks that there is some unflushed data
> for this region.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563283#comment-15563283 ] Vladimir Rodionov edited comment on HBASE-14123 at 10/10/16 7:52 PM: - v29. Contains refactoring patch HBASE-16727. All unit and IT test pass. was (Author: vrodionov): v29. Contains refactoring patch HBASE-16727. > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: 14123-master.v14.txt, 14123-master.v15.txt, > 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, > 14123-master.v19.txt, 14123-master.v2.txt, 14123-master.v20.txt, > 14123-master.v21.txt, 14123-master.v24.txt, 14123-master.v25.txt, > 14123-master.v27.txt, 14123-master.v28.txt, 14123-master.v29.full.txt, > 14123-master.v3.txt, 14123-master.v5.txt, 14123-master.v6.txt, > 14123-master.v7.txt, 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, > HBASE-14123-for-7912-v1.patch, HBASE-14123-for-7912-v6.patch, > HBASE-14123-v1.patch, HBASE-14123-v10.patch, HBASE-14123-v11.patch, > HBASE-14123-v12.patch, HBASE-14123-v13.patch, HBASE-14123-v15.patch, > HBASE-14123-v16.patch, HBASE-14123-v2.patch, HBASE-14123-v3.patch, > HBASE-14123-v4.patch, HBASE-14123-v5.patch, HBASE-14123-v6.patch, > HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16801) The Append/Increment may return the data came from future
[ https://issues.apache.org/jira/browse/HBASE-16801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563294#comment-15563294 ] Ted Yu commented on HBASE-16801: +1 > The Append/Increment may return the data came from future > - > > Key: HBASE-16801 > URL: https://issues.apache.org/jira/browse/HBASE-16801 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ChiaPing Tsai >Assignee: ChiaPing Tsai >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16801.v0.patch > > > OperationContext maintains the mvcc as a static member, so any Append’s and > Increment’s read point will be changed by others. That is, a retrying > Append/Increment may “see” the future data. > This is a master only issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14123: -- Attachment: 14123-master.v29.full.txt v29. Contains refactoring patch HBASE-16727. > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: 14123-master.v14.txt, 14123-master.v15.txt, > 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, > 14123-master.v19.txt, 14123-master.v2.txt, 14123-master.v20.txt, > 14123-master.v21.txt, 14123-master.v24.txt, 14123-master.v25.txt, > 14123-master.v27.txt, 14123-master.v28.txt, 14123-master.v29.full.txt, > 14123-master.v3.txt, 14123-master.v5.txt, 14123-master.v6.txt, > 14123-master.v7.txt, 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, > HBASE-14123-for-7912-v1.patch, HBASE-14123-for-7912-v6.patch, > HBASE-14123-v1.patch, HBASE-14123-v10.patch, HBASE-14123-v11.patch, > HBASE-14123-v12.patch, HBASE-14123-v13.patch, HBASE-14123-v15.patch, > HBASE-14123-v16.patch, HBASE-14123-v2.patch, HBASE-14123-v3.patch, > HBASE-14123-v4.patch, HBASE-14123-v5.patch, HBASE-14123-v6.patch, > HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
