[jira] [Commented] (HBASE-20204) Add locking to RefreshFileConnections in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-20204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409089#comment-16409089
]
Hadoop QA commented on HBASE-20204:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
59s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
59s{color} | {color:red} hbase-server: The patch generated 4 new + 7 unchanged
- 0 fixed = 11 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 12s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}156m
14s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}195m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20204 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915553/HBASE-20204.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 9f1573638a2e 4.4.0-89-generic #112-Ubuntu SMP Mon Jul 31
19:38:41 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d7475b92e8 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/12066/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12066/testReport/ |
| Max. process+thread count | 4177 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-20219) An error occurs when scanning with reversed=true and loadColumnFamiliesOnDemand=true
[
https://issues.apache.org/jira/browse/HBASE-20219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409059#comment-16409059
]
Toshihiro Suzuki commented on HBASE-20219:
--
I just attached the v2 patch. I realized that it is not enough to add a client
side check, because the default loadColumnFamiliesOnDemand is decided in server
side (by the property "hbase.hregion.scan.loadColumnFamiliesOnDemand"). So I
added a server side check in the v2 patch. Thanks.
> An error occurs when scanning with reversed=true and
> loadColumnFamiliesOnDemand=true
>
>
> Key: HBASE-20219
> URL: https://issues.apache.org/jira/browse/HBASE-20219
> Project: HBase
> Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Attachments: HBASE-20219-UT.patch, HBASE-20219.master.001.patch,
> HBASE-20219.master.002.patch
>
>
> I'm facing the following error when scanning with reversed=true and
> loadColumnFamiliesOnDemand=true:
> {code}
> java.lang.IllegalStateException: requestSeek cannot be called on
> ReversedKeyValueHeap
> at
> org.apache.hadoop.hbase.regionserver.ReversedKeyValueHeap.requestSeek(ReversedKeyValueHeap.java:66)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.joinedHeapMayHaveData(HRegion.java:6725)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6652)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:6364)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3108)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3345)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41548)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {code}
> I will attach a UT patch to reproduce this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20111) Able to split region explicitly even on shouldSplit return false from split policy
[ https://issues.apache.org/jira/browse/HBASE-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409056#comment-16409056 ] Rajeshbabu Chintaguntla commented on HBASE-20111: - [~elserj] Yes I am looking into them. > Able to split region explicitly even on shouldSplit return false from split > policy > -- > > Key: HBASE-20111 > URL: https://issues.apache.org/jira/browse/HBASE-20111 > Project: HBase > Issue Type: Bug >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-20111.001.branch-2.0.patch, HBASE-20111.patch, > HBASE-20111_test.patch > > > Currently able to split the region explicitly even when the split policy > returns from shouldSplit. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20219) An error occurs when scanning with reversed=true and loadColumnFamiliesOnDemand=true
[
https://issues.apache.org/jira/browse/HBASE-20219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki updated HBASE-20219:
-
Attachment: HBASE-20219.master.002.patch
> An error occurs when scanning with reversed=true and
> loadColumnFamiliesOnDemand=true
>
>
> Key: HBASE-20219
> URL: https://issues.apache.org/jira/browse/HBASE-20219
> Project: HBase
> Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Attachments: HBASE-20219-UT.patch, HBASE-20219.master.001.patch,
> HBASE-20219.master.002.patch
>
>
> I'm facing the following error when scanning with reversed=true and
> loadColumnFamiliesOnDemand=true:
> {code}
> java.lang.IllegalStateException: requestSeek cannot be called on
> ReversedKeyValueHeap
> at
> org.apache.hadoop.hbase.regionserver.ReversedKeyValueHeap.requestSeek(ReversedKeyValueHeap.java:66)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.joinedHeapMayHaveData(HRegion.java:6725)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6652)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:6364)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3108)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3345)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41548)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {code}
> I will attach a UT patch to reproduce this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-13300) Fix casing in getTimeStamp() and setTimestamp() for Mutations
[ https://issues.apache.org/jira/browse/HBASE-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13300: -- Status: Patch Available (was: In Progress) > Fix casing in getTimeStamp() and setTimestamp() for Mutations > - > > Key: HBASE-13300 > URL: https://issues.apache.org/jira/browse/HBASE-13300 > Project: HBase > Issue Type: Bug > Components: API >Affects Versions: 1.0.0 >Reporter: Lars George >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-13300.branch-2.001.patch, > HBASE-13300.master.001.patch, HBASE-13300.master.002.patch, > HBASE-13300.master.003.patch, HBASE-13300.xlsx > > > For some reason we have two ways of writing this method. It should be > consistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-13300) Fix casing in getTimeStamp() and setTimestamp() for Mutations
[ https://issues.apache.org/jira/browse/HBASE-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reassigned HBASE-13300: - Assignee: Jan Hentschel (was: stack) > Fix casing in getTimeStamp() and setTimestamp() for Mutations > - > > Key: HBASE-13300 > URL: https://issues.apache.org/jira/browse/HBASE-13300 > Project: HBase > Issue Type: Bug > Components: API >Affects Versions: 1.0.0 >Reporter: Lars George >Assignee: Jan Hentschel >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-13300.branch-2.001.patch, > HBASE-13300.master.001.patch, HBASE-13300.master.002.patch, > HBASE-13300.master.003.patch, HBASE-13300.xlsx > > > For some reason we have two ways of writing this method. It should be > consistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-13300) Fix casing in getTimeStamp() and setTimestamp() for Mutations
[ https://issues.apache.org/jira/browse/HBASE-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reassigned HBASE-13300: - Assignee: stack (was: Jan Hentschel) > Fix casing in getTimeStamp() and setTimestamp() for Mutations > - > > Key: HBASE-13300 > URL: https://issues.apache.org/jira/browse/HBASE-13300 > Project: HBase > Issue Type: Bug > Components: API >Affects Versions: 1.0.0 >Reporter: Lars George >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-13300.branch-2.001.patch, > HBASE-13300.master.001.patch, HBASE-13300.master.002.patch, > HBASE-13300.master.003.patch, HBASE-13300.xlsx > > > For some reason we have two ways of writing this method. It should be > consistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20237) Put back getClosestRowBefore and throw UnknownProtocolException instead... for asynchbase client
[ https://issues.apache.org/jira/browse/HBASE-20237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20237: -- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to branch-2.0, branch-2, and master. > Put back getClosestRowBefore and throw UnknownProtocolException instead... > for asynchbase client > > > Key: HBASE-20237 > URL: https://issues.apache.org/jira/browse/HBASE-20237 > Project: HBase > Issue Type: Bug > Components: compatibility, Operability >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-20237.branch-2.001.patch, > HBASE-20237.branch-2.002.patch > > > asychbase can work against all hbase versions. This fact has saved us a few > times; e.g. LINE were able to migrate from 0.94 to 1.2 going in part via > asynchbase and its ability to work against all servers. > hbase2 breaks this ability of asynchbase. There is nothing for asynchbase to > figure definitively that it is talking to an hbase2 server (See HBASE-20225). > Lets add back something it can leverage. HBASE-13954 did a nice job purging > getClosestRowBefore. Lets put back an RPC stuff that throws an exception if > it gets called. Thats enough for asynchbase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408991#comment-16408991 ] stack edited comment on HBASE-20188 at 3/22/18 4:17 AM: h2. HBase1 vs HBase2 I ran a 2.5B ITBLL against 7 node cluster with NO chaos, first against 1.2.7, then against branch-2.0 tip. Same default configs in both cases (16G heaps). First ITBLL Generate (Random Writes) and then Verify (Reads). HBase2 completed in less time using less CPU and GC. Less memstore used too (the in-memory compaction segments won't show because no metrics exposed). Looking at graphs, hbase2 runs smoother than hbase1; less erratic fluctuation in reads and writes (The Master UI must have changed how it accounts operations because hbase1 shows 100k ops/second where hbase2 shows hundreds of ops a second). When hbase1 was running MR tasks were all RegionTooBusyExceptions. I didn't check hbase2 but I did check the metrics for RegionTooBusy and it said zero (This metric seems broke on branch-1). See attached graphs. Let me do straight YCSB next. was (Author: stack): h2. HBase1 vs HBase2 I ran a 2.5B ITBLL against 7 node cluster with NO chaos, first against 1.2.7, then against branch-2.0 tip. Same default configs in both cases (16G heaps). First ITBLL Generate (Random Writes) and then Verify (Reads). HBase2 completed in less time using less CPU and GC. Less memstore used too (the in-memory compaction segments won't show because no metrics exposed). Looking at graphs, hbase2 runs smoother than hbase1; less erratic fluctuation in reads and writes (The Master UI must have changed how it accounts operations because hbase1 shows 100k ops/second where hbase2 shows hundreds of ops a second). See attached graphs. Let me do straight YCSB next. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, flamegraph-1072.1.svg, > flamegraph-1072.2.svg, tree.txt > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408991#comment-16408991 ] stack edited comment on HBASE-20188 at 3/22/18 4:17 AM: h2. HBase1 vs HBase2 h3. ITBLL 2.5B, 6 RegionServers I ran a 2.5B ITBLL against 7 node cluster with NO chaos, first against 1.2.7, then against branch-2.0 tip. Same default configs in both cases (16G heaps). First ITBLL Generate (Random Writes) and then Verify (Reads). HBase2 completed in less time using less CPU and GC. Less memstore used too (the in-memory compaction segments won't show because no metrics exposed). Looking at graphs, hbase2 runs smoother than hbase1; less erratic fluctuation in reads and writes (The Master UI must have changed how it accounts operations because hbase1 shows 100k ops/second where hbase2 shows hundreds of ops a second). When hbase1 was running MR tasks were all RegionTooBusyExceptions. I didn't check hbase2 but I did check the metrics for RegionTooBusy and it said zero (This metric seems broke on branch-1). See attached graphs. Let me do straight YCSB next. was (Author: stack): h2. HBase1 vs HBase2 I ran a 2.5B ITBLL against 7 node cluster with NO chaos, first against 1.2.7, then against branch-2.0 tip. Same default configs in both cases (16G heaps). First ITBLL Generate (Random Writes) and then Verify (Reads). HBase2 completed in less time using less CPU and GC. Less memstore used too (the in-memory compaction segments won't show because no metrics exposed). Looking at graphs, hbase2 runs smoother than hbase1; less erratic fluctuation in reads and writes (The Master UI must have changed how it accounts operations because hbase1 shows 100k ops/second where hbase2 shows hundreds of ops a second). When hbase1 was running MR tasks were all RegionTooBusyExceptions. I didn't check hbase2 but I did check the metrics for RegionTooBusy and it said zero (This metric seems broke on branch-1). See attached graphs. Let me do straight YCSB next. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, flamegraph-1072.1.svg, > flamegraph-1072.2.svg, tree.txt > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20229) ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on
[
https://issues.apache.org/jira/browse/HBASE-20229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409008#comment-16409008
]
Toshihiro Suzuki commented on HBASE-20229:
--
I attached the v3 patch to fix the checkstyle error.
> ConnectionImplementation.locateRegions() returns duplicated entries when
> region replication is on
> -
>
> Key: HBASE-20229
> URL: https://issues.apache.org/jira/browse/HBASE-20229
> Project: HBase
> Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20229.master.001.patch,
> HBASE-20229.master.002.patch, HBASE-20229.master.003.patch
>
>
> The issue is when region replication is on,
> ConnectionImplementation.locateRegions() returns duplicated entries.
> In the test in my patch, the table should have 1 primary region + 2
> secondaries, but ConnectionImplementation.locateRegions() returns 9 regions.
> Every region repeats 3 times (3 = replicas count).
> I think this is because the following code calls locateRegion() even for
> replica regions and then the result triples.
> {code:java}
> for (RegionInfo regionInfo : regions) {
> RegionLocations list = locateRegion(tableName,
> regionInfo.getStartKey(), useCache, true);
> if (list != null) {
> for (HRegionLocation loc : list.getRegionLocations()) {
> if (loc != null) {
> locations.add(loc);
> }
> }
> }
> {code}
> The fix in my patch is to make it call locateRegion() only for a primary
> region.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20229) ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on
[
https://issues.apache.org/jira/browse/HBASE-20229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki updated HBASE-20229:
-
Attachment: HBASE-20229.master.003.patch
> ConnectionImplementation.locateRegions() returns duplicated entries when
> region replication is on
> -
>
> Key: HBASE-20229
> URL: https://issues.apache.org/jira/browse/HBASE-20229
> Project: HBase
> Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20229.master.001.patch,
> HBASE-20229.master.002.patch, HBASE-20229.master.003.patch
>
>
> The issue is when region replication is on,
> ConnectionImplementation.locateRegions() returns duplicated entries.
> In the test in my patch, the table should have 1 primary region + 2
> secondaries, but ConnectionImplementation.locateRegions() returns 9 regions.
> Every region repeats 3 times (3 = replicas count).
> I think this is because the following code calls locateRegion() even for
> replica regions and then the result triples.
> {code:java}
> for (RegionInfo regionInfo : regions) {
> RegionLocations list = locateRegion(tableName,
> regionInfo.getStartKey(), useCache, true);
> if (list != null) {
> for (HRegionLocation loc : list.getRegionLocations()) {
> if (loc != null) {
> locations.add(loc);
> }
> }
> }
> {code}
> The fix in my patch is to make it call locateRegion() only for a primary
> region.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20116) Optimize the region last pushed sequence id layout on zk
[
https://issues.apache.org/jira/browse/HBASE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409002#comment-16409002
]
Hadoop QA commented on HBASE-20116:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m
56s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m
49s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m
52s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
17s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20116 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915582/HBASE-20116-addendum.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 9ad499cbf9ae 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d7475b92e8 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| hadoopcheck |
https://builds.apache.org/job/PreCommit-HBASE-Build/12067/artifact/patchprocess/patch-javac-2.6.5.txt
|
|
[jira] [Commented] (HBASE-19258) IntegrationTest for Backup and Restore
[
https://issues.apache.org/jira/browse/HBASE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408998#comment-16408998
]
Hadoop QA commented on HBASE-19258:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 24s{color}
| {color:red} hbase-it generated 1 new + 51 unchanged - 0 fixed = 52 total (was
51) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} hbase-it: The patch generated 4 new + 2 unchanged - 0
fixed = 6 total (was 2) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
18s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 24s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
36s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
44s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 50m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19258 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915585/HBASE-19258.v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux efcd33f5baff 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12
21:23:04 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d7475b92e8 |
| maven | version: Apache Maven 3.5.3
[jira] [Updated] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20188: -- Attachment: ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png ITBLL2.5B_1.2.7vs2.0.0_memstore.png ITBLL2.5B_1.2.7vs2.0.0_ops.png ITBLL2.5B_1.2.7vs2.0.0_iops.png ITBLL2.5B_1.2.7vs2.0.0_memheap.png ITBLL2.5B_1.2.7vs2.0.0_gctime.png ITBLL2.5B_1.2.7vs2.0.0_cpu.png ITBLL2.5B_1.2.7vs2.0.0_load.png > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, flamegraph-1072.1.svg, > flamegraph-1072.2.svg, tree.txt > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408991#comment-16408991 ] stack edited comment on HBASE-20188 at 3/22/18 3:29 AM: h2. HBase1 vs HBase2 I ran a 2.5B ITBLL against 7 node cluster with NO chaos, first against 1.2.7, then against branch-2.0 tip. Same default configs in both cases (16G heaps). First ITBLL Generate (Random Writes) and then Verify (Reads). HBase2 completed in less time using less CPU and GC. Less memstore used too (the in-memory compaction segments won't show because no metrics exposed). Looking at graphs, hbase2 runs smoother than hbase1; less erratic fluctuation in reads and writes (The Master UI must have changed how it accounts operations because hbase1 shows 100k ops/second where hbase2 shows hundreds of ops a second). See attached graphs. Let me do straight YCSB next. was (Author: stack): h2. HBase1 vs HBase2 I ran a 2.5B ITBLL against 7 node cluster with NO chaos, first against 1.2.7, then against branch-2.0 tip. Same default configs in both cases (16G heaps). First ITBLL Generate (Random Writes) and then Verify (Reads). HBase2 completed in less time using less CPU and GC. Looking at graphs, hbase2 runs smoother than hbase1; less erratic fluctuation in reads and writes (The Master UI must have changed how it accounts operations because hbase1 shows 100k ops/second where hbase2 shows hundreds of ops a second). See attached graphs. Let me do straight YCSB next. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, flamegraph-1072.1.svg, > flamegraph-1072.2.svg, tree.txt > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19258) IntegrationTest for Backup and Restore
[
https://issues.apache.org/jira/browse/HBASE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408995#comment-16408995
]
Ted Yu commented on HBASE-19258:
{code}
+if (plugins != null) {
+ conf.set(HFileCleaner.MASTER_HFILE_CLEANER_PLUGINS, (plugins == null ?
"" : plugins + ",") +
+BackupHFileCleaner.class.getName());
{code}
In the if block, check whether BackupHFileCleaner is already in the plugins.
{code}
+ } catch (IOException e) {
+LOG.error("Failed", e);
{code}
Shouldn't the IOE propagate back ?
> IntegrationTest for Backup and Restore
> --
>
> Key: HBASE-19258
> URL: https://issues.apache.org/jira/browse/HBASE-19258
> Project: HBase
> Issue Type: Test
> Components: integration tests
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HBASE-19258.v1.patch
>
>
> See chatter at https://docs.google.com/document/d/1xbPlLKjOcPq2LDqjbSkF6uND
> AG0mzgOxek6P3POLeMc/edit?usp=sharing
> We need to get an IntegrationTest in place for backup and restore.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408991#comment-16408991 ] stack commented on HBASE-20188: --- h2. HBase1 vs HBase2 I ran a 2.5B ITBLL against 7 node cluster with NO chaos, first against 1.2.7, then against branch-2.0 tip. Same default configs in both cases (16G heaps). First ITBLL Generate (Random Writes) and then Verify (Reads). HBase2 completed in less time using less CPU and GC. Looking at graphs, hbase2 runs smoother than hbase1; less erratic fluctuation in reads and writes (The Master UI must have changed how it accounts operations because hbase1 shows 100k ops/second where hbase2 shows hundreds of ops a second). See attached graphs. Let me do straight YCSB next. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: flamegraph-1072.1.svg, flamegraph-1072.2.svg, tree.txt > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20204) Add locking to RefreshFileConnections in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-20204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408988#comment-16408988
]
Hadoop QA commented on HBASE-20204:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
38s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
3s{color} | {color:red} hbase-server: The patch generated 4 new + 7 unchanged -
0 fixed = 11 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
25s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}160m
51s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}204m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20204 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915553/HBASE-20204.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f47b676f9d11 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12
21:23:04 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 91075276e7 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/12064/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12064/testReport/ |
| Max. process+thread count | 4093 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408975#comment-16408975
]
stack commented on HBASE-20233:
---
The metric numRegions in MBean
{{Hadoop:service=HBase,name=RegionServer,sub=Regions}} is redundant as it
happens.
The MBase {{Hadoop:service=HBase,name=RegionServer,sub=Server}} has
regionCount: 10. Purge {{numRegions}}.
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Priority: Trivial
> Labels: beginner
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18785) Blocker doc issues
[ https://issues.apache.org/jira/browse/HBASE-18785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408967#comment-16408967 ] stack commented on HBASE-18785: --- Don't forget to purge serial replication from branch-2.0 doc. HBASE-17918 > Blocker doc issues > -- > > Key: HBASE-18785 > URL: https://issues.apache.org/jira/browse/HBASE-18785 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Blocker > Fix For: 2.0.0 > > > Hang blocker documentation issues off this one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17918) document serial replication
[
https://issues.apache.org/jira/browse/HBASE-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408966#comment-16408966
]
stack commented on HBASE-17918:
---
[~Apache9] This patch is in branch-2, branch-2.0, and master. Should we close
it? (I'll do an edit of the refguide once I copy it from master to branch-2.0
and will remove this part of the doc from branch-2.0 version). Thanks.
> document serial replication
> ---
>
> Key: HBASE-17918
> URL: https://issues.apache.org/jira/browse/HBASE-17918
> Project: HBase
> Issue Type: Task
> Components: documentation, Replication
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Yi Mei
>Priority: Critical
> Fix For: 2.0.0-beta-1, 2.0.0
>
> Attachments: HBASE-17918.v1.patch, HBASE-17918.v2.patch,
> HBASE-17918.v3.patch
>
>
> It looks like HBASE-9465 addresses one of the major flaws in our existing
> replication (namely that order of delivery is not assured). All I see in the
> reference guide is a note on {{hbase.serial.replication.waitingMs}}. Instead
> we should cover this in the replication section, especially given that we
> call out the order of delivery limitation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20103) [pv2] AssignmentProcedure is too coarse grained
[ https://issues.apache.org/jira/browse/HBASE-20103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20103: -- Fix Version/s: (was: 2.0.0) 3.0.0 > [pv2] AssignmentProcedure is too coarse grained > --- > > Key: HBASE-20103 > URL: https://issues.apache.org/jira/browse/HBASE-20103 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 3.0.0 > > > Comes of work on HBASE-20100 but in particular, in feedback from [~Apache9] > https://mail.google.com/mail/u/0/#inbox/161d8e41054be406 > The AP is too coarse-grained. There is precheck+start, then transform state > is edit meta setting state to OPENING and then dispatch (rpc) Finish is > edit of meta and setting internal state. The edit of meta should be distinct > step at least. > Would save on duplicated ops -- e.g. re-editing hbase:meta and dispatching > another RPC -- if we fail going into finishing. [~Apache9] brings up our > perhaps masking other state change hiccups when steps are so coarse-grained. > Do same for unassignprocedure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20137) TestRSGroups is flakey
[ https://issues.apache.org/jira/browse/HBASE-20137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-20137. --- Resolution: Cannot Reproduce Fix Version/s: (was: 2.0.0) Not on the flakies list anymore, with a while. Nothing committed here. Marking as cannot reproduce. Will open new issue if it shows up again. > TestRSGroups is flakey > -- > > Key: HBASE-20137 > URL: https://issues.apache.org/jira/browse/HBASE-20137 > Project: HBase > Issue Type: Bug > Components: flakey >Affects Versions: 2.0.0-beta-2 >Reporter: stack >Assignee: stack >Priority: Major > Attachments: HBASE-20137.branch-2.001.patch, > HBASE-20137.branch-2.002.patch, HBASE-20137.branch-2.003.patch, > HBASE-20137.branch-2.003.patch > > > It was the single test that failed the hbase-2 nightlies in #440 at the > hadoop2 stage. > The failure manifests as a timeout. It actually has an interesting cause > calling into question some of the clauses in > UnassignProcedure#remoteCallFailed. > We are running a disabletable concurrent with a shutdown. pid=309 is the > disable. pid=311 is the interesting one. The below is a little hard to read > -- the exception 'message' is the the current procedure as a String... hard > to parse, fixing -- but we are trying to unassign as part of a the > disabletable. Our RPC fails because the server we are trying to rpc too is > currently being processed as crashed (pid=308 is a servercrashprocedure for > this server). As part of the processing of the failed RPC we will expire the > server -- if we can't RPC to it, it must be gone. The current procedure is > then suspended until it gets woken up by the servercrashprocedure triggered > by the expire only in this case we are shutting down so the expire is > ignored... The current procedure is left in its suspend state. This prevents > the Master going down. So we time out. > 2018-03-05 11:29:22,507 INFO [PEWorker-13] > assignment.RegionTransitionProcedure(213): Dispatch pid=311, ppid=309, > state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure > table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043, > server=1cfd208ff882,40584,1520249102524; rit=CLOSING, > location=1cfd208ff882,40584,1520249102524 > 2018-03-05 11:29:22,508 WARN [PEWorker-13] > assignment.RegionTransitionProcedure(187): Remote call failed pid=311, > ppid=309, state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure > table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043, > server=1cfd208ff882,40584,1520249102524; rit=CLOSING, > location=1cfd208ff882,40584,1520249102524; exception=pid=311, ppid=309, > state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure > table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043, > server=1cfd208ff882,40584,1520249102524 to 1cfd208ff882,40584,1520249102524 > 2018-03-05 11:29:22,508 WARN [PEWorker-13] > assignment.UnassignProcedure(276): Expiring server pid=311, ppid=309, > state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure > table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043, > server=1cfd208ff882,40584,1520249102524; rit=CLOSING, > location=1cfd208ff882,40584,1520249102524, > exception=org.apache.hadoop.hbase.master.assignment.FailedRemoteDispatchException: > pid=311, ppid=309, state=RUNNABLE:REGION_TRANSITION_DISPATCH; > UnassignProcedure table=Group_ns:testKillRS, > region=de7534c208a06502537cd95c248b3043, > server=1cfd208ff882,40584,1520249102524 to 1cfd208ff882,40584,1520249102524 > 2018-03-05 11:29:22,508 WARN [PEWorker-13] master.ServerManager(580): > Expiration of 1cfd208ff882,40584,1520249102524 but server shutdown already in > progress > I need to cater for case where the expire server is rejected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20224) Web UI is broken in standalone mode
[ https://issues.apache.org/jira/browse/HBASE-20224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408960#comment-16408960 ] Umesh Agashe commented on HBASE-20224: -- Thanks for the review, [~apurtell]! > Web UI is broken in standalone mode > --- > > Key: HBASE-20224 > URL: https://issues.apache.org/jira/browse/HBASE-20224 > Project: HBase > Issue Type: Bug > Components: UI, Usability >Affects Versions: 2.0.0-beta-2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > Attachments: hbase-20224.master.001.patch, > hbase-20224.master.002.patch > > > Web UI doesn't show up in standalone mode on default port. This can be seen > on master and branch-2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-12963) Add note about jdk8 compilation to the guide
[ https://issues.apache.org/jira/browse/HBASE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408950#comment-16408950 ] stack commented on HBASE-12963: --- Unscheduling. Seems fixed. Resolve [~busbey]? > Add note about jdk8 compilation to the guide > > > Key: HBASE-12963 > URL: https://issues.apache.org/jira/browse/HBASE-12963 > Project: HBase > Issue Type: Sub-task > Components: documentation >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > > HBASE-12695 fixed building 2.0.0-SNAP with JDK8, but right now it's only > documented in a release note. We should add a note to the "building hbase" > section of the ref guide. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-12963) Add note about jdk8 compilation to the guide
[ https://issues.apache.org/jira/browse/HBASE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12963: -- Fix Version/s: (was: 2.0.0) > Add note about jdk8 compilation to the guide > > > Key: HBASE-12963 > URL: https://issues.apache.org/jira/browse/HBASE-12963 > Project: HBase > Issue Type: Sub-task > Components: documentation >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > > HBASE-12695 fixed building 2.0.0-SNAP with JDK8, but right now it's only > documented in a release note. We should add a note to the "building hbase" > section of the ref guide. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17383) Improve log msg when offheap memstore breaches higher water mark
[
https://issues.apache.org/jira/browse/HBASE-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408947#comment-16408947
]
stack commented on HBASE-17383:
---
Unscheduling. Looks like we don't need this anymore? Resolve [~anoop.hbase] /
[~ram_krish] ?
> Improve log msg when offheap memstore breaches higher water mark
>
>
> Key: HBASE-17383
> URL: https://issues.apache.org/jira/browse/HBASE-17383
> Project: HBase
> Issue Type: Improvement
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Trivial
>
> Currently we get this log
> {code}
> 2016-12-28 21:11:14,349 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=39,queue=9,port=16041]
> regionserver.MemStoreFlusher: Blocking updates on
> stobdtserver5,16041,1482938527980: the global offheap memstore size 12.6 G +
> global memstore heap overhead 4.0 G is >= than blocking 12.6 G size
> {code}
> Here the global offheap memstore size is greater than the blocking size. The
> memstore heap overhead need not be included in this log unless the higher
> water mark breach is only due to the heap overhead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-17383) Improve log msg when offheap memstore breaches higher water mark
[
https://issues.apache.org/jira/browse/HBASE-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17383:
--
Fix Version/s: (was: 2.0.0)
> Improve log msg when offheap memstore breaches higher water mark
>
>
> Key: HBASE-17383
> URL: https://issues.apache.org/jira/browse/HBASE-17383
> Project: HBase
> Issue Type: Improvement
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Trivial
>
> Currently we get this log
> {code}
> 2016-12-28 21:11:14,349 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=39,queue=9,port=16041]
> regionserver.MemStoreFlusher: Blocking updates on
> stobdtserver5,16041,1482938527980: the global offheap memstore size 12.6 G +
> global memstore heap overhead 4.0 G is >= than blocking 12.6 G size
> {code}
> Here the global offheap memstore size is greater than the blocking size. The
> memstore heap overhead need not be included in this log unless the higher
> water mark breach is only due to the heap overhead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17819) Reduce the heap overhead for BucketCache
[ https://issues.apache.org/jira/browse/HBASE-17819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408942#comment-16408942 ] stack commented on HBASE-17819: --- Patch looks good. Where is tested? Thanks [~anoop.hbase] > Reduce the heap overhead for BucketCache > > > Key: HBASE-17819 > URL: https://issues.apache.org/jira/browse/HBASE-17819 > Project: HBase > Issue Type: Sub-task > Components: BucketCache >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-17819_V1.patch, HBASE-17819_V2.patch, > HBASE-17819_V3.patch, HBASE-17819_new_V1.patch, HBASE-17819_new_V1.patch > > > We keep Bucket entry map in BucketCache. Below is the math for heapSize for > the key , value into this map. > BlockCacheKey > --- > String hfileName - Ref - 4 > long offset - 8 > BlockType blockType - Ref - 4 > boolean isPrimaryReplicaBlock - 1 > Total = align(12 (Object) + 17 )= 32 > BucketEntry > > int offsetBase - 4 > int length - 4 > byte offset1 - 1 > byte deserialiserIndex - 1 > long accessCounter - 8 > BlockPriority priority - Ref - 4 > volatile boolean markedForEvict - 1 > AtomicInteger refCount - 16 + 4 > long cachedTime - 8 > Total = align(12 (Object) + 51) = 64 > ConcurrentHashMap Map.Entry - 40 > blocksByHFile ConcurrentSkipListSet Entry - 40 > Total = 32 + 64 + 80 = 176 > For 10 million blocks we will end up having 1.6GB of heap size. > This jira aims to reduce this as much as possible -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20224) Web UI is broken in standalone mode
[ https://issues.apache.org/jira/browse/HBASE-20224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408933#comment-16408933 ] stack commented on HBASE-20224: --- Ok. Patch seems good. Is branch-1 ok? Will apply to branch-2 . Waiting on hadoopqa. > Web UI is broken in standalone mode > --- > > Key: HBASE-20224 > URL: https://issues.apache.org/jira/browse/HBASE-20224 > Project: HBase > Issue Type: Bug > Components: UI, Usability >Affects Versions: 2.0.0-beta-2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > Attachments: hbase-20224.master.001.patch, > hbase-20224.master.002.patch > > > Web UI doesn't show up in standalone mode on default port. This can be seen > on master and branch-2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20116) Optimize the region last pushed sequence id layout on zk
[ https://issues.apache.org/jira/browse/HBASE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408923#comment-16408923 ] Ted Yu commented on HBASE-20116: lgtm > Optimize the region last pushed sequence id layout on zk > > > Key: HBASE-20116 > URL: https://issues.apache.org/jira/browse/HBASE-20116 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20116-addendum.patch, HBASE-20116.v1.patch, > HBASE-20116.v2.patch, HBASE-20116.v3.patch, HBASE-20116.v4.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20090) Properly handle Preconditions check failure in MemStoreFlusher$FlushHandler.run
[
https://issues.apache.org/jira/browse/HBASE-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408915#comment-16408915
]
Hudson commented on HBASE-20090:
Results for branch branch-2
[build #513 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/513/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/513//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/513//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/513//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Properly handle Preconditions check failure in
> MemStoreFlusher$FlushHandler.run
> ---
>
> Key: HBASE-20090
> URL: https://issues.apache.org/jira/browse/HBASE-20090
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: 20090-server-61260-01-07.log, 20090.v10.txt,
> 20090.v10.txt, 20090.v6.txt, 20090.v7.txt, 20090.v8.txt, 20090.v9.txt
>
>
> Copied the following from a comment since this was better description of the
> race condition.
> The original description was merged to the beginning of my first comment
> below.
> Observed the following in region server log (running on hadoop3 cluster):
> {code}
> 2018-02-26 16:06:50,044 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=16020]
> regionserver.MemStoreFlusher: Memstore is above high water mark and block
> 135ms
> 2018-02-26 16:06:50,049 ERROR [MemStoreFlusher.1]
> regionserver.MemStoreFlusher: Cache flusher failed for entry
> org.apache.hadoop.hbase.regionserver.
> MemStoreFlusher$WakeupFlushThread@2adfadd7
> java.lang.IllegalStateException
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkState(Preconditions.java:441)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:174)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$600(MemStoreFlusher.java:69)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:237)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Here is the Precondition from MemStoreFlusher#flushOneForGlobalPressure() :
> {code}
> Preconditions.checkState(
> (regionToFlush != null && regionToFlushSize > 0) ||
> bestRegionReplicaSize > 0);
> {code}
> When the Preconditions check fails, IllegalStateException would be raised.
> With more debug logging, we can see the scenario where the exception was
> triggered.
> {code}
> 2018-03-02 17:28:30,097 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit:
> Splitting TestTable,,1520011528142.0453f29030757eedb6e6a1c57e88c085.,
> compaction_queue=(0:0), split_queue=1
> 2018-03-02 17:28:30,098 DEBUG
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> regionserver.IncreasingToUpperBoundRegionSplitPolicy: ShouldSplit because
> info size=6.9G, sizeToCheck=256.0M, regionsWithCommonTable=1
> 2018-03-02 17:28:30,296 INFO
> [RpcServer.default.FPBQ.Fifo.handler=24,queue=0,port=16020]
> regionserver.MemStoreFlusher: wake up flusher due to ABOVE_ONHEAP_LOWER_MARK
> 2018-03-02 17:28:30,297 DEBUG [MemStoreFlusher.1]
> regionserver.MemStoreFlusher: Flush thread woke up because memory above low
> water=381.5 M
> 2018-03-02 17:28:30,297 INFO
> [RpcServer.default.FPBQ.Fifo.handler=25,queue=1,port=16020]
> regionserver.MemStoreFlusher: wake up flusher due to ABOVE_ONHEAP_LOWER_MARK
> 2018-03-02 17:28:30,298 DEBUG [MemStoreFlusher.1]
> regionserver.MemStoreFlusher: region
> TestTable,,1520011528142.0453f29030757eedb6e6a1c57e88c085. with size 400432696
> 2018-03-02 17:28:30,298 DEBUG [MemStoreFlusher.1]
> regionserver.MemStoreFlusher: region
> atlas_janus,,1519927429371.fbcb5e495344542daf8b499e4bac03ae. with size 0
> 2018-03-02 17:28:30,298 INFO [MemStoreFlusher.1]
> regionserver.MemStoreFlusher: Flush of region
> atlas_janus,,1519927429371.fbcb5e495344542daf8b499e4bac03ae. due to global
> heap pressure. Flush type=ABOVE_ONHEAP_LOWER_MARKTotal Memstore Heap
> size=381.9 MTotal Memstore Off-Heap size=0, Region memstore size=0
> 2018-03-02 17:28:30,298 INFO [MemStoreFlusher.1]
>
[jira] [Updated] (HBASE-20212) Make all Public classes have InterfaceAudience category
[ https://issues.apache.org/jira/browse/HBASE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20212: --- Attachment: HBASE-20212.branch-2.0.v0.patch > Make all Public classes have InterfaceAudience category > --- > > Key: HBASE-20212 > URL: https://issues.apache.org/jira/browse/HBASE-20212 > Project: HBase > Issue Type: Task > Components: Usability >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.0 > > Attachments: HBASE-20212.branch-2.0.v0.patch, > HBASE-20212.branch-2.v0.patch, HBASE-20212.v0.patch, HBASE-20212.v0.patch > > > The tasks will be resolved are shown below. > # add warbucks-maven-plugin to root pom > # make sure all sub modules ref the warbucks-maven-plugin > # remove old checker (TestInterfaceAudienceAnnotations) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20246) Remove the spark module
[ https://issues.apache.org/jira/browse/HBASE-20246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20246: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Push to branch-2 and branch-2.0 > Remove the spark module > --- > > Key: HBASE-20246 > URL: https://issues.apache.org/jira/browse/HBASE-20246 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-20246.branch-2.v0.patch.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20212) Make all Public classes have InterfaceAudience category
[ https://issues.apache.org/jira/browse/HBASE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408889#comment-16408889 ] Chia-Ping Tsai commented on HBASE-20212: -HBASE-20246- is resolved so the patch to branch-2 need to be rebased. > Make all Public classes have InterfaceAudience category > --- > > Key: HBASE-20212 > URL: https://issues.apache.org/jira/browse/HBASE-20212 > Project: HBase > Issue Type: Task > Components: Usability >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.0 > > Attachments: HBASE-20212.branch-2.v0.patch, HBASE-20212.v0.patch, > HBASE-20212.v0.patch > > > The tasks will be resolved are shown below. > # add warbucks-maven-plugin to root pom > # make sure all sub modules ref the warbucks-maven-plugin > # remove old checker (TestInterfaceAudienceAnnotations) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19258) IntegrationTest for Backup and Restore
[
https://issues.apache.org/jira/browse/HBASE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408873#comment-16408873
]
Mike Drob commented on HBASE-19258:
---
{code}
+ private synchronized String backup(BackupRequest request, BackupAdmin client)
+ private synchronized void restore(RestoreRequest request, BackupAdmin client)
+ private synchronized void merge(String[] backupIds, BackupAdmin client)
{code}
I wish we didn't have to synchronize on these... the expectation for clients is
that they will retry if there is an operation in progress already I guess?
> IntegrationTest for Backup and Restore
> --
>
> Key: HBASE-19258
> URL: https://issues.apache.org/jira/browse/HBASE-19258
> Project: HBase
> Issue Type: Test
> Components: integration tests
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HBASE-19258.v1.patch
>
>
> See chatter at https://docs.google.com/document/d/1xbPlLKjOcPq2LDqjbSkF6uND
> AG0mzgOxek6P3POLeMc/edit?usp=sharing
> We need to get an IntegrationTest in place for backup and restore.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19258) IntegrationTest for Backup and Restore
[
https://issues.apache.org/jira/browse/HBASE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408870#comment-16408870
]
Mike Drob commented on HBASE-19258:
---
{code}
+if (plugins != null) {
+ conf.set(HFileCleaner.MASTER_HFILE_CLEANER_PLUGINS, (plugins == null ?
"" : plugins + ",") +
+BackupHFileCleaner.class.getName());
+}
{code}
redundant null checks
{code}
LOG.debug("Added log cleaner: " + cleanerClass + "\n" + "Added master
procedure manager: "
- + masterProcedureClass);
+ + masterProcedureClass+"\n"+ "Added HFile cleaner: "+
BackupHFileCleaner.class.getName());
+
{code}
nit: parameterized logging
{code}
+import com.google.common.util.concurrent.Uninterruptibles;
{code}
should be thirdparty?
I see a few things that look like they will be checkstyle issues, I won't list
them because I assume QA will find them.
{code}
+LOG.info(String.format("Creating table %s with %d splits.", tableName,
regionsCountPerServer));
-LOG.info(String.format("Pre-split table created successfully in %dms.",
- (endTime - startTime)));
+LOG.info(String.format("Pre-split table created successfully in %dms.",
(endTime - startTime)));
{code}
parameterized logging again, please. There's more places later, I'm not going
to list each individually
{code}
+ LOG.info("create full backup image for ");
{code}
Missing table name?
{code}
+rowsInIteration =
+Integer.parseInt(cmd.getOptionValue(ROWS_PER_ITERATION_KEY,
+ Integer.toString(DEFAULT_ROWS_IN_ITERATION)));
{code}
This is super gross but I don't know if there's a better way
> IntegrationTest for Backup and Restore
> --
>
> Key: HBASE-19258
> URL: https://issues.apache.org/jira/browse/HBASE-19258
> Project: HBase
> Issue Type: Test
> Components: integration tests
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HBASE-19258.v1.patch
>
>
> See chatter at https://docs.google.com/document/d/1xbPlLKjOcPq2LDqjbSkF6uND
> AG0mzgOxek6P3POLeMc/edit?usp=sharing
> We need to get an IntegrationTest in place for backup and restore.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20242) The open sequence number will grow if we fail to open a region after writing the max sequence id file
[ https://issues.apache.org/jira/browse/HBASE-20242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20242: -- Resolution: Fixed Hadoop Flags: Reviewed Release Note: Now when opening a region, we will store the current max sequence id of the region to its max sequence id file instead of the 'next sequence id'. This could avoid the sequence id bumping when we fail to open a region, and also align to the behavior when we close a region. Status: Resolved (was: Patch Available) Pushed to master. Thanks [~openinx] for reviewing. > The open sequence number will grow if we fail to open a region after writing > the max sequence id file > - > > Key: HBASE-20242 > URL: https://issues.apache.org/jira/browse/HBASE-20242 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20242.patch > > > We write the current max sequence id plus one as the new max sequence id so > if we fail before report RIT to master, when assign next time the open > sequence number will increase by one. > But it is possible that we fail before writing the open region marker so > there will no actual WAL with the sequence id which is the open sequence > number minus one. This will cause the serial replication to be stuck. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19258) IntegrationTest for Backup and Restore
[ https://issues.apache.org/jira/browse/HBASE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408853#comment-16408853 ] Vladimir Rodionov commented on HBASE-19258: --- I have run this test several times successfully with ChaosMonkey enabled (random RS restarts). It loads in batches data (50M rows), creates several full and incremental backups, runs periodically merges and restores and verifies data. cc: [~te...@apache.org], [~elserj] > IntegrationTest for Backup and Restore > -- > > Key: HBASE-19258 > URL: https://issues.apache.org/jira/browse/HBASE-19258 > Project: HBase > Issue Type: Test > Components: integration tests >Reporter: Josh Elser >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: 3.0.0 > > Attachments: HBASE-19258.v1.patch > > > See chatter at https://docs.google.com/document/d/1xbPlLKjOcPq2LDqjbSkF6uND > AG0mzgOxek6P3POLeMc/edit?usp=sharing > We need to get an IntegrationTest in place for backup and restore. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19258) IntegrationTest for Backup and Restore
[ https://issues.apache.org/jira/browse/HBASE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-19258: -- Status: Patch Available (was: Open) > IntegrationTest for Backup and Restore > -- > > Key: HBASE-19258 > URL: https://issues.apache.org/jira/browse/HBASE-19258 > Project: HBase > Issue Type: Test > Components: integration tests >Reporter: Josh Elser >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: 3.0.0 > > Attachments: HBASE-19258.v1.patch > > > See chatter at https://docs.google.com/document/d/1xbPlLKjOcPq2LDqjbSkF6uND > AG0mzgOxek6P3POLeMc/edit?usp=sharing > We need to get an IntegrationTest in place for backup and restore. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19258) IntegrationTest for Backup and Restore
[ https://issues.apache.org/jira/browse/HBASE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-19258: -- Attachment: HBASE-19258.v1.patch > IntegrationTest for Backup and Restore > -- > > Key: HBASE-19258 > URL: https://issues.apache.org/jira/browse/HBASE-19258 > Project: HBase > Issue Type: Test > Components: integration tests >Reporter: Josh Elser >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: 3.0.0 > > Attachments: HBASE-19258.v1.patch > > > See chatter at https://docs.google.com/document/d/1xbPlLKjOcPq2LDqjbSkF6uND > AG0mzgOxek6P3POLeMc/edit?usp=sharing > We need to get an IntegrationTest in place for backup and restore. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20212) Make all Public classes have InterfaceAudience category
[
https://issues.apache.org/jira/browse/HBASE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408839#comment-16408839
]
Hadoop QA commented on HBASE-20212:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
36s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 12m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-annotations hbase-spark-it . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 13m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 6m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 6m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} The patch hbase-annotations passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} The patch hbase-protocol-shaded passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} hbase-common: The patch generated 0 new + 12
unchanged - 2 fixed = 12 total (was 14) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} The patch hbase-metrics-api passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch hbase-hadoop-compat passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} The patch hbase-metrics passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} hbase-hadoop2-compat: The patch generated 0 new + 6
unchanged - 3 fixed = 6 total (was 9) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} The patch hbase-protocol passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} The patch hbase-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch hbase-zookeeper passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch hbase-replication passed checkstyle
{color} |
|
[jira] [Updated] (HBASE-20116) Optimize the region last pushed sequence id layout on zk
[ https://issues.apache.org/jira/browse/HBASE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20116: -- Status: Patch Available (was: Reopened) Fix the javadoc. And also done a simple optimization to remove the leading 4 chars in the final znode name since they are already presented as the directory names. > Optimize the region last pushed sequence id layout on zk > > > Key: HBASE-20116 > URL: https://issues.apache.org/jira/browse/HBASE-20116 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20116-addendum.patch, HBASE-20116.v1.patch, > HBASE-20116.v2.patch, HBASE-20116.v3.patch, HBASE-20116.v4.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20116) Optimize the region last pushed sequence id layout on zk
[ https://issues.apache.org/jira/browse/HBASE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20116: -- Attachment: HBASE-20116-addendum.patch > Optimize the region last pushed sequence id layout on zk > > > Key: HBASE-20116 > URL: https://issues.apache.org/jira/browse/HBASE-20116 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20116-addendum.patch, HBASE-20116.v1.patch, > HBASE-20116.v2.patch, HBASE-20116.v3.patch, HBASE-20116.v4.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20116) Optimize the region last pushed sequence id layout on zk
[ https://issues.apache.org/jira/browse/HBASE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20116: -- Fix Version/s: (was: 2.1.0) 3.0.0 > Optimize the region last pushed sequence id layout on zk > > > Key: HBASE-20116 > URL: https://issues.apache.org/jira/browse/HBASE-20116 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20116.v1.patch, HBASE-20116.v2.patch, > HBASE-20116.v3.patch, HBASE-20116.v4.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-20116) Optimize the region last pushed sequence id layout on zk
[ https://issues.apache.org/jira/browse/HBASE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-20116: --- > Optimize the region last pushed sequence id layout on zk > > > Key: HBASE-20116 > URL: https://issues.apache.org/jira/browse/HBASE-20116 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Zheng Hu >Priority: Major > Fix For: 2.1.0 > > Attachments: HBASE-20116.v1.patch, HBASE-20116.v2.patch, > HBASE-20116.v3.patch, HBASE-20116.v4.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18828) [2.0] Generate CHANGES.txt
[ https://issues.apache.org/jira/browse/HBASE-18828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408783#comment-16408783 ] stack commented on HBASE-18828: --- Just messed around with HBASE-14175 "Adopt releasedocmaker for better generated release notes". Also took a look again at notes in HBASE-14025. Let me play more w/ releasedocmaker. Need to minimally follow the suggested cleanup in HBASE-14025. Meantime, toward the goal of generating clean releasenotes/changes, I just did a first pass over JIRA where I did an mass-edit of fixedVersions in-line with the thinking expressed in HBASE-14025. * High-level I added 2.0.0 to issues that had only 2.0.0-beta-1 or 2.0.0-alpha-2, etc., as fixVersion so now they have 2.0.0 AND 2.0.0-beta-2. This way I can pull 2.0.0 from JIRA and it will include fixes in all alphas and betas too. * No JIRAs had 2.0.0-alpha-1 as fixVersion. Issues had 2.0.0 as their fix version at time of the 2.0.0-alpha-1 release. * Any 2.0.0 issue that also had 3.0.0 as fixVersion had the 3.0.0 removed (in the HBASE-14025 spirit. I was able to successfully generate 2.0.0 'release notes' from JIRA though there are thousands. TODO: Get HBASE-14025 notions into refguide. TODO: See if place for releasedocmaker. > [2.0] Generate CHANGES.txt > -- > > Key: HBASE-18828 > URL: https://issues.apache.org/jira/browse/HBASE-18828 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > > See [~busbey] writeup on generating CHANGES.txt here HBASE-14025 (fold it > into refguide while at it..) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20224) Web UI is broken in standalone mode
[ https://issues.apache.org/jira/browse/HBASE-20224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408765#comment-16408765 ] Umesh Agashe commented on HBASE-20224: -- Cahnges per review comment on JIRA. Added the property 'hbase.localcluster.assign.random.ports' with value 'true' in all test/resources/hbase-site.xml. > Web UI is broken in standalone mode > --- > > Key: HBASE-20224 > URL: https://issues.apache.org/jira/browse/HBASE-20224 > Project: HBase > Issue Type: Bug > Components: UI, Usability >Affects Versions: 2.0.0-beta-2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > Attachments: hbase-20224.master.001.patch, > hbase-20224.master.002.patch > > > Web UI doesn't show up in standalone mode on default port. This can be seen > on master and branch-2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20224) Web UI is broken in standalone mode
[ https://issues.apache.org/jira/browse/HBASE-20224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-20224: - Attachment: hbase-20224.master.002.patch > Web UI is broken in standalone mode > --- > > Key: HBASE-20224 > URL: https://issues.apache.org/jira/browse/HBASE-20224 > Project: HBase > Issue Type: Bug > Components: UI, Usability >Affects Versions: 2.0.0-beta-2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > Attachments: hbase-20224.master.001.patch, > hbase-20224.master.002.patch > > > Web UI doesn't show up in standalone mode on default port. This can be seen > on master and branch-2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20111) Able to split region explicitly even on shouldSplit return false from split policy
[ https://issues.apache.org/jira/browse/HBASE-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408739#comment-16408739 ] Josh Elser commented on HBASE-20111: [~rajeshbabu], still on your radar? > Able to split region explicitly even on shouldSplit return false from split > policy > -- > > Key: HBASE-20111 > URL: https://issues.apache.org/jira/browse/HBASE-20111 > Project: HBase > Issue Type: Bug >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-20111.001.branch-2.0.patch, HBASE-20111.patch, > HBASE-20111_test.patch > > > Currently able to split the region explicitly even when the split policy > returns from shouldSplit. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-18408) AM consumes CPU and fills up the logs really fast when there is no RS to assign
[
https://issues.apache.org/jira/browse/HBASE-18408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-18408.
---
Resolution: Later
Fix Version/s: (was: 2.0.0)
I've not seen this. What is reported here is admittedly from a very old master.
The high-level takeaway I think is that if things go wrong in AM, then it can
spew loads of log that is probably still the case; we just go wrong less
often now.
Let me resolve this as later. Lets open new issue w/ updated complaint if we
run into this again.
Thanks [~elserj]
> AM consumes CPU and fills up the logs really fast when there is no RS to
> assign
> ---
>
> Key: HBASE-18408
> URL: https://issues.apache.org/jira/browse/HBASE-18408
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Priority: Critical
>
> I was testing something else when I discovered that when there is no RS to
> assign a region to (but master is alive), then AM/LB creates GB's of logs.
> Logs like this:
> {code}
> 2017-07-18 16:40:00,712 WARN [AssignmentThread] balancer.BaseLoadBalancer:
> Wanted to do round robin assignment but no servers to assign to
> 2017-07-18 16:40:00,712 WARN [AssignmentThread]
> assignment.AssignmentManager: unable to round-robin assignment
> org.apache.hadoop.hbase.HBaseIOException: unable to compute plans for
> regions=1
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.acceptPlan(AssignmentManager.java:1725)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:1711)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$300(AssignmentManager.java:108)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager$2.run(AssignmentManager.java:1587)
> 2017-07-18 16:40:00,865 WARN [AssignmentThread] balancer.BaseLoadBalancer:
> Wanted to do round robin assignment but no servers to assign to
> 2017-07-18 16:40:00,866 WARN [AssignmentThread]
> assignment.AssignmentManager: unable to round-robin assignment
> org.apache.hadoop.hbase.HBaseIOException: unable to compute plans for
> regions=1
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.acceptPlan(AssignmentManager.java:1725)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:1711)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$300(AssignmentManager.java:108)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager$2.run(AssignmentManager.java:1587)
> 2017-07-18 16:40:01,019 WARN [AssignmentThread] balancer.BaseLoadBalancer:
> Wanted to do round robin assignment but no servers to assign to
> 2017-07-18 16:40:01,019 WARN [AssignmentThread]
> assignment.AssignmentManager: unable to round-robin assignment
> org.apache.hadoop.hbase.HBaseIOException: unable to compute plans for
> regions=1
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.acceptPlan(AssignmentManager.java:1725)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:1711)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$300(AssignmentManager.java:108)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager$2.run(AssignmentManager.java:1587)
> 2017-07-18 16:40:01,173 WARN [AssignmentThread] balancer.BaseLoadBalancer:
> Wanted to do round robin assignment but no servers to assign to
> 2017-07-18 16:40:01,173 WARN [AssignmentThread]
> assignment.AssignmentManager: unable to round-robin assignment
> org.apache.hadoop.hbase.HBaseIOException: unable to compute plans for
> regions=1
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.acceptPlan(AssignmentManager.java:1725)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:1711)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$300(AssignmentManager.java:108)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager$2.run(AssignmentManager.java:1587)
> {code}
> Reproduction is easy:
> - Start pseudo-distributed cluster
> - Create a table
> - kill region server
> I have also noticed that we are just spinning CPU in another case consuming
> 100-200% (but this is in a very old code base from master) in this cycle:
> {code}
> "ProcedureExecutor-0" #106 daemon prio=5 os_prio=0 tid=0x7fab54851800
> nid=0xcf1 runnable [0x7fab4e7b]
>java.lang.Thread.State: RUNNABLE
> at java.lang.Object.hashCode(Native Method)
> at
>
[jira] [Commented] (HBASE-20247) Set version as 2.0.0 in branch-2.0 in prep for first RC
[
https://issues.apache.org/jira/browse/HBASE-20247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408728#comment-16408728
]
Hudson commented on HBASE-20247:
Results for branch branch-2.0
[build #69 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/69/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/69//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/69//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/69//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Set version as 2.0.0 in branch-2.0 in prep for first RC
> ---
>
> Key: HBASE-20247
> URL: https://issues.apache.org/jira/browse/HBASE-20247
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20090) Properly handle Preconditions check failure in MemStoreFlusher$FlushHandler.run
[
https://issues.apache.org/jira/browse/HBASE-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408727#comment-16408727
]
Hudson commented on HBASE-20090:
Results for branch branch-2.0
[build #69 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/69/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/69//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/69//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/69//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Properly handle Preconditions check failure in
> MemStoreFlusher$FlushHandler.run
> ---
>
> Key: HBASE-20090
> URL: https://issues.apache.org/jira/browse/HBASE-20090
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: 20090-server-61260-01-07.log, 20090.v10.txt,
> 20090.v10.txt, 20090.v6.txt, 20090.v7.txt, 20090.v8.txt, 20090.v9.txt
>
>
> Copied the following from a comment since this was better description of the
> race condition.
> The original description was merged to the beginning of my first comment
> below.
> Observed the following in region server log (running on hadoop3 cluster):
> {code}
> 2018-02-26 16:06:50,044 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=16020]
> regionserver.MemStoreFlusher: Memstore is above high water mark and block
> 135ms
> 2018-02-26 16:06:50,049 ERROR [MemStoreFlusher.1]
> regionserver.MemStoreFlusher: Cache flusher failed for entry
> org.apache.hadoop.hbase.regionserver.
> MemStoreFlusher$WakeupFlushThread@2adfadd7
> java.lang.IllegalStateException
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkState(Preconditions.java:441)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:174)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$600(MemStoreFlusher.java:69)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:237)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Here is the Precondition from MemStoreFlusher#flushOneForGlobalPressure() :
> {code}
> Preconditions.checkState(
> (regionToFlush != null && regionToFlushSize > 0) ||
> bestRegionReplicaSize > 0);
> {code}
> When the Preconditions check fails, IllegalStateException would be raised.
> With more debug logging, we can see the scenario where the exception was
> triggered.
> {code}
> 2018-03-02 17:28:30,097 DEBUG [MemStoreFlusher.0] regionserver.CompactSplit:
> Splitting TestTable,,1520011528142.0453f29030757eedb6e6a1c57e88c085.,
> compaction_queue=(0:0), split_queue=1
> 2018-03-02 17:28:30,098 DEBUG
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020]
> regionserver.IncreasingToUpperBoundRegionSplitPolicy: ShouldSplit because
> info size=6.9G, sizeToCheck=256.0M, regionsWithCommonTable=1
> 2018-03-02 17:28:30,296 INFO
> [RpcServer.default.FPBQ.Fifo.handler=24,queue=0,port=16020]
> regionserver.MemStoreFlusher: wake up flusher due to ABOVE_ONHEAP_LOWER_MARK
> 2018-03-02 17:28:30,297 DEBUG [MemStoreFlusher.1]
> regionserver.MemStoreFlusher: Flush thread woke up because memory above low
> water=381.5 M
> 2018-03-02 17:28:30,297 INFO
> [RpcServer.default.FPBQ.Fifo.handler=25,queue=1,port=16020]
> regionserver.MemStoreFlusher: wake up flusher due to ABOVE_ONHEAP_LOWER_MARK
> 2018-03-02 17:28:30,298 DEBUG [MemStoreFlusher.1]
> regionserver.MemStoreFlusher: region
> TestTable,,1520011528142.0453f29030757eedb6e6a1c57e88c085. with size 400432696
> 2018-03-02 17:28:30,298 DEBUG [MemStoreFlusher.1]
> regionserver.MemStoreFlusher: region
> atlas_janus,,1519927429371.fbcb5e495344542daf8b499e4bac03ae. with size 0
> 2018-03-02 17:28:30,298 INFO [MemStoreFlusher.1]
> regionserver.MemStoreFlusher: Flush of region
> atlas_janus,,1519927429371.fbcb5e495344542daf8b499e4bac03ae. due to global
> heap pressure. Flush type=ABOVE_ONHEAP_LOWER_MARKTotal Memstore Heap
> size=381.9 MTotal Memstore Off-Heap size=0, Region memstore size=0
> 2018-03-02 17:28:30,298 INFO [MemStoreFlusher.1]
>
[jira] [Commented] (HBASE-14175) Adopt releasedocmaker for better generated release notes
[
https://issues.apache.org/jira/browse/HBASE-14175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408729#comment-16408729
]
stack commented on HBASE-14175:
---
Updated my yetus and then did this: {{yetus.git stack$
./release-doc-maker/releasedocmaker.py -p HBASE --version 2.0.0
--dirversions}} I now have lovely .md release notes and changes.
How should I integrate them? Currently we do not ship a release notes and our
Changes is a .txt file currently last updated for 0.92 in branch-2. Over in
branch-1, I see that 1.2.7 has a nice text changes that goes back to 1.0.0.
{{yetus.git stack$ ./release-doc-maker/releasedocmaker.py -p HBASE --range
--version 1.0.0 --version 2.0.0 --dirversions}}
will dump out files per version. I can collapse the whole lot and edit to make
a new CHANGES.txt file for branch-2. Could also check in these release notes
and the markdown (checking in release notes might encourage devs to do a better
job writing them).
> Adopt releasedocmaker for better generated release notes
>
>
> Key: HBASE-14175
> URL: https://issues.apache.org/jira/browse/HBASE-14175
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Priority: Critical
> Fix For: 2.0.0
>
>
> We should consider adopting Hadoop's releasedocmaker for better release
> notes. This would pull out text from the JIRA 'release notes' field with
> clean presentation and is vastly superior to our current notes, which are
> simply JIRA's list of issues by fix version. Could hook it into the site
> build. A convenient part of Yetus to get up and running with.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18408) AM consumes CPU and fills up the logs really fast when there is no RS to assign
[
https://issues.apache.org/jira/browse/HBASE-18408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408693#comment-16408693
]
Josh Elser commented on HBASE-18408:
[~stack], you haven't noticed this recently in your testing? Odds that it's
still a problem after your recent work with Duo?
> AM consumes CPU and fills up the logs really fast when there is no RS to
> assign
> ---
>
> Key: HBASE-18408
> URL: https://issues.apache.org/jira/browse/HBASE-18408
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Priority: Critical
> Fix For: 2.0.0
>
>
> I was testing something else when I discovered that when there is no RS to
> assign a region to (but master is alive), then AM/LB creates GB's of logs.
> Logs like this:
> {code}
> 2017-07-18 16:40:00,712 WARN [AssignmentThread] balancer.BaseLoadBalancer:
> Wanted to do round robin assignment but no servers to assign to
> 2017-07-18 16:40:00,712 WARN [AssignmentThread]
> assignment.AssignmentManager: unable to round-robin assignment
> org.apache.hadoop.hbase.HBaseIOException: unable to compute plans for
> regions=1
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.acceptPlan(AssignmentManager.java:1725)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:1711)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$300(AssignmentManager.java:108)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager$2.run(AssignmentManager.java:1587)
> 2017-07-18 16:40:00,865 WARN [AssignmentThread] balancer.BaseLoadBalancer:
> Wanted to do round robin assignment but no servers to assign to
> 2017-07-18 16:40:00,866 WARN [AssignmentThread]
> assignment.AssignmentManager: unable to round-robin assignment
> org.apache.hadoop.hbase.HBaseIOException: unable to compute plans for
> regions=1
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.acceptPlan(AssignmentManager.java:1725)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:1711)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$300(AssignmentManager.java:108)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager$2.run(AssignmentManager.java:1587)
> 2017-07-18 16:40:01,019 WARN [AssignmentThread] balancer.BaseLoadBalancer:
> Wanted to do round robin assignment but no servers to assign to
> 2017-07-18 16:40:01,019 WARN [AssignmentThread]
> assignment.AssignmentManager: unable to round-robin assignment
> org.apache.hadoop.hbase.HBaseIOException: unable to compute plans for
> regions=1
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.acceptPlan(AssignmentManager.java:1725)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:1711)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$300(AssignmentManager.java:108)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager$2.run(AssignmentManager.java:1587)
> 2017-07-18 16:40:01,173 WARN [AssignmentThread] balancer.BaseLoadBalancer:
> Wanted to do round robin assignment but no servers to assign to
> 2017-07-18 16:40:01,173 WARN [AssignmentThread]
> assignment.AssignmentManager: unable to round-robin assignment
> org.apache.hadoop.hbase.HBaseIOException: unable to compute plans for
> regions=1
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.acceptPlan(AssignmentManager.java:1725)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:1711)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$300(AssignmentManager.java:108)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager$2.run(AssignmentManager.java:1587)
> {code}
> Reproduction is easy:
> - Start pseudo-distributed cluster
> - Create a table
> - kill region server
> I have also noticed that we are just spinning CPU in another case consuming
> 100-200% (but this is in a very old code base from master) in this cycle:
> {code}
> "ProcedureExecutor-0" #106 daemon prio=5 os_prio=0 tid=0x7fab54851800
> nid=0xcf1 runnable [0x7fab4e7b]
>java.lang.Thread.State: RUNNABLE
> at java.lang.Object.hashCode(Native Method)
> at
> java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1106)
> at
> java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1097)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.close(HRegion.java:6158)
> - locked
[jira] [Updated] (HBASE-16993) BucketCache throw java.io.IOException: Invalid HFile block magic when configuring hbase.bucketcache.bucket.sizes
[
https://issues.apache.org/jira/browse/HBASE-16993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16993:
--
Fix Version/s: (was: 3.0.0)
> BucketCache throw java.io.IOException: Invalid HFile block magic when
> configuring hbase.bucketcache.bucket.sizes
>
>
> Key: HBASE-16993
> URL: https://issues.apache.org/jira/browse/HBASE-16993
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 1.1.3
> Environment: hbase version 1.1.3
>Reporter: liubangchen
>Assignee: Anoop Sam John
>Priority: Major
> Fix For: 1.4.0, 2.0.0-alpha-2, 2.0.0
>
> Attachments: HBASE-16993.000.patch, HBASE-16993.001.patch,
> HBASE-16993.master.001.patch, HBASE-16993.master.002.patch,
> HBASE-16993.master.003.patch, HBASE-16993.master.004.patch,
> HBASE-16993.master.005.patch, HBASE-16993_V2.patch, HBASE-16993_V6.patch,
> HBASE-16993_branch-1.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> hbase-site.xml setting
>
> hbase.bucketcache.bucket.sizes
> 16384,32768,40960,
> 46000,49152,51200,65536,131072,524288
>
>
> hbase.bucketcache.size
> 16384
>
>
> hbase.bucketcache.ioengine
> offheap
>
>
> hfile.block.cache.size
> 0.3
>
>
> hfile.block.bloom.cacheonwrite
> true
>
>
> hbase.rs.cacheblocksonwrite
> true
>
>
> hfile.block.index.cacheonwrite
> true
> n_splits = 200
> create 'usertable',{NAME =>'family', COMPRESSION => 'snappy', VERSIONS =>
> 1,DATA_BLOCK_ENCODING => 'DIFF',CONFIGURATION =>
> {'hbase.hregion.memstore.block.multiplier' => 5}},{DURABILITY =>
> 'SKIP_WAL'},{SPLITS => (1..n_splits).map {|i|
> "user#{1000+i*(-1000)/n_splits}"}}
> load data
> bin/ycsb load hbase10 -P workloads/workloada -p table=usertable -p
> columnfamily=family -p fieldcount=10 -p fieldlength=100 -p
> recordcount=2 -p insertorder=hashed -p insertstart=0 -p
> clientbuffering=true -p durability=SKIP_WAL -threads 20 -s
> run
> bin/ycsb run hbase10 -P workloads/workloadb -p table=usertable -p
> columnfamily=family -p fieldcount=10 -p fieldlength=100 -p
> operationcount=2000 -p readallfields=true -p clientbuffering=true -p
> requestdistribution=zipfian -threads 10 -s
> log info
> 2016-11-02 20:20:20,261 ERROR
> [RW.default.readRpcServer.handler=36,queue=21,port=6020] bucket.BucketCache:
> Failed reading block fdcc7ed6f3b2498b9ef316cc8206c233_44819759 from bucket
> cache
> java.io.IOException: Invalid HFile block magic:
> \x00\x00\x00\x00\x00\x00\x00\x00
> at
> org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:154)
> at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:167)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock.(HFileBlock.java:273)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$1.deserialize(HFileBlock.java:134)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$1.deserialize(HFileBlock.java:121)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getBlock(BucketCache.java:427)
> at
> org.apache.hadoop.hbase.io.hfile.CombinedBlockCache.getBlock(CombinedBlockCache.java:85)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.getCachedBlock(HFileReaderV2.java:266)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:403)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:269)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:634)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:584)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:247)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:156)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:363)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:217)
> at
> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2071)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:5369)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2546)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2532)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2514)
> at
[jira] [Updated] (HBASE-18312) Ineffective handling of FileNotFoundException in FileLink$FileLinkInputStream.tryOpen()
[
https://issues.apache.org/jira/browse/HBASE-18312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18312:
--
Fix Version/s: (was: 3.0.0)
> Ineffective handling of FileNotFoundException in
> FileLink$FileLinkInputStream.tryOpen()
> ---
>
> Key: HBASE-18312
> URL: https://issues.apache.org/jira/browse/HBASE-18312
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 1.5.0, 2.0.0-alpha-2, 2.0.0
>
> Attachments: 18312.v1.txt, 18312.v2.txt, 18312.v4.txt, 18312.v5.txt,
> 18312.v6.txt
>
>
> Found the following in region server log:
> {code}
> 2017-07-03 11:22:04,669 WARN
> [regionserver/a.b.c.d:16020-shortCompactions-1499094046361]
> retry.RetryInvocationHandler: Exception while invoking
> ClientNamenodeProtocolTranslatorPB.getBlockLocations over e.f.g.h:8020. Not
> retrying because try once and fail.
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): File
> does not exist: /hbase/data/default/X/4d61af9d1cbcc5fe2a5cbddbfc92fe7e/
> K/47222a9cbd294f499f49de92ecf330ee
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:71)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:61)
> ...
> at
> org.apache.hadoop.hbase.io.FileLink$FileLinkInputStream.tryOpen(FileLink.java:291)
> at
> org.apache.hadoop.hbase.io.FileLink$FileLinkInputStream.(FileLink.java:122)
> at
> org.apache.hadoop.hbase.io.FileLink$FileLinkInputStream.(FileLink.java:113)
> at org.apache.hadoop.hbase.io.FileLink.open(FileLink.java:404)
> at
> org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.(FSDataInputStreamWrapper.java:98)
> at
> org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.(FSDataInputStreamWrapper.java:83)
> {code}
> Here is related code:
> {code}
> private FSDataInputStream tryOpen() throws IOException {
> for (Path path: fileLink.getLocations()) {
> ...
> } catch (FileNotFoundException e) {
> // Try another file location
> }
> {code}
> The intention is to try possible locations for the linked file.
> However, RemoteException was the exception encountered. This makes the above
> catch clause ineffective.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18288) Declared dependency on specific javax.ws.rs
[ https://issues.apache.org/jira/browse/HBASE-18288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18288: -- Fix Version/s: (was: 3.0.0) > Declared dependency on specific javax.ws.rs > --- > > Key: HBASE-18288 > URL: https://issues.apache.org/jira/browse/HBASE-18288 > Project: HBase > Issue Type: Task > Components: dependencies, REST >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 2.0.0-alpha-2, 2.0.0 > > Attachments: HBASE-18288.0.patch > > > We make use of the javax.ws.rs API, but we rely on getting a jar for it > transitively. since we use the classes, we should declare an explicit > dependency. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16585) Rewrite the delegation token tests with Parameterized pattern
[ https://issues.apache.org/jira/browse/HBASE-16585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16585: -- Fix Version/s: (was: 3.0.0) > Rewrite the delegation token tests with Parameterized pattern > - > > Key: HBASE-16585 > URL: https://issues.apache.org/jira/browse/HBASE-16585 > Project: HBase > Issue Type: Improvement > Components: security, test >Affects Versions: 3.0.0, 1.4.0, 2.0.0-alpha-1 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 1.4.0, 2.0.0-alpha-2, 2.0.0 > > Attachments: HBASE-16585-branch-1-v1.patch, > HBASE-16585-branch-1.patch, HBASE-16585-v1.patch, HBASE-16585.patch > > > TestDelegationTokenWithEncryption and TestGenerateDelegationToken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18212) In Standalone mode with local filesystem HBase logs Warning message:Failed to invoke 'unbuffer' method in class class org.apache.hadoop.fs.FSDataInputStream
[
https://issues.apache.org/jira/browse/HBASE-18212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18212:
--
Fix Version/s: (was: 3.0.0)
> In Standalone mode with local filesystem HBase logs Warning message:Failed to
> invoke 'unbuffer' method in class class org.apache.hadoop.fs.FSDataInputStream
>
>
> Key: HBASE-18212
> URL: https://issues.apache.org/jira/browse/HBASE-18212
> Project: HBase
> Issue Type: Bug
> Components: Operability
>Affects Versions: 1.4.0, 2.0.0
>Reporter: Umesh Agashe
>Assignee: Ashish Singhi
>Priority: Minor
> Fix For: 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12, 2.0.0
>
> Attachments: HBASE-18212.patch
>
>
> New users may get nervous after seeing following warning level log messages
> (considering new users will most likely run HBase in Standalone mode first):
> {code}
> WARN [MemStoreFlusher.1] io.FSDataInputStreamWrapper: Failed to invoke
> 'unbuffer' method in class class org.apache.hadoop.fs.FSDataInputStream . So
> there may be a TCP socket connection left open in CLOSE_WAIT state.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18527) update nightly builds to compensate for jenkins plugin upgrades
[ https://issues.apache.org/jira/browse/HBASE-18527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18527: -- Fix Version/s: (was: 3.0.0) > update nightly builds to compensate for jenkins plugin upgrades > --- > > Key: HBASE-18527 > URL: https://issues.apache.org/jira/browse/HBASE-18527 > Project: HBase > Issue Type: Task > Components: community, test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12, 2.0.0 > > Attachments: HBASE-18527.0.patch > > > Last night as a part of some infra work a bunch of plugins updated. one of > them was the git-branch-source thingy. Its upgrade stopped reusing stuff from > the general plugin for checking out git stuff, so our checking out into a > directory stopped working. > Tracked upstream as > [JENKINS-46013|https://issues.jenkins-ci.org/browse/JENKINS-46013]. Until we > get a fix we need a workaround. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18426) nightly job should use independent stages to check supported jdks
[ https://issues.apache.org/jira/browse/HBASE-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18426: -- Fix Version/s: (was: 3.0.0) > nightly job should use independent stages to check supported jdks > - > > Key: HBASE-18426 > URL: https://issues.apache.org/jira/browse/HBASE-18426 > Project: HBase > Issue Type: Improvement > Components: community, test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12, 2.0.0 > > Attachments: HBASE-18426.0.patch > > > follow on from HBASE-18147 to handle the lack of multijdk support for unit > tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18041: -- Fix Version/s: (was: 3.0.0) > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang >Assignee: Alex Leblang >Priority: Major > Fix For: 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12, 2.0.0 > > Attachments: HBASE-18041.branch-1.2.001.patch, > diff-patch-pylint-noRC.txt, diff-patch-pylint.txt > > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
[ https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18502: -- Fix Version/s: (was: 3.0.0) > Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor > --- > > Key: HBASE-18502 > URL: https://issues.apache.org/jira/browse/HBASE-18502 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, master >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 2.0.0-alpha-2, 2.0.0 > > Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, > HBASE-18502.v1.patch, HBASE-18502.v2.patch > > > MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 > and 2.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18555) Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and Query
[
https://issues.apache.org/jira/browse/HBASE-18555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18555:
--
Fix Version/s: (was: 3.0.0)
> Remove redundant familyMap.put() from addxxx() of sub-classes of Mutation and
> Query
> ---
>
> Key: HBASE-18555
> URL: https://issues.apache.org/jira/browse/HBASE-18555
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 1.4.0, 2.0.0-alpha-2, 2.0.0
>
> Attachments: HBASE-18555.master.000.patch,
> HBASE-18555.master.001.patch, HBASE-18555.master.002.patch
>
>
> In addxxx() functions of Mutation(Append, Delete, Increment and Put) and
> Query(Get and Scan), there are redundant Map#put() calls which could be
> removed to improve the performance.
> For example, in Put#addColumn() and addImmutable(), after getting the cell
> list of the given family and add the cell into the list, the code puts
> (key=family, value=list) into familyMap.
> In addColumn(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value);
> list.add(kv);
> familyMap.put(CellUtil.cloneFamily(kv), list); // <-- here
> return this;
> {code}
> In addImmutable(), it is like
> {code}
> List list = getCellList(family);
> KeyValue kv = createPutKeyValue(family, qualifier, ts, value, tag);
> list.add(kv);
> familyMap.put(family, list); // <-- here
> return this;
> {code}
> I think those put() for Map only take effect when getCellList(family) returns
> a new allocated ArrayList. When the list for a family already exist, put()
> for Map will update the value to the reference of the list, but actually, the
> reference of the list is not changed.
> Those put() do not do any harm in terms of the correctness when they are
> here. But it could be removed to improve the performance. familyMap searches
> for key and set the new value and return the old value. Those operation take
> some time but actually are not needed.
> The put() could be moved into Mutation#getCellList(family)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18412) [Shell] Support unset of list of configuration for a table
[ https://issues.apache.org/jira/browse/HBASE-18412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18412: -- Fix Version/s: (was: 3.0.0) > [Shell] Support unset of list of configuration for a table > -- > > Key: HBASE-18412 > URL: https://issues.apache.org/jira/browse/HBASE-18412 > Project: HBase > Issue Type: Improvement >Reporter: Yun Zhao >Assignee: Yun Zhao >Priority: Minor > Fix For: 2.0.0-alpha-2, 2.0.0 > > Attachments: HBASE-18412.master.001.patch > > > Add a method 'table_conf_unset' in admin.rb to reset the configuration as the > system default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17922) TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17922:
--
Fix Version/s: (was: 3.0.0)
> TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
> -
>
> Key: HBASE-17922
> URL: https://issues.apache.org/jira/browse/HBASE-17922
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0-alpha-2, 2.0.0
>
> Attachments: HBASE-17922.patch
>
>
> {code}
> Running org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 126.363 sec
> <<< FAILURE! - in
> org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
> testRegionServerHostname(org.apache.hadoop.hbase.regionserver.TestRegionServerHostname)
> Time elapsed: 120.029 sec <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 12
> milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:221)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:405)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:225)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1123)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1077)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:948)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:942)
> at
> org.apache.hadoop.hbase.regionserver.TestRegionServerHostname.testRegionServerHostname(TestRegionServerHostname.java:88)
> Results :
> Tests in error:
> TestRegionServerHostname.testRegionServerHostname:88 » TestTimedOut test
> timed...
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18310) LoadTestTool unable to write data
[
https://issues.apache.org/jira/browse/HBASE-18310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18310:
--
Fix Version/s: (was: 3.0.0)
> LoadTestTool unable to write data
> -
>
> Key: HBASE-18310
> URL: https://issues.apache.org/jira/browse/HBASE-18310
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Samir Ahmic
>Assignee: Samir Ahmic
>Priority: Major
> Fix For: 2.0.0-alpha-2, 2.0.0
>
> Attachments: HBASE-18310-master-01.patch
>
>
> I was trying to write some data to cluster with hbase ltt but i have this one:
> {code}
> $ hbase ltt -write 3:30:10 -num_keys 100
> 2017-07-03 14:40:47,372 ERROR [main] util.AbstractHBaseTool: Error running
> command-line tool
> java.lang.UnsupportedOperationException: HColumnDescriptor is read-only
> at
> org.apache.hadoop.hbase.client.ImmutableHColumnDescriptor.getDelegateeForModification(ImmutableHColumnDescriptor.java:44)
> at
> org.apache.hadoop.hbase.HColumnDescriptor.setBloomFilterType(HColumnDescriptor.java:475)
> at
> org.apache.hadoop.hbase.util.LoadTestTool.applyColumnFamilyOptions(LoadTestTool.java:288)
> at
> org.apache.hadoop.hbase.util.LoadTestTool.initTestTable(LoadTestTool.java:557)
> at
> org.apache.hadoop.hbase.util.LoadTestTool.loadTable(LoadTestTool.java:584)
> at
> org.apache.hadoop.hbase.util.LoadTestTool.doWork(LoadTestTool.java:565)
> at
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:154)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.AbstractHBaseTool.doStaticMain(AbstractHBaseTool.java:262)
> at
> org.apache.hadoop.hbase.util.LoadTestTool.main(LoadTestTool.java:831)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18317) Implement async admin operations for Normalizer/CleanerChore/CatalogJanitor
[ https://issues.apache.org/jira/browse/HBASE-18317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18317: -- Fix Version/s: (was: 3.0.0) > Implement async admin operations for Normalizer/CleanerChore/CatalogJanitor > --- > > Key: HBASE-18317 > URL: https://issues.apache.org/jira/browse/HBASE-18317 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.0.0-alpha-2, 2.0.0 > > Attachments: HBASE-18317.master.001.patch, > HBASE-18317.master.002.patch, HBASE-18317.master.002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17823) Migrate to Apache Yetus Audience Annotations
[ https://issues.apache.org/jira/browse/HBASE-17823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17823: -- Fix Version/s: 2.0.0 > Migrate to Apache Yetus Audience Annotations > > > Key: HBASE-17823 > URL: https://issues.apache.org/jira/browse/HBASE-17823 > Project: HBase > Issue Type: Improvement > Components: API >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-17823-branch-2.v3.patch, HBASE-17823.0.patch, > HBASE-17823.1.patch, HBASE-17823.2.patch, HBASE-17823.3.patch > > > Migrate from our own audience annotation handling to apache yetus' > implementation. > [discussion thread on > dev@hbase|https://lists.apache.org/thread.html/5a83d37c9c763b3fc4114231489a073167ac69dbade9774af5ca4fb4@%3Cdev.hbase.apache.org%3E] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18783) Declare the builder of ClusterStatus as IA.Private, and remove the Writables from ClusterStatus
[ https://issues.apache.org/jira/browse/HBASE-18783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18783: -- Fix Version/s: 2.0.0 > Declare the builder of ClusterStatus as IA.Private, and remove the Writables > from ClusterStatus > --- > > Key: HBASE-18783 > URL: https://issues.apache.org/jira/browse/HBASE-18783 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18783.v0.patch > > > To clarify the usage of ClusterStatus API. > # It makes no sense that user want to create custom ClusterStatus; As a > result, we should declare the builder of ClusterStatus as IA.Private. > # HBASE-6038 removed the write/readFields from ClusterStatus; thus the > Writable is meaningless for ClusterStatus now. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18739) Make all TimeRange Constructors InterfaceAudience Private.
[ https://issues.apache.org/jira/browse/HBASE-18739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18739: -- Fix Version/s: 2.0.0 > Make all TimeRange Constructors InterfaceAudience Private. > -- > > Key: HBASE-18739 > URL: https://issues.apache.org/jira/browse/HBASE-18739 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18739.master.001.patch > > > Make all constructors InterfaceAudience Private. Add comments from parent > issue on how to use this class -- no construction but may be passed about > Client-side -- and update the deprecate note that action will happen in 3.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18821) Enforcer plugin NPEs again when generating site
[ https://issues.apache.org/jira/browse/HBASE-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18821: -- Fix Version/s: 2.0.0 > Enforcer plugin NPEs again when generating site > --- > > Key: HBASE-18821 > URL: https://issues.apache.org/jira/browse/HBASE-18821 > Project: HBase > Issue Type: Sub-task > Components: site >Affects Versions: 2.0.0-alpha-3 >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18821-addendum.patch > > > The parent issue came back, HBASE-17351. Using 1.4.1 version of plugin NPEs. > Specify 1.4. I played w/ the suggestions over in MENFORCER-248 but to no > avail. There is a jenkins repository with a 1.4.2 which probably fixes this > and then there is the new 3.0.0 stuff Need to come back here. For now, > let me get a fix in so can make progress on releases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18700) Document the new changes on mapreduce stuffs
[ https://issues.apache.org/jira/browse/HBASE-18700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18700: -- Fix Version/s: 2.0.0 > Document the new changes on mapreduce stuffs > > > Key: HBASE-18700 > URL: https://issues.apache.org/jira/browse/HBASE-18700 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Duo Zhang >Assignee: stack >Priority: Major > Fix For: 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18700.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18681) Introduce a new constructor for StoreScanner for MOB
[
https://issues.apache.org/jira/browse/HBASE-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18681:
--
Fix Version/s: 2.0.0
> Introduce a new constructor for StoreScanner for MOB
>
>
> Key: HBASE-18681
> URL: https://issues.apache.org/jira/browse/HBASE-18681
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Affects Versions: 3.0.0, 2.0.0-alpha-2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-alpha-3, 2.0.0
>
>
> The mob compactor will create {{StoreScanner}} with several {{StoreFile}} s
> directly, so it can not use the constructors for the normal compaction as
> they all need a {{Store}} instance. But this constructor is only used by UTs
> and is expected to be removed, so let's introduce a new one for mob.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18691) [compat 1-2] HCD remove and removeConfiguration change return type
[
https://issues.apache.org/jira/browse/HBASE-18691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18691:
--
Fix Version/s: 2.0.0
> [compat 1-2] HCD remove and removeConfiguration change return type
> --
>
> Key: HBASE-18691
> URL: https://issues.apache.org/jira/browse/HBASE-18691
> Project: HBase
> Issue Type: Sub-task
> Components: API
>Reporter: stack
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0-alpha-3, 2.0.0
>
> Attachments: HBASE-18691.v0.patch
>
>
> Change made in HBASE-18008. Asking [~chia7712] if ok to undo.
> Here is complaint from JACC:
> {code}
> hbase-client-1.2.6.jar, HColumnDescriptor.class
> package org.apache.hadoop.hbase
> [−] HColumnDescriptor.remove ( byte[ ] key ) : void 1
> org/apache/hadoop/hbase/HColumnDescriptor.remove:([B)V
> ChangeEffect
> 1 Return value type has been changed from void to HColumnDescriptor.
> This method has been removed because the return type is part of the method
> signature.
> [−] HColumnDescriptor.removeConfiguration ( String key ) : void 1
> org/apache/hadoop/hbase/HColumnDescriptor.removeConfiguration:(Ljava/lang/String;)V
> ChangeEffect
> 1 Return value type has been changed from void to HColumnDescriptor.
> This method has been removed because the return type is part of the method
> signature.
> {code}
> Binary breaking but not src breaking. See
> https://stackoverflow.com/questions/3589946/retrofitting-void-methods-to-return-its-argument-to-facilitate-fluency-breaking
> for good discussion. Probably not a prob. but just in case and if not really
> needed, will purge.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18568) Correct metric of numRegions
[
https://issues.apache.org/jira/browse/HBASE-18568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18568:
--
Fix Version/s: 2.0.0
> Correct metric of numRegions
> --
>
> Key: HBASE-18568
> URL: https://issues.apache.org/jira/browse/HBASE-18568
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 3.0.0
>Reporter: Shibin Zhang
>Assignee: Shibin Zhang
>Priority: Critical
> Fix For: 3.0.0, 1.4.0, 2.0.0-alpha-3, 2.0.0
>
> Attachments: HBASE-18568-V1.patch
>
>
> i found the value of metric numReigons in Regions is not correct.
> the metric can not add or remove region correctly as region close or open.
> the metric as follow:
> "name" : "Hadoop:service=HBase,name=RegionServer,sub=Regions",
> "numRegions" : 2,
> after trouble shooting ,i found the reason is in
> MetricsRegionSourceImpl#MetricsRegionSourceImpl
> {code:java}
> agg.register(this);
> ...
> hashCode = regionWrapper.getRegionHashCode();
> {code}
> when add the MetricsRegionSource to set ,but the hashCode has not yet
> initialized.
> So, the setFromMap can not put or remove the object correctly.
> it will be better like this :
> {code:java}
> hashCode = regionWrapper.getRegionHashCode();
> agg.register(this);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18627) Fix TestRegionServerReadRequestMetrics
[ https://issues.apache.org/jira/browse/HBASE-18627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18627: -- Fix Version/s: 2.0.0 > Fix TestRegionServerReadRequestMetrics > -- > > Key: HBASE-18627 > URL: https://issues.apache.org/jira/browse/HBASE-18627 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18627.v0.patch, HBASE-18627.v0.patch > > > HBASE-18511 enable no-regions-on-master as default. That feature mess up the > metrics in TestRegionServerReadRequestMetrics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-14997) Move compareOp and Comparators out of filter to client package
[
https://issues.apache.org/jira/browse/HBASE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14997:
--
Fix Version/s: 2.0.0
> Move compareOp and Comparators out of filter to client package
> --
>
> Key: HBASE-14997
> URL: https://issues.apache.org/jira/browse/HBASE-14997
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-3, 2.0.0
>
> Attachments: HBASE-14997.master.001.patch,
> HBASE-14997.master.002.patch, HBASE-14997.master.003.patch,
> HBASE-14997.master.004.patch
>
>
> {{Table.checkAndPut()}} and its cousins depend on CompareOp from the filter
> package. Originally, ComparaOp and ByteArrayComparable, and various
> "comparators" have been used in filters, so these are in the filter package.
> However, for checkAndPut(), etc we depend on the filter subpackage although
> these are not filter related operations. We can use some clean up.
> {code}
> boolean checkAndPut(byte[] row, byte[] family, byte[] qualifier,
> CompareFilter.CompareOp compareOp, byte[] value, Put put) throws
> IOException;
> {code}
> Some ideas
> - Cleanup ByteArrayComparable interface (see the TODO at the class)
> - Maybe introduce a {{Condition}} or a similar concept and do
> {{checkAndPut(Condition condition, Put put)}} and change filters to use that
> as well.
> - Introducing Condition like thing will allow us to have an interface like:
> {{checkAndMutate(List conditions, List mutations)}}.
> - BinaryComparator, etc are not "Comparators", they are comparables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18721) Cleanup unused configs and private declaration
[ https://issues.apache.org/jira/browse/HBASE-18721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18721: -- Fix Version/s: 2.0.0 > Cleanup unused configs and private declaration > -- > > Key: HBASE-18721 > URL: https://issues.apache.org/jira/browse/HBASE-18721 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18721.branch-2.v0.patch, HBASE-18721.v0.patch, > HBASE-18721.v1.patch > > > Some patches cause the unused configs and private declaration. > -- > *Remove the useless codes* > HBASE-17471 > HRegion#HREGION_MVCC_PRE_ASSIGN > HRegion#DEFAULT_HREGION_MVCC_PRE_ASSIGN > HBASE-13194 > NamespaceAuditor#NS_AUDITOR_INIT_TIMEOUT > NamespaceAuditor#DEFAULT_NS_AUDITOR_INIT_TIMEOUT > HBASE-16812 > HRegionServer#REGION_LOCK_AWAIT_TIME_SEC > HRegionServer#DEFAULT_REGION_LOCK_AWAIT_TIME_SEC > HBASE-14614 > DisableTableProcedure#MarkRegionOfflineOpResult > HBASE-17001 > QuotaSnapshotStore#ViolationState > HBASE-9949 > StoreScanner#StoreScannerCompactionRace > HBASE-2840 > DeleteTracker#DeleteCompare > HBASE-13387 > ByteBufferUtils#lessThanUnsignedLong > HBASE-17877 > ByteBufferUtils#lessThanUnsignedInt > ByteBufferUtils#lessThanUnsignedShort > HBASE-13954 > CellComparator#RowComparator > HBASE-17001 > QuotaSnapshotStore#ViolationState > HBASE-17877 > Bytes#lessThanUnsignedLong > Bytes#lessThanUnsignedInt > Bytes#lessThanUnsignedShort > HBASE-18516 > RegionOpeningState > ResponseConverter#getRegionOpeningState > ResponseConverter#getRegionOpeningStateList > HBASE-17277 > BufferedMutatorImpl#HBASE_BUFFEREDMUTATOR_CLASSNAME_KEY > HBASE-2692 > HConstants#Modify > HBASE-15158 > CellUtil.EMPTY_TAGS_ITR > -- > *Deprecate the useless codes exposed as Public* > HBASE-3065 > HConstants#ZOOKEEPER_RECOVERABLE_WAITTIME > HConstants#DEFAULT_ZOOKEPER_RECOVERABLE_WAITIME > HBASE-13636 > HConstants#CLUSTER_IS_DISTRIBUTED > HBASE-7460 > HConstants#HBASECLIENT_IMPL > HBASE-12054 > HConstants#LIB_DIR > HBASE-2692 > HConstants#WEEK_IN_SECONDS > HBASE-3789 > HConstants#HBCK_CODE_NAME > HBASE-10569 > HConstants#MASTER_HANDLER_COUNT > HConstants#DEFAULT_MASTER_HANLDER_COUNT > HBASE-13616 > HConstants#LOG_REPLAY_WAIT_REGION_TIMEOUT -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18779) Move CompareOperator to hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-18779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18779: -- Fix Version/s: 2.0.0 > Move CompareOperator to hbase-client module > --- > > Key: HBASE-18779 > URL: https://issues.apache.org/jira/browse/HBASE-18779 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-3, 2.0.0 > > > Comes of [~enis] review over in HBASE-14997. Do this review feedback after > HBASE-18769 goes in. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18629) Enhance ChaosMonkeyRunner with interruptibility
[
https://issues.apache.org/jira/browse/HBASE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18629:
--
Fix Version/s: 2.0.0
> Enhance ChaosMonkeyRunner with interruptibility
> ---
>
> Key: HBASE-18629
> URL: https://issues.apache.org/jira/browse/HBASE-18629
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 2.0.0-alpha-3, 2.0.0
>
> Attachments: 18629.addendum, 18629.v1.txt, 18629.v2.txt
>
>
> Currently ChaosMonkeyRunner performs looping unconditionally:
> {code}
> while (true) {// loop here until got killed
> Thread.sleep(1);
> }
> {code}
> When ChaosMonkeyRunner is invoked programmatically, it is desirable to add
> interruptibility to the runner so that the caller can manage its lifetime.
> Another enhancement is to allow passing the path to hbase-site.xml where
> chaos monkey parameters are specified.
> This is useful when the underlying hbase-site.xml is not on classpath.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18662) The default values for many configuration items in the code are not consistent with hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-18662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18662:
--
Fix Version/s: 2.0.0
> The default values for many configuration items in the code are not
> consistent with hbase-default.xml
> -
>
> Key: HBASE-18662
> URL: https://issues.apache.org/jira/browse/HBASE-18662
> Project: HBase
> Issue Type: Improvement
>Reporter: Yun Zhao
>Assignee: Yun Zhao
>Priority: Minor
> Fix For: 2.0.0-alpha-3, 2.0.0
>
> Attachments: HBASE-18662.master.002.patch,
> HBASE-18662.master.003.patch
>
>
> Such as
> {code}
> hbase.ipc.server.callqueue.handler.factor
> hbase.regionserver.logroll.errors.tolerated
> hbase.regionserver.region.split.policy
> zookeeper.session.timeout
> hbase.client.retries.number
> hbase.client.max.perserver.tasks
> hbase.client.keyvalue.maxsize
> hbase.normalizer.period
> hbase.hstore.blockingStoreFiles
> hbase.snapshot.restore.take.failsafe.snapshot
> hbase.lease.recovery.dfs.timeout
> hbase.rest.filter.classes
> hbase.http.max.threads
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18818) TestConnectionImplemenation fails
[ https://issues.apache.org/jira/browse/HBASE-18818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18818: -- Fix Version/s: 2.0.0 > TestConnectionImplemenation fails > - > > Key: HBASE-18818 > URL: https://issues.apache.org/jira/browse/HBASE-18818 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-3, 1.1.13, 2.0.0 > > Attachments: 18818.txt > > > TestConnectionImplementation fails for me because it doesn't throw > UnknownHost exception when passed bogus name. > Bogus name ends in example.com. Should be .invalid per > https://tools.ietf.org/html/rfc2606 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-3935) HServerLoad.storefileIndexSizeMB should be changed to storefileIndexSizeKB
[ https://issues.apache.org/jira/browse/HBASE-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-3935: - Fix Version/s: 2.0.0 > HServerLoad.storefileIndexSizeMB should be changed to storefileIndexSizeKB > -- > > Key: HBASE-3935 > URL: https://issues.apache.org/jira/browse/HBASE-3935 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Andy Yang >Priority: Major > Fix For: 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-3935.branch-2.v0.patch, > HBASE-3935.branch-2.v1.patch, HBASE-3935.branch-2.v2.patch, > HBASE-3935.branch-2.v3.patch, HBASE-3935.branch-2.v4.patch, > HBASE-3935.branch-2.v5.patch, HBASE-3935.branch-2.v6.patch, > HBASE-3935.master.v0.patch > > > Related to HBASE-3927, Matt proposed changing > HServerLoad.storefileIndexSizeMB to storefileIndexSizeKB so that user can see > the size of small store file index. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18741) Remove cancel command from backup code
[ https://issues.apache.org/jira/browse/HBASE-18741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18741: -- Fix Version/s: 2.0.0 > Remove cancel command from backup code > -- > > Key: HBASE-18741 > URL: https://issues.apache.org/jira/browse/HBASE-18741 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Peter Somogyi >Priority: Major > Fix For: 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18741.master.001.patch > > > After recent refactoring of backup code, cancel command is no longer > applicable since there is no server procedure performing backup / restore. > This issue is to remove cancel command from BackupCommands.java -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-7320) Remove KeyValue.getBuffer()
[ https://issues.apache.org/jira/browse/HBASE-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7320: - Fix Version/s: 2.0.0 > Remove KeyValue.getBuffer() > --- > > Key: HBASE-7320 > URL: https://issues.apache.org/jira/browse/HBASE-7320 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: stack >Priority: Blocker > Fix For: 3.0.0, 2.0.0-alpha-3, 2.0.0 > > Attachments: 7320-simple.txt > > > In many places this is simple task of just replacing the method name. > There, however, quite a few places where we assume that: > # the entire KV is backed by a single byte array > # the KVs key portion is backed by a single byte array > Some of those can easily be fixed, others will need their own jiras. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18723) [pom cleanup] Do a pass with dependency:analyze; remove unused and explicity list the dependencies we exploit
[ https://issues.apache.org/jira/browse/HBASE-18723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18723: -- Fix Version/s: 2.0.0 > [pom cleanup] Do a pass with dependency:analyze; remove unused and explicity > list the dependencies we exploit > - > > Key: HBASE-18723 > URL: https://issues.apache.org/jira/browse/HBASE-18723 > Project: HBase > Issue Type: Bug > Components: pom >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-alpha-3, 2.0.0 > > Attachments: > HBASE-18723-pom-cleanup-Do-a-pass-with-dependency.addendum.patch, > HBASE-18723.master.001.patch, HBASE-18723.master.002.patch, > HBASE-18723.master.003.patch, HBASE-18723.master.004.patch, > HBASE-18723.master.005.patch > > > Do a pass over our poms. They are sloppy including unused jars and not > listing actually used dependencies. Undo 'required' dependencies like junit > and mockito; not all modules need these anymore. > This cleanup motivated by failures up on jenkins where a build step is not > finding transitive includes; explicit mention is needed (See failures in > HBASE-18674). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18704) Upgrade hbase to commons-collections 4
[ https://issues.apache.org/jira/browse/HBASE-18704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18704: -- Fix Version/s: 2.0.0 > Upgrade hbase to commons-collections 4 > -- > > Key: HBASE-18704 > URL: https://issues.apache.org/jira/browse/HBASE-18704 > Project: HBase > Issue Type: Improvement > Components: dependencies >Affects Versions: 2.0.0-alpha-2 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18704.master.001.patch, > HBASE-18704.master.002.patch, HBASE-18704.master.003.patch, > HBASE-18704.master.003.patch > > > Upgrade hbase to use commons-collections 4.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16390) Fix documentation around setAutoFlush
[
https://issues.apache.org/jira/browse/HBASE-16390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16390:
--
Fix Version/s: 2.0.0
> Fix documentation around setAutoFlush
> -
>
> Key: HBASE-16390
> URL: https://issues.apache.org/jira/browse/HBASE-16390
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: stack
>Assignee: Sahil Aggarwal
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0-alpha-3, 2.0.0
>
> Attachments: HBASE-16390.master.001.patch
>
>
> Our documentation is a little confused around setAutoFlush. Talks of Table
> but setAutoFlush is not in the Table interface. It was on HTable but was
> deprecated and since removed. Clean up the doc:
> {code}
> 100.4. HBase Client: AutoFlush
> When performing a lot of Puts, make sure that setAutoFlush is set to false
> on your Table
>
> instance.
> Otherwise, the Puts will be sent one at a time to the RegionServer. Puts
> added via table.add(Put) and table.add( Put) wind up in the same
> write buffer. If autoFlush = false, these messages are not sent until the
> write-buffer is filled. To explicitly flush the messages, call flushCommits.
> Calling close on the Table instance will invoke flushCommits
> {code}
> Spotted by Jeff Shmain.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18448) EndPoint example for refreshing HFiles for stores
[ https://issues.apache.org/jira/browse/HBASE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18448: -- Fix Version/s: 2.0.0 > EndPoint example for refreshing HFiles for stores > -- > > Key: HBASE-18448 > URL: https://issues.apache.org/jira/browse/HBASE-18448 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 1.3.1, 2.0.0 >Reporter: Ajay Jadhav >Assignee: Ajay Jadhav >Priority: Minor > Fix For: 1.4.0, 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18448.branch-1.001.patch, > HBASE-18448.branch-1.002.patch, HBASE-18448.branch-1.003.patch, > HBASE-18448.branch-1.004.patch, HBASE-18448.branch-1.005.patch, > HBASE-18448.branch-1.006.patch, HBASE-18448.branch-1.007.patch, > HBASE-18448.master.001.patch, HBASE-18448.master.002.patch > > > In the case where multiple HBase clusters are sharing a common rootDir, even > after flushing the data from > one cluster doesn't mean that other clusters (replicas) will automatically > pick the new HFile. Through this patch, > we are exposing the refresh HFiles API which when issued from a replica will > update the in-memory file handle list > with the newly added file. > This allows replicas to be consistent with the data written through the > primary cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18793) Remove deprecated methods in RegionObserver
[ https://issues.apache.org/jira/browse/HBASE-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18793: -- Fix Version/s: 2.0.0 > Remove deprecated methods in RegionObserver > --- > > Key: HBASE-18793 > URL: https://issues.apache.org/jira/browse/HBASE-18793 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 3.0.0, 2.0.0-alpha-2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18793-v1.patch, HBASE-18793.patch > > > As we have already done lots of incomplete changes on CP APIs and it is > allowed to do breaking changes on major release. > Let's make the API clean. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18740) Upgrade Zookeeper version to 3.4.10
[ https://issues.apache.org/jira/browse/HBASE-18740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18740: -- Fix Version/s: 2.0.0 > Upgrade Zookeeper version to 3.4.10 > --- > > Key: HBASE-18740 > URL: https://issues.apache.org/jira/browse/HBASE-18740 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.0, 1.5.0 >Reporter: Jerry He >Assignee: Jerry He >Priority: Major > Fix For: 1.4.0, 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18740-branch-1.patch, HBASE-18740-branch-1.patch, > HBASE-18740-master.patch > > > Branch 1.4 and branch 1 are still on Zookeeper 3.4.6. > Branch 2 and master branch have upgraded to 3.4.9. > There are some important fixes we'd like to have. See the linked JIRAs. > Another critical fix is ZOOKEEPER-2146, which can be explored maliciously. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18765) The value of balancerRan is true even though no plans are executed
[
https://issues.apache.org/jira/browse/HBASE-18765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18765:
--
Fix Version/s: 2.0.0
> The value of balancerRan is true even though no plans are executed
> --
>
> Key: HBASE-18765
> URL: https://issues.apache.org/jira/browse/HBASE-18765
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-alpha-3, 2.0.0
>
> Attachments: HBASE-18765.v0.patch
>
>
> {code}
> //We balance per group instead of per table
> List plans = new ArrayList<>();
> for(Map.Entry> tableMap:
> getRSGroupAssignmentsByTable(groupName).entrySet()) {
> LOG.info("Creating partial plan for table " + tableMap.getKey() + ": "
> + tableMap.getValue());
> List partialPlans =
> balancer.balanceCluster(tableMap.getValue());
> LOG.info("Partial plan for table " + tableMap.getKey() + ": " +
> partialPlans);
> if (partialPlans != null) {
> plans.addAll(partialPlans);
> }
> }
> long startTime = System.currentTimeMillis();
> balancerRan = plans != null;
> {code}
> The *plans* is never null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18732) [compat 1-2] HBASE-14047 removed Cell methods w/o deprecation cycle
[ https://issues.apache.org/jira/browse/HBASE-18732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18732: -- Fix Version/s: 2.0.0 > [compat 1-2] HBASE-14047 removed Cell methods w/o deprecation cycle > --- > > Key: HBASE-18732 > URL: https://issues.apache.org/jira/browse/HBASE-18732 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Priority: Major > Fix For: 2.0.0-alpha-3, 2.0.0 > > > This might be ok. We should just eat this. Cell was useless in 1.0.0. Opening > this just so we have a bit of a discussion about it. See HBASE-14047 where > removal was done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18768) Move TestTableName to hbase-common from hbase-server
[ https://issues.apache.org/jira/browse/HBASE-18768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18768: -- Fix Version/s: 2.0.0 > Move TestTableName to hbase-common from hbase-server > > > Key: HBASE-18768 > URL: https://issues.apache.org/jira/browse/HBASE-18768 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18768.master.001.patch > > > HBASE-18444 notices that TableName test is in a different module altogether > (hbase-server). Fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18614) Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18614:
--
Fix Version/s: 2.0.0
> Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI
> --
>
> Key: HBASE-18614
> URL: https://issues.apache.org/jira/browse/HBASE-18614
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.1.2
>Reporter: Biju Nair
>Assignee: Biju Nair
>Priority: Major
> Fix For: 3.0.0, 1.4.0, 2.0.0-alpha-3, 2.0.0
>
> Attachments: HBASE-18614-1.1.2.PATCH, HBASE-18614-BRANCH-1.PATCH,
> HBASE-18614-V1.0.PATCH, HBASE-18614-V2.0.PATCH, HBASE-18614.PATCH
>
>
> Enabling offheap cache and setting {{BUCKET_CACHE_COMBINED_KEY}} to {{false}}
> in site xml to make offheap cache a strict L2 cache to LRU cache, disables
> the L2 stats normally rendered on region server UI.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-16479) Move WALEdit from hbase.regionserver.wal package to hbase.wal package
[
https://issues.apache.org/jira/browse/HBASE-16479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16479:
--
Fix Version/s: 2.0.0
> Move WALEdit from hbase.regionserver.wal package to hbase.wal package
> -
>
> Key: HBASE-16479
> URL: https://issues.apache.org/jira/browse/HBASE-16479
> Project: HBase
> Issue Type: Sub-task
> Components: wal
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Fix For: 2.0.0-alpha-3, 2.0.0
>
> Attachments: HBASE-16479.branch-2.001.patch, hbase-16479_v1.patch
>
>
> {{hbase.wal}} is the new home for WAL related code. WALEdit should be there.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18699) Copy LoadIncrementalHFiles to another package and mark the old one as deprecated
[ https://issues.apache.org/jira/browse/HBASE-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18699: -- Fix Version/s: 2.0.0 > Copy LoadIncrementalHFiles to another package and mark the old one as > deprecated > > > Key: HBASE-18699 > URL: https://issues.apache.org/jira/browse/HBASE-18699 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 3.0.0, 2.0.0-alpha-2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-alpha-3, 2.0.0 > > Attachments: HBASE-18699-v1.patch, HBASE-18699-v2.patch, > HBASE-18699-v3.patch, HBASE-18699-v3.patch, HBASE-18699.patch > > > LoadIncrementalHFiles does not depend on map reduce. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
