[jira] [Commented] (HBASE-20506) Add doc and test for unused RetryCounter, useful-looking utility
[
https://issues.apache.org/jira/browse/HBASE-20506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459455#comment-16459455
]
Hadoop QA commented on HBASE-20506:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 26s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
56s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20506 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12921365/HBASE-20506.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 86da81690058 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5a071dbe2b |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12696/testReport/ |
| Max. process+thread count | 328 (vs. ulimit of 1) |
| modules | C: hbase-common U: hbase-common |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12696/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Add doc and test for
[jira] [Updated] (HBASE-20506) Add doc and test for unused RetryCounter, useful-looking utility
[ https://issues.apache.org/jira/browse/HBASE-20506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20506: -- Attachment: HBASE-20506.master.002.patch > Add doc and test for unused RetryCounter, useful-looking utility > > > Key: HBASE-20506 > URL: https://issues.apache.org/jira/browse/HBASE-20506 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Minor > Attachments: 20506.txt, HBASE-20506.master.001.patch, > HBASE-20506.master.002.patch > > > I thought I could use RetryCounter, old facility added years ago, for doing > backoff calculations. In the end, it didn't work for me because it is lacking > pb serialization. While trying to use it, I added a bit of doc and a test. > Might help the next dev that trips along this way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20455) [IHC] Workloadx
[
https://issues.apache.org/jira/browse/HBASE-20455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20455:
--
Fix Version/s: 2.1.0
> [IHC] Workloadx
> ---
>
> Key: HBASE-20455
> URL: https://issues.apache.org/jira/browse/HBASE-20455
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
>
> I tried workloadx from parent issue, the ycsb workload that is meant to make
> IHC shine. I'm doing something wrong. I tried just plugging it in and
> doing this:
> {code}
> ycsb run hbase12 -P /home/stack/ycsb/workloads/workloadx -p table=ycsb
> -threads 48 -cp /home/stack/conf_hbase -p columnfamily=family -p
> clientbuffering=true -p writebuffersize=2097152 -s -p maxexecutiontime=1200
> -jvm-args=-Xmx8192
> m -Djava.security.egd=file:/dev/./urandom -p recordcount=2500 -p
> operationcount=2500 -p
> exportfile=logs/ycsb-workloadx-measurements-ve0524-20180419T02:58:14Z.json -p
> exporter=com.yahoo.ycsb.measurements.exporter.JSONArrayMeasurementsExporter
> {code}
> It completes in a minute and a half. Claims stuff like this for throughput:
> 236825 ops/second. In this case I have in-memory-compaction set to NONE. I
> was going to compare before and after.
> [~eshcar]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20281) [DOC] upgrade section needs an explanation of changes to Bucket Cache
[
https://issues.apache.org/jira/browse/HBASE-20281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20281:
--
Fix Version/s: 2.1.0
> [DOC] upgrade section needs an explanation of changes to Bucket Cache
> -
>
> Key: HBASE-20281
> URL: https://issues.apache.org/jira/browse/HBASE-20281
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache, documentation
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 2.1.0
>
>
> the first pass version of the upgrade docs has a brief note about the bucket
> cache changing:
> {quote}
> * hbase.bucketcache.combinedcache.enabled
> * hbase.bucketcache.ioengine no longer supports the 'heap' value.
> {quote}
> But the changes are substantial and warrant a section of their own under the
> "Changes of Note!" banner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20413) IntegrationTestRegionReplicaReplication fails
[ https://issues.apache.org/jira/browse/HBASE-20413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20413: -- Fix Version/s: 2.1.0 > IntegrationTestRegionReplicaReplication fails > - > > Key: HBASE-20413 > URL: https://issues.apache.org/jira/browse/HBASE-20413 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.1.0 > > > Following [~psomogyi]'s inspiration, I've been running unit tests on a GCE > instance. IntegrationTestRegionReplicaReplication fails always for > branch-2.0. It fails for branch-1 too but for a different looking reason. On > branch-2.0, its OOME'ing. Downing the size of ingested data, it seems to hang > unable to update because it is unable to flush. > Marking this a blocker for now. Fix or disable it for branch-2.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20249) Investigate behavior of hbase.client.operation.timeout
[ https://issues.apache.org/jira/browse/HBASE-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20249: -- Fix Version/s: 2.1.0 > Investigate behavior of hbase.client.operation.timeout > -- > > Key: HBASE-20249 > URL: https://issues.apache.org/jira/browse/HBASE-20249 > Project: HBase > Issue Type: Bug >Reporter: Appy >Priority: Critical > Fix For: 3.0.0, 2.1.0 > > > hbase.client.operation.timeout should be hard limit on client operations, and > independent of number of retires. > Previous discussions here: > https://issues.apache.org/jira/browse/HBASE-17449?focusedCommentId=16390085=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16390085 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20252) Admin.move will not fail if we move region to a nonexistent region server
[ https://issues.apache.org/jira/browse/HBASE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20252: -- Fix Version/s: 2.1.0 > Admin.move will not fail if we move region to a nonexistent region server > -- > > Key: HBASE-20252 > URL: https://issues.apache.org/jira/browse/HBASE-20252 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20252-UT.patch > > > The region will just be reopened on the source regionserver... > This is a bit confusing I think... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20151) Bug with SingleColumnValueFilter and FamilyFilter
[
https://issues.apache.org/jira/browse/HBASE-20151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20151:
--
Fix Version/s: 2.1.0
> Bug with SingleColumnValueFilter and FamilyFilter
> -
>
> Key: HBASE-20151
> URL: https://issues.apache.org/jira/browse/HBASE-20151
> Project: HBase
> Issue Type: Bug
> Environment: MacOS 10.13.3
> HBase 1.3.1
>Reporter: Steven Sadowski
>Assignee: Reid Chan
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20151.master.001.patch,
> HBASE-20151.master.002.patch, HBASE-20151.master.003.patch,
> HBASE-20151.master.004.patch, HBASE-20151.master.004.patch
>
>
> When running the following queries, the result is sometimes return correctly
> and other times incorrectly based on the qualifier queried.
> Setup:
> {code:java}
> create 'test', 'a', 'b'
> test = get_table 'test'
> test.put '1', 'a:1', nil
> test.put '1', 'a:10', nil
> test.put '1', 'b:2', nil
> {code}
>
> This query works fine when the SCVF's qualifier has length 1 (i.e. '1') :
> {code:java}
> test.scan({ FILTER => "(
> SingleColumnValueFilter('a','1',=,'binary:',true,true) AND
> FamilyFilter(=,'binary:b') )"})
> ROW COLUMN+CELL
> 1column=b:2,
> timestamp=1520455888059, value=
> 1 row(s) in 0.0060 seconds
> {code}
>
> The query should return the same result when passed a qualifier of length 2
> (i.e. '10') :
> {code:java}
> test.scan({ FILTER => "(
> SingleColumnValueFilter('a','10',=,'binary:',true,true) AND
> FamilyFilter(=,'binary:b') )"})
> ROW COLUMN+CELL
> 0 row(s) in 0.0110 seconds
> {code}
> However, in this case, it does not return any row (expected result would be
> to return the same result as the first query).
>
> Removing the family filter while the qualifier is '10' yields expected
> results:
> {code:java}
> test.scan({ FILTER => "(
> SingleColumnValueFilter('a','10',=,'binary:',true,true) )"})
> ROW COLUMN+CELL
> 1column=a:1,
> timestamp=1520455887954, value=
> 1column=a:10,
> timestamp=1520455888024, value=
> 1column=b:2,
> timestamp=1520455888059, value=
> 1 row(s) in 0.0140 seconds
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20383) [AMv2] AssignmentManager: Failed transition XYZ is not OPEN
[
https://issues.apache.org/jira/browse/HBASE-20383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20383:
--
Fix Version/s: 2.1.0
> [AMv2] AssignmentManager: Failed transition XYZ is not OPEN
> ---
>
> Key: HBASE-20383
> URL: https://issues.apache.org/jira/browse/HBASE-20383
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20383.master.001.patch
>
>
> Seeing a bunch of this testing 2.0.0:
> {code}
> 2018-04-10 13:57:09,430 WARN
> [RpcServer.default.FPBQ.Fifo.handler=46,queue=1,port=16000]
> assignment.AssignmentManager: Failed transition
>
>
> org.apache.hadoop.hbase.client.DoNotRetryRegionException:
> 19a2cd6f88abae0036415ee1ea041c2e is not OPEN
> at
> org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure.checkOnline(AbstractStateMachineTableProcedure.java:193)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:112)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:769)
>
>
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.updateRegionSplitTransition(AssignmentManager.java:911)
>
>
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportRegionStateTransition(AssignmentManager.java:819)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.reportRegionStateTransition(MasterRpcServices.java:1538)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:11093)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>
>
>at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {code}
> Looks like report back from Master OK'ing a split to go ahead but the split
> is already running. Figure how to shut these down. They are noisy at least.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20188: -- Fix Version/s: 2.1.0 > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 3.0.0, 2.1.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, hits.png, hits_with_fp_scheduler.png, > lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, perregion.png, run_ycsb.sh, > total.png, tree.txt, workloadx, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20331) clean up shaded packaging for 2.0
[ https://issues.apache.org/jira/browse/HBASE-20331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20331: -- Fix Version/s: 2.1.0 > clean up shaded packaging for 2.0 > - > > Key: HBASE-20331 > URL: https://issues.apache.org/jira/browse/HBASE-20331 > Project: HBase > Issue Type: Umbrella > Components: Client, mapreduce, shading >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 2.1.0 > > > polishing pass on shaded modules for 2.0 based on trying to use them in more > contexts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20266) Review current set of ignored tests
[
https://issues.apache.org/jira/browse/HBASE-20266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20266:
--
Fix Version/s: 2.1.0
> Review current set of ignored tests
> ---
>
> Key: HBASE-20266
> URL: https://issues.apache.org/jira/browse/HBASE-20266
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 3.0.0, 2.1.0
>
>
> [~Apache9] turned up a list of currently ignored tests. At first blush, its
> fine to ignore some such as TestHTraceHooks and TestRegionsOnMaster but
> others could do with a review. This issue is about looking at list to make
> sure nothing important missed for hbase2 and that we for sure marked why a
> test was ignored with comment and that there is a follow-on to enable JIRA.
> {code}
> TestRpcHandlerException
> TestRSKilledWhenInitializing
> TestHTraceHooks
> TestAsyncTableGetMultiThreadedWithEagerCompaction
> TestStochasticBalancerJmxMetrics
> TestReplicator
> TestQuotaThrottle
> TestFavoredStochasticLoadBalancer
> TestAsyncTableGetMultiThreadedWithBasicCompaction
> TestRegionPlacement
> TestMasterTransitions
> TestMemstoreLABWithoutPool
> TestRegionsOnMasterOptions
> TestRestoreSnapshotFromClientWithRegionReplicas
> TestMasterBalanceThrottling
> TestMasterProcedureWalLease
> TestRegionServerReadRequestMetrics
> TestHttpServerLifecycle
> TestHRegionServerBulkLoadWithOldSecureEndpoint
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18785) Blocker doc issues for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-18785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18785: -- Fix Version/s: 2.1.0 > Blocker doc issues for 2.0.0 > > > Key: HBASE-18785 > URL: https://issues.apache.org/jira/browse/HBASE-18785 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.1.0 > > > Hang blocker documentation issues off this one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18495) [AMv2] Make methods isServerOnline() and isServerDead() in ServerManager mutually exclusive
[ https://issues.apache.org/jira/browse/HBASE-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18495: -- Fix Version/s: 2.1.0 > [AMv2] Make methods isServerOnline() and isServerDead() in ServerManager > mutually exclusive > --- > > Key: HBASE-18495 > URL: https://issues.apache.org/jira/browse/HBASE-18495 > Project: HBase > Issue Type: Bug > Components: amv2 >Affects Versions: 2.0.0 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 3.0.0, 2.1.0 > > > During debugging HBASE-18366, it was found that > ServerManager.isServerOnline() and ServerManager.isServerDead() can return > true at the same time. This causes problems with the code depending on which > method is used. Making them mutually exclusive seems line right thing to do. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20492) UnassignProcedure is stuck in retry loop on region stuck in OPENING state
[
https://issues.apache.org/jira/browse/HBASE-20492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20492:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.1.0
3.0.0
Status: Resolved (was: Patch Available)
Pushed to branch-2.0, branch-2, and to master branch.
> UnassignProcedure is stuck in retry loop on region stuck in OPENING state
> -
>
> Key: HBASE-20492
> URL: https://issues.apache.org/jira/browse/HBASE-20492
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Umesh Agashe
>Assignee: stack
>Priority: Critical
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20492.branch-2.0.001.patch,
> HBASE-20492.branch-2.0.002.patch, HBASE-20492.branch-2.0.003.patch
>
>
> UnassignProcedure gets stuck in a retry loop for a region stuck in OPENING
> state. From logs:
> {code:java}
> 2018-04-25 15:59:53,825 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure:
> Retryable error trying to transition: pid=142564,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
> table=IntegrationTestBigLinkedList_20180331004141,
> region=bd2fb2c7d39236c9b9085f350358df7c,
> server=vb1122.halxg.cloudera.com,22101,1522626198450; rit=OPENING,
> location=vb1122.halxg.cloudera.com,22101,1522626198450
> org.apache.hadoop.hbase.exceptions.UnexpectedStateException: Expected
> [SPLITTING, SPLIT, MERGING, OPEN, CLOSING] so could move to CLOSING but
> current state=OPENING
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates$RegionStateNode.transitionState(RegionStates.java:158)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1514)
> at
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:179)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:309)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:85)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
> 2018-04-25 15:59:53,892 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure:
> Retryable error trying to transition: pid=142564,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
> table=IntegrationTestBigLinkedList_20180331004141,
> region=bd2fb2c7d39236c9b9085f350358df7c,
> server=vb1122.halxg.cloudera.com,22101,1522626198450; rit=OPENING,
> location=vb1122.halxg.cloudera.com,22101,1522626198450
> org.apache.hadoop.hbase.exceptions.UnexpectedStateException: Expected
> [SPLITTING, SPLIT, MERGING, OPEN, CLOSING] so could move to CLOSING but
> current state=OPENING
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates$RegionStateNode.transitionState(RegionStates.java:158)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1514)
> at
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:179)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:309)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:85)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18622) Mitigate API compatibility concerns between branch-1 and branch-2
[ https://issues.apache.org/jira/browse/HBASE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18622: -- Fix Version/s: 2.1.0 > Mitigate API compatibility concerns between branch-1 and branch-2 > - > > Key: HBASE-18622 > URL: https://issues.apache.org/jira/browse/HBASE-18622 > Project: HBase > Issue Type: Bug > Components: API >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 3.0.0, 2.1.0 > > Attachments: report.1.2_2.0.html.gz > > > This project is to do what [~apurtell] did in the issue "HBASE-18431 Mitigate > compatibility concerns between branch-1.3 and branch-1.4" only do it between > branch-1 and branch-2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15648) Reduce number of concurrent region location lookups when MetaCache entry is cleared
[ https://issues.apache.org/jira/browse/HBASE-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15648: -- Fix Version/s: 2.1.0 > Reduce number of concurrent region location lookups when MetaCache entry is > cleared > --- > > Key: HBASE-15648 > URL: https://issues.apache.org/jira/browse/HBASE-15648 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 1.2.0, 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-15648-branch-1.3.v1.patch > > > It seems in HConnectionImplementation#locateRegionInMeta if region location > is removed from the cache, with large number of client threads we could have > many of them getting cache miss and doing meta scan, which looks unnecessary > - we could empty mechanism similar to what we have in IdLock in HFileReader > to fetch the block to cache, do ensure that if one thread is already looking > up location for region R1, other threads who need it's location wait until > first thread finishes his work. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20492) UnassignProcedure is stuck in retry loop on region stuck in OPENING state
[
https://issues.apache.org/jira/browse/HBASE-20492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459351#comment-16459351
]
Umesh Agashe commented on HBASE-20492:
--
+1, after fixing new checkstyle errors.
> UnassignProcedure is stuck in retry loop on region stuck in OPENING state
> -
>
> Key: HBASE-20492
> URL: https://issues.apache.org/jira/browse/HBASE-20492
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Umesh Agashe
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments: HBASE-20492.branch-2.0.001.patch,
> HBASE-20492.branch-2.0.002.patch, HBASE-20492.branch-2.0.003.patch
>
>
> UnassignProcedure gets stuck in a retry loop for a region stuck in OPENING
> state. From logs:
> {code:java}
> 2018-04-25 15:59:53,825 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure:
> Retryable error trying to transition: pid=142564,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
> table=IntegrationTestBigLinkedList_20180331004141,
> region=bd2fb2c7d39236c9b9085f350358df7c,
> server=vb1122.halxg.cloudera.com,22101,1522626198450; rit=OPENING,
> location=vb1122.halxg.cloudera.com,22101,1522626198450
> org.apache.hadoop.hbase.exceptions.UnexpectedStateException: Expected
> [SPLITTING, SPLIT, MERGING, OPEN, CLOSING] so could move to CLOSING but
> current state=OPENING
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates$RegionStateNode.transitionState(RegionStates.java:158)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1514)
> at
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:179)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:309)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:85)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
> 2018-04-25 15:59:53,892 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure:
> Retryable error trying to transition: pid=142564,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
> table=IntegrationTestBigLinkedList_20180331004141,
> region=bd2fb2c7d39236c9b9085f350358df7c,
> server=vb1122.halxg.cloudera.com,22101,1522626198450; rit=OPENING,
> location=vb1122.halxg.cloudera.com,22101,1522626198450
> org.apache.hadoop.hbase.exceptions.UnexpectedStateException: Expected
> [SPLITTING, SPLIT, MERGING, OPEN, CLOSING] so could move to CLOSING but
> current state=OPENING
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates$RegionStateNode.transitionState(RegionStates.java:158)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1514)
> at
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:179)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:309)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:85)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-13428) [DOC] Migration to hbase-2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13428: -- Fix Version/s: 2.1.0 > [DOC] Migration to hbase-2.0.0 > -- > > Key: HBASE-13428 > URL: https://issues.apache.org/jira/browse/HBASE-13428 > Project: HBase > Issue Type: Umbrella > Components: migration >Reporter: stack >Priority: Blocker > Fix For: 3.0.0, 2.1.0 > > > Opening a 2.0 umbrella migration issue. Lets hang off this one any tools and > expectations migrating from 1.0 (or earlier) to 2.0. So far there are none > that I know of though there is an expectation in HBASE-13373 that hfiles are > at least major version 2 and minor version 3. Lets list all such > expectations, etc., here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15654) Optimize client's MetaCache handling
[ https://issues.apache.org/jira/browse/HBASE-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15654: -- Fix Version/s: 2.1.0 > Optimize client's MetaCache handling > > > Key: HBASE-15654 > URL: https://issues.apache.org/jira/browse/HBASE-15654 > Project: HBase > Issue Type: Umbrella > Components: Client >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0 > > > This is an umbrella jira to track all individual issues, bugfixes and small > optimizations around MetaCache (region locations cache) in the client. > Motivation is that under the load one could see a spikes in the number of > requests going to meta - reaching tens of thousands requests per second. > That covers issues when we clear entries from location cache unnecessary, as > well as when we do more lookups than necessary when entries are legitimately > evicted. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15539) HBase Client region location is expensive
[ https://issues.apache.org/jira/browse/HBASE-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15539: -- Fix Version/s: 2.1.0 > HBase Client region location is expensive > -- > > Key: HBASE-15539 > URL: https://issues.apache.org/jira/browse/HBASE-15539 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Vladimir Rodionov >Assignee: Mikhail Antonov >Priority: Major > Fix For: 3.0.0, 2.1.0 > > > ConnectionImplementation.locateRegion and MetaCache.getTableLocations are hot > spots in a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17553) Make a 2.0.0 Release
[ https://issues.apache.org/jira/browse/HBASE-17553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459345#comment-16459345 ] stack commented on HBASE-17553: --- This issue HBASE-15317 had an announcement template. I forgot about it and failed to use it in hbase-2.0.0. Use it in future. > Make a 2.0.0 Release > > > Key: HBASE-17553 > URL: https://issues.apache.org/jira/browse/HBASE-17553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > > Umbrella issue to keep account of tasks to make a 2.0.0 release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-15317) document release announcement template
[ https://issues.apache.org/jira/browse/HBASE-15317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459340#comment-16459340 ] stack commented on HBASE-15317: --- Ugh. I failed to use this. > document release announcement template > -- > > Key: HBASE-15317 > URL: https://issues.apache.org/jira/browse/HBASE-15317 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0 > > > Our release docs should include a release announcement template for folks to > use (and suggested email lists to send to) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15809) Basic Replication WebUI
[ https://issues.apache.org/jira/browse/HBASE-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15809: -- Fix Version/s: 2.1.0 > Basic Replication WebUI > --- > > Key: HBASE-15809 > URL: https://issues.apache.org/jira/browse/HBASE-15809 > Project: HBase > Issue Type: New Feature > Components: Replication, UI >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Jingyun Tian >Priority: Critical > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-15809-v0.patch, HBASE-15809-v0.png, > HBASE-15809-v1.patch, rep_web_ui.zip > > > At the moment the only way to have some insight on replication from the webui > is looking at zkdump and metrics. > the basic information useful to get started debugging are: peer information > and the view of WALs offsets for each peer. > https://reviews.apache.org/r/47275/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-17553) Make a 2.0.0 Release
[ https://issues.apache.org/jira/browse/HBASE-17553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-17553. --- Resolution: Fixed Fix Version/s: (was: 3.0.0) 2.0.0 Resolving as done, as part of 2.0.0 though we are resolving post-release. The announce is currently hung up in an apache filter. Resolving elsewhere. > Make a 2.0.0 Release > > > Key: HBASE-17553 > URL: https://issues.apache.org/jira/browse/HBASE-17553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > > Umbrella issue to keep account of tasks to make a 2.0.0 release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20142) Copy master doc into branch-2 and edit to make it suit 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-20142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-20142. --- Resolution: Fixed Assignee: stack Fix Version/s: 2.0.0 Resolving as done, as part of 2.0.0 though we are resolving post-release > Copy master doc into branch-2 and edit to make it suit 2.0.0 > > > Key: HBASE-20142 > URL: https://issues.apache.org/jira/browse/HBASE-20142 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0 > > > Place-holder for task to be done before we RC... copy the master refguide > into branch-2 and do a pass to make sure it hbase-2 appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20319) Run a check that branch-2.0 has all it needs from branch-2
[ https://issues.apache.org/jira/browse/HBASE-20319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-20319. --- Resolution: Fixed Assignee: stack Fix Version/s: 2.0.0 Resolving as done, as part of 2.0.0 though we are resolving post-release > Run a check that branch-2.0 has all it needs from branch-2 > -- > > Key: HBASE-20319 > URL: https://issues.apache.org/jira/browse/HBASE-20319 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0 > > > Note to myself. Make sure to check branch-2 for stuff that belongs in > branch-2.0 and that didn't make it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20315) Document post release process steps for RM
[
https://issues.apache.org/jira/browse/HBASE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459283#comment-16459283
]
stack edited comment on HBASE-20315 at 4/30/18 11:56 PM:
-
Example of moving an RC from dev to release:
{code}$ svn mv dev/hbase/hbase-2.0.0RC2 release/hbase/2.0.0
$ # // Remove an old release
$ mvn rm release/hbase/2.0.0-beta-2
$ # Commit the removal and move.
$ svn commit -m "Release 2.0.0 copying 2.0.0RC2 to release. Remove 2.0.0-beta-2"
{code}
Writing the release announcement, I noticed [~busbey]'s stamping an easily
finding link to our release notes up in JIRA using apache url shortener
e.g. I used https://s.apache.org/new to create
https://s.apache.org/hbase-2.0.0-JIRA-changes
I tried to push to announce but got blocked because mirrors don't have
signatures and hashes. There needs to be a connection between hashes and
signatures hosted on apache and the artifacts gotten from mirrors. Projects
usually do this on a 'download' page. Added one with 2.0.0 as first entry over
on HBASE-20510. Let me integrate into the steps filling out a row in this
table...
was (Author: stack):
Example of moving an RC from dev to release:
{code}$ svn mv dev/hbase/hbase-2.0.0RC2 release/hbase/2.0.0
$ # // Remove an old release
$ mvn rm release/hbase/2.0.0-beta-2
$ # Commit the removal and move.
$ svn commit -m "Release 2.0.0 copying 2.0.0RC2 to release. Remove 2.0.0-beta-2"
{code}
Writing the release announcement, I noticed [~busbey]'s stamping an easily
finding link to our release notes up in JIRA using apache url shortener
e.g. I used https://s.apache.org/new to create
https://s.apache.org/hbase-2.0.0-JIRA-releasenotes
I tried to push to announce but got blocked because mirrors don't have
signatures and hashes. There needs to be a connection between hashes and
signatures hosted on apache and the artifacts gotten from mirrors. Projects
usually do this on a 'download' page. Added one with 2.0.0 as first entry over
on HBASE-20510. Let me integrate into the steps filling out a row in this
table...
> Document post release process steps for RM
> --
>
> Key: HBASE-20315
> URL: https://issues.apache.org/jira/browse/HBASE-20315
> Project: HBase
> Issue Type: Task
> Components: build, documentation
>Reporter: Andrew Purtell
>Priority: Critical
> Fix For: 3.0.0
>
>
> We should document post release steps that RMs have to take and add it to the
> 'How To Release' section of the refguide.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20510) Add a downloads page to hbase.apache.org to tie mirrored artifacts to their hash and signature
[ https://issues.apache.org/jira/browse/HBASE-20510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20510: -- Attachment: Screen Shot 2018-04-30 at 4.50.45 PM.png > Add a downloads page to hbase.apache.org to tie mirrored artifacts to their > hash and signature > -- > > Key: HBASE-20510 > URL: https://issues.apache.org/jira/browse/HBASE-20510 > Project: HBase > Issue Type: Task > Components: website >Reporter: stack >Priority: Major > Attachments: > 0001-HBASE-20510-Add-a-downloads-page-to-hbase.apache.org.patch, Screen Shot > 2018-04-30 at 4.50.45 PM.png > > > I tried to push to announce for 2.0.0 but it got blocked because mirrors > don't have signatures and hashes (I only just noticed. Its probably been this > way for ages). Signatures and hashes are hosted on www.apache.org only. > So, there needs to be a connection made between hashes and signatures hosted > on apache.org and the associated artifacts gotten from mirrors. Projects > usually do this on a 'download' page. Add one with 2.0.0 as first entry. > [~misty] FYI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20510) Add a downloads page to hbase.apache.org to tie mirrored artifacts to their hash and signature
[ https://issues.apache.org/jira/browse/HBASE-20510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459288#comment-16459288 ] stack commented on HBASE-20510: --- Leaving open for [~misty]'s review. > Add a downloads page to hbase.apache.org to tie mirrored artifacts to their > hash and signature > -- > > Key: HBASE-20510 > URL: https://issues.apache.org/jira/browse/HBASE-20510 > Project: HBase > Issue Type: Task > Components: website >Reporter: stack >Priority: Major > Attachments: > 0001-HBASE-20510-Add-a-downloads-page-to-hbase.apache.org.patch, Screen Shot > 2018-04-30 at 4.50.45 PM.png > > > I tried to push to announce for 2.0.0 but it got blocked because mirrors > don't have signatures and hashes (I only just noticed. Its probably been this > way for ages). Signatures and hashes are hosted on www.apache.org only. > So, there needs to be a connection made between hashes and signatures hosted > on apache.org and the associated artifacts gotten from mirrors. Projects > usually do this on a 'download' page. Add one with 2.0.0 as first entry. > [~misty] FYI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20510) Add a downloads page to hbase.apache.org to tie mirrored artifacts to their hash and signature
[ https://issues.apache.org/jira/browse/HBASE-20510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459286#comment-16459286 ] stack commented on HBASE-20510: --- What I pushed to master branch. Let me attach a picture. > Add a downloads page to hbase.apache.org to tie mirrored artifacts to their > hash and signature > -- > > Key: HBASE-20510 > URL: https://issues.apache.org/jira/browse/HBASE-20510 > Project: HBase > Issue Type: Task > Components: website >Reporter: stack >Priority: Major > Attachments: > 0001-HBASE-20510-Add-a-downloads-page-to-hbase.apache.org.patch, Screen Shot > 2018-04-30 at 4.50.45 PM.png > > > I tried to push to announce for 2.0.0 but it got blocked because mirrors > don't have signatures and hashes (I only just noticed. Its probably been this > way for ages). Signatures and hashes are hosted on www.apache.org only. > So, there needs to be a connection made between hashes and signatures hosted > on apache.org and the associated artifacts gotten from mirrors. Projects > usually do this on a 'download' page. Add one with 2.0.0 as first entry. > [~misty] FYI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20510) Add a downloads page to hbase.apache.org to tie mirrored artifacts to their hash and signature
[ https://issues.apache.org/jira/browse/HBASE-20510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20510: -- Attachment: 0001-HBASE-20510-Add-a-downloads-page-to-hbase.apache.org.patch > Add a downloads page to hbase.apache.org to tie mirrored artifacts to their > hash and signature > -- > > Key: HBASE-20510 > URL: https://issues.apache.org/jira/browse/HBASE-20510 > Project: HBase > Issue Type: Task > Components: website >Reporter: stack >Priority: Major > Attachments: > 0001-HBASE-20510-Add-a-downloads-page-to-hbase.apache.org.patch > > > I tried to push to announce for 2.0.0 but it got blocked because mirrors > don't have signatures and hashes (I only just noticed. Its probably been this > way for ages). Signatures and hashes are hosted on www.apache.org only. > So, there needs to be a connection made between hashes and signatures hosted > on apache.org and the associated artifacts gotten from mirrors. Projects > usually do this on a 'download' page. Add one with 2.0.0 as first entry. > [~misty] FYI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20315) Document post release process steps for RM
[
https://issues.apache.org/jira/browse/HBASE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459283#comment-16459283
]
stack commented on HBASE-20315:
---
Example of moving an RC from dev to release:
{code}$ svn mv dev/hbase/hbase-2.0.0RC2 release/hbase/2.0.0
$ # // Remove an old release
$ mvn rm release/hbase/2.0.0-beta-2
$ # Commit the removal and move.
$ svn commit -m "Release 2.0.0 copying 2.0.0RC2 to release. Remove 2.0.0-beta-2"
{code}
Writing the release announcement, I noticed [~busbey]'s stamping an easily
finding link to our release notes up in JIRA using apache url shortener
e.g. I used https://s.apache.org/new to create
https://s.apache.org/hbase-2.0.0-JIRA-releasenotes
I tried to push to announce but got blocked because mirrors don't have
signatures and hashes. There needs to be a connection between hashes and
signatures hosted on apache and the artifacts gotten from mirrors. Projects
usually do this on a 'download' page. Added one with 2.0.0 as first entry over
on HBASE-20510. Let me integrate into the steps filling out a row in this
table...
> Document post release process steps for RM
> --
>
> Key: HBASE-20315
> URL: https://issues.apache.org/jira/browse/HBASE-20315
> Project: HBase
> Issue Type: Task
> Components: build, documentation
>Reporter: Andrew Purtell
>Priority: Critical
> Fix For: 3.0.0
>
>
> We should document post release steps that RMs have to take and add it to the
> 'How To Release' section of the refguide.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20509) Put List in HashSet directly without using addAll function to improve performance
[ https://issues.apache.org/jira/browse/HBASE-20509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] taiyinglee updated HBASE-20509: --- Summary: Put List in HashSet directly without using addAll function to improve performance (was: put List in HashSet directly when initialized without using addAll function to improve performance) > Put List in HashSet directly without using addAll function to improve > performance > - > > Key: HBASE-20509 > URL: https://issues.apache.org/jira/browse/HBASE-20509 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: taiyinglee >Assignee: taiyinglee >Priority: Trivial > Attachments: HBASE-20509.v0.patch > > > [https://docs.oracle.com/javase/7/docs/api/java/util/Collection.html] > change > Set serverSet = new HashSet<>(); > serverSet.addAll(serverList); > to > Set serverSet = new HashSet<>(serverList); > in > ./hbase-server/src/main/java/org/apache/hadoop/hbase/favored/FavoredNodeAssignmentHelper.java > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20509) put List in HashSet directly when initialized without using addAll function to improve performance
[ https://issues.apache.org/jira/browse/HBASE-20509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] taiyinglee updated HBASE-20509: --- Summary: put List in HashSet directly when initialized without using addAll function to improve performance (was: put List in HashSet directly without using addAll function to improve performance) > put List in HashSet directly when initialized without using addAll function > to improve performance > -- > > Key: HBASE-20509 > URL: https://issues.apache.org/jira/browse/HBASE-20509 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: taiyinglee >Assignee: taiyinglee >Priority: Trivial > Attachments: HBASE-20509.v0.patch > > > [https://docs.oracle.com/javase/7/docs/api/java/util/Collection.html] > change > Set serverSet = new HashSet<>(); > serverSet.addAll(serverList); > to > Set serverSet = new HashSet<>(serverList); > in > ./hbase-server/src/main/java/org/apache/hadoop/hbase/favored/FavoredNodeAssignmentHelper.java > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20509) put List in HashSet directly without using addAll function to improve performance
[ https://issues.apache.org/jira/browse/HBASE-20509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] taiyinglee updated HBASE-20509: --- Summary: put List in HashSet directly without using addAll function to improve performance (was: putting List in HashSet directly without using addAll function to improve performance) > put List in HashSet directly without using addAll function to improve > performance > - > > Key: HBASE-20509 > URL: https://issues.apache.org/jira/browse/HBASE-20509 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: taiyinglee >Assignee: taiyinglee >Priority: Trivial > Attachments: HBASE-20509.v0.patch > > > [https://docs.oracle.com/javase/7/docs/api/java/util/Collection.html] > change > Set serverSet = new HashSet<>(); > serverSet.addAll(serverList); > to > Set serverSet = new HashSet<>(serverList); > in > ./hbase-server/src/main/java/org/apache/hadoop/hbase/favored/FavoredNodeAssignmentHelper.java > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20509) putting List in HashSet directly without using addAll function to improve performance
[ https://issues.apache.org/jira/browse/HBASE-20509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459247#comment-16459247 ] taiyinglee commented on HBASE-20509: Yes, there are some similar issues, I can add more fixes for this jira. Thank you. > putting List in HashSet directly without using addAll function to improve > performance > - > > Key: HBASE-20509 > URL: https://issues.apache.org/jira/browse/HBASE-20509 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: taiyinglee >Assignee: taiyinglee >Priority: Trivial > Attachments: HBASE-20509.v0.patch > > > [https://docs.oracle.com/javase/7/docs/api/java/util/Collection.html] > change > Set serverSet = new HashSet<>(); > serverSet.addAll(serverList); > to > Set serverSet = new HashSet<>(serverList); > in > ./hbase-server/src/main/java/org/apache/hadoop/hbase/favored/FavoredNodeAssignmentHelper.java > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20510) Add a downloads page to hbase.apache.org to tie mirrored artifacts to their hash and signature
stack created HBASE-20510: - Summary: Add a downloads page to hbase.apache.org to tie mirrored artifacts to their hash and signature Key: HBASE-20510 URL: https://issues.apache.org/jira/browse/HBASE-20510 Project: HBase Issue Type: Task Components: website Reporter: stack I tried to push to announce for 2.0.0 but it got blocked because mirrors don't have signatures and hashes (I only just noticed. Its probably been this way for ages). Signatures and hashes are hosted on www.apache.org only. So, there needs to be a connection made between hashes and signatures hosted on apache.org and the associated artifacts gotten from mirrors. Projects usually do this on a 'download' page. Add one with 2.0.0 as first entry. [~misty] FYI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16668) Admin class should have a synchronous Admin#mergeRegions* method
[
https://issues.apache.org/jira/browse/HBASE-16668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458991#comment-16458991
]
Ted Yu commented on HBASE-16668:
{code}
[ERROR] Run 1:
TestInterfaceAlign.testAdminWithAsyncAdmin:62->lambda$testAdminWithAsyncAdmin$0:67
Admin method mergeRegionsSync should in AsyncAdmin too
{code}
[~zghaobac]:
Do you have suggestion on how the above should be addressed ?
I wonder if mergeRegionsSync should return CompletableFuture.
> Admin class should have a synchronous Admin#mergeRegions* method
> -
>
> Key: HBASE-16668
> URL: https://issues.apache.org/jira/browse/HBASE-16668
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Stephen Yuan Jiang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 16668.v1.txt
>
>
> In trunk from HBASE-14552, we have deprecated {{void Admin#mergeRegions}} (in
> 1.x this was an asynchronous call) and replaced it with {{Future
> Admin#mergeRegionsAsync}} which is clearly async.
> This leaves us only with the async version.
> We should have an easy way to make {{mergeRegions}} or an equivalant behave
> synchronously.
> For normal java Futures, we could just call the future's {{get()}} method.
> Unforutnately, the future this method returns doesn't follow java Future
> convention and throws Unimplemented operation when a plain {{get()}} is
> called and makes the api harder to use and read. We could make this future
> act more normally, and have the timeout throw an InterruptedException.
> Alternately, we could expose a new method in {{Admin}} that behaves
> synchronously such as {{HBaseAdmin#mergeRegionsSync}}. The caveat here is
> that we shouldn't use the name {{#mergeRegions}} since it exists in 1.x with
> async semantics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Attachment: (was: HBASE-20505.patch) > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Attachment: HBASE-20505.patch HBASE-20505-branch-1.patch > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458924#comment-16458924 ] Andrew Purtell commented on HBASE-20505: Initial patch. Support the new option for all test types. Specifying multiple families is not useful for checkAndXXX tests so always only utilize the first column family for those. For all other tests, assumption is specification of multiple CFs is a request to raise the work factor accordingly. > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Attachment: HBASE-20505.patch > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0 > > Attachments: HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16668) Admin class should have a synchronous Admin#mergeRegions* method
[
https://issues.apache.org/jira/browse/HBASE-16668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458900#comment-16458900
]
Hadoop QA commented on HBASE-16668:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
1s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} hbase-client: The patch generated 1 new + 170
unchanged - 0 fixed = 171 total (was 170) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
55s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 43s{color}
| {color:red} hbase-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 42m 17s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestInterfaceAlign |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-16668 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12921281/16668.v1.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7b89088562e1 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6d080762ef |
| maven | version: Apache Maven 3.5.3
[jira] [Updated] (HBASE-16668) Admin class should have a synchronous Admin#mergeRegions* method
[
https://issues.apache.org/jira/browse/HBASE-16668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16668:
---
Attachment: 16668.v1.txt
> Admin class should have a synchronous Admin#mergeRegions* method
> -
>
> Key: HBASE-16668
> URL: https://issues.apache.org/jira/browse/HBASE-16668
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Stephen Yuan Jiang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 16668.v1.txt
>
>
> In trunk from HBASE-14552, we have deprecated {{void Admin#mergeRegions}} (in
> 1.x this was an asynchronous call) and replaced it with {{Future
> Admin#mergeRegionsAsync}} which is clearly async.
> This leaves us only with the async version.
> We should have an easy way to make {{mergeRegions}} or an equivalant behave
> synchronously.
> For normal java Futures, we could just call the future's {{get()}} method.
> Unforutnately, the future this method returns doesn't follow java Future
> convention and throws Unimplemented operation when a plain {{get()}} is
> called and makes the api harder to use and read. We could make this future
> act more normally, and have the timeout throw an InterruptedException.
> Alternately, we could expose a new method in {{Admin}} that behaves
> synchronously such as {{HBaseAdmin#mergeRegionsSync}}. The caveat here is
> that we shouldn't use the name {{#mergeRegions}} since it exists in 1.x with
> async semantics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-16668) Admin class should have a synchronous Admin#mergeRegions* method
[
https://issues.apache.org/jira/browse/HBASE-16668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16668:
---
Status: Patch Available (was: Reopened)
> Admin class should have a synchronous Admin#mergeRegions* method
> -
>
> Key: HBASE-16668
> URL: https://issues.apache.org/jira/browse/HBASE-16668
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Stephen Yuan Jiang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 16668.v1.txt
>
>
> In trunk from HBASE-14552, we have deprecated {{void Admin#mergeRegions}} (in
> 1.x this was an asynchronous call) and replaced it with {{Future
> Admin#mergeRegionsAsync}} which is clearly async.
> This leaves us only with the async version.
> We should have an easy way to make {{mergeRegions}} or an equivalant behave
> synchronously.
> For normal java Futures, we could just call the future's {{get()}} method.
> Unforutnately, the future this method returns doesn't follow java Future
> convention and throws Unimplemented operation when a plain {{get()}} is
> called and makes the api harder to use and read. We could make this future
> act more normally, and have the timeout throw an InterruptedException.
> Alternately, we could expose a new method in {{Admin}} that behaves
> synchronously such as {{HBaseAdmin#mergeRegionsSync}}. The caveat here is
> that we shouldn't use the name {{#mergeRegions}} since it exists in 1.x with
> async semantics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20495) REST unit test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458799#comment-16458799
]
Ted Yu commented on HBASE-20495:
Ping [~busbey]
> REST unit test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-20495
> URL: https://issues.apache.org/jira/browse/HBASE-20495
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20495.v1.txt
>
>
> The following was first observed in the test output of rest.TestDeleteRow
> against hadoop3:
> {code}
> java.lang.NoClassDefFoundError:
> com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
> Caused by: java.lang.ClassNotFoundException:
> com.sun.jersey.core.spi.factory.AbstractRuntimeDelegate
> {code}
> This was due to the following transitive dependency on jersey 1.19:
> {code}
> [INFO] +- org.apache.hbase:hbase-testing-util:jar:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-zookeeper:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hbase:hbase-hadoop2-compat:test-jar:tests:2.0.0.3.0.0.0-SNAPSHOT:test
> [INFO] | +-
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.0.0:compile
> [INFO] | | \-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.0.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0:test
> [INFO] | | \- com.sun.jersey:jersey-server:jar:1.19:compile
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-13428) [DOC] Migration to hbase-2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13428: -- Fix Version/s: (was: 2.0.0) 3.0.0 > [DOC] Migration to hbase-2.0.0 > -- > > Key: HBASE-13428 > URL: https://issues.apache.org/jira/browse/HBASE-13428 > Project: HBase > Issue Type: Umbrella > Components: migration >Reporter: stack >Priority: Blocker > Fix For: 3.0.0 > > > Opening a 2.0 umbrella migration issue. Lets hang off this one any tools and > expectations migrating from 1.0 (or earlier) to 2.0. So far there are none > that I know of though there is an expectation in HBASE-13373 that hfiles are > at least major version 2 and minor version 3. Lets list all such > expectations, etc., here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-8075) HRegion preClose hook is getting called multiple times for single close
[
https://issues.apache.org/jira/browse/HBASE-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8075:
-
Fix Version/s: (was: 2.0.0)
3.0.0
> HRegion preClose hook is getting called multiple times for single close
> ---
>
> Key: HBASE-8075
> URL: https://issues.apache.org/jira/browse/HBASE-8075
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.3.0, 1.2.1, 1.1.4, 0.98.19, 2.0.0
>Reporter: rajeshbabu
>Assignee: Stephen Yuan Jiang
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HBASE-8075.v1-master.patch
>
>
> 1) HRegionServer#closeRegion(final RpcController controller,
> final CloseRegionRequest request)
> {code}
> // Can be null if we're calling close on a region that's not online
> final HRegion region = this.getFromOnlineRegions(encodedRegionName);
> if ((region != null) && (region .getCoprocessorHost() != null)) {
> region.getCoprocessorHost().preClose(false);
> }
> {code}
> 2) HRegionServer#closeRegion(String encodedName, final boolean abort,
> final boolean zk, final int versionOfClosingNode, final ServerName sn)
> {code}
> if ((actualRegion != null) && (actualRegion.getCoprocessorHost() !=
> null)) {
> try {
> actualRegion.getCoprocessorHost().preClose(false);
> } catch (IOException exp) {
> LOG.warn("Unable to close region: the coprocessor launched an error
> ", exp);
> return false;
> }
> }
> {code}
> 3) HRegion# private List doClose(
> final boolean abort, MonitoredTask status)
> {code}
> if (coprocessorHost != null) {
> status.setStatus("Running coprocessor pre-close hooks");
> this.coprocessorHost.preClose(abort);
> }
> {code}
> IMO 3rd one is enough and remaining two are not needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18474) Document how HRegion#doMiniBatchMutation is acquiring read row locks
[
https://issues.apache.org/jira/browse/HBASE-18474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18474:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> Document how HRegion#doMiniBatchMutation is acquiring read row locks
>
>
> Key: HBASE-18474
> URL: https://issues.apache.org/jira/browse/HBASE-18474
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Andrew Purtell
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0
>
>
> Looking at 1.3, HRegion#doMiniBatchMutation is acquiring read row locks in
> step 1.
> {code}
> // If we haven't got any rows in our batch, we should block to
>
> // get the next one.
>
> RowLock rowLock = null;
> try {
> rowLock = getRowLockInternal(mutation.getRow(), true);
> } catch (TimeoutIOException e) {
> // We will retry when other exceptions, but we should stop if we
> timeout .
> throw e;
> } catch (IOException ioe) {
> LOG.warn("Failed getting lock in batch put, row="
> + Bytes.toStringBinary(mutation.getRow()), ioe);
> }
> if (rowLock == null) {
> // We failed to grab another lock
>
> break; // stop acquiring more rows for this batch
>
> } else {
> acquiredRowLocks.add(rowLock);
> }
> {code}
> Other code paths that apply mutations are acquiring write locks.
> In HRegion#append
> {code}
> try {
> rowLock = getRowLockInternal(row, false);
> assert rowLock != null;
> ...
> {code}
> In HRegion#doIn
> {code}
> try {
> rowLock = getRowLockInternal(increment.getRow(), false);
> ...
> {code}
> In HRegion#checkAndMutate
> {code}
> // Lock row - note that doBatchMutate will relock this row if called
>
> RowLock rowLock = getRowLockInternal(get.getRow(), false);
> // wait for all previous transactions to complete (with lock held)
>
> mvcc.await();
> {code}
> In HRegion#processRowsWithLocks
> {code}
> // 2. Acquire the row lock(s)
>
> acquiredRowLocks = new ArrayList(rowsToLock.size());
> for (byte[] row : rowsToLock) {
> // Attempt to lock all involved rows, throw if any lock times out
>
> // use a writer lock for mixed reads and writes
>
> acquiredRowLocks.add(getRowLockInternal(row, false));
> }
> {code}
> and so on.
> What doMiniBatchMutation is doing looks wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18638) There are version-related dirty data caused by delete/ttl
[ https://issues.apache.org/jira/browse/HBASE-18638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18638: -- Fix Version/s: (was: 2.0.0) 3.0.0 > There are version-related dirty data caused by delete/ttl > - > > Key: HBASE-18638 > URL: https://issues.apache.org/jira/browse/HBASE-18638 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1, 1.2.6, 2.0.0-alpha-1 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0, 1.5.0 > > Attachments: HBASE-18638-ut.patch, HBASE-18638-ut.patch > > > |put_0(t0)| > |put_1(t1)| <-- the latest cell > If we call get, the put_1 will return. That is good. > If we call get after a delete, the put_0 will return. That is weird. The > put_0 is old data, and it should be dropped in flush. For client, put_0 > should not exist after the put_1 happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15809) Basic Replication WebUI
[ https://issues.apache.org/jira/browse/HBASE-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15809: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Basic Replication WebUI > --- > > Key: HBASE-15809 > URL: https://issues.apache.org/jira/browse/HBASE-15809 > Project: HBase > Issue Type: New Feature > Components: Replication, UI >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Jingyun Tian >Priority: Critical > Fix For: 3.0.0, 1.5.0 > > Attachments: HBASE-15809-v0.patch, HBASE-15809-v0.png, > HBASE-15809-v1.patch, rep_web_ui.zip > > > At the moment the only way to have some insight on replication from the webui > is looking at zkdump and metrics. > the basic information useful to get started debugging are: peer information > and the view of WALs offsets for each peer. > https://reviews.apache.org/r/47275/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16525) [2.0] Cell timestamps can be assigned out of order with sequenceId
[
https://issues.apache.org/jira/browse/HBASE-16525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16525:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> [2.0] Cell timestamps can be assigned out of order with sequenceId
> --
>
> Key: HBASE-16525
> URL: https://issues.apache.org/jira/browse/HBASE-16525
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Priority: Critical
> Fix For: 3.0.0
>
>
> While working on something else, noticed that in 2.0, we can end up with
> assigning timestamps out of order of sequenceId for the same row, thus ending
> with a case that a "later" mutation will not be visible due to an earlier
> mutation with a lower sequenceId. This can happen only in 2.0 code bases
> where we have the read-write lock based rowlocks. In 1.x, due to row locks
> being exclusive, we always order the cell timestamps in order of sequenceIds.
> In HRegion.doMiniBatchMutate(), step 2 is to assign cell timestamps:
> {code}
> // STEP 2. Update any LATEST_TIMESTAMP timestamps
> // We should record the timestamp only after we have acquired the
> rowLock,
> // otherwise, newer puts/deletes are not guaranteed to have a newer
> timestamp
> {code}
> If two transactions that modify the same row starts concurrently, and t1
> executes step 2 first, while t2 gets the sequenceId first, we can end up with
> two transactions where t1 has higher timestamp but lower seqId, and t2 lower
> timestamp but higher seqId.
> Not sure how big a problem is this. One can say that the "order" of
> transactions is the order they execute step 2 (assign cell timestamps) rather
> than assign sequenceIds.
> [~saint@gmail.com] , [~eclark] FYI.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-16668) Admin class should have a synchronous Admin#mergeRegions* method
[
https://issues.apache.org/jira/browse/HBASE-16668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16668:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> Admin class should have a synchronous Admin#mergeRegions* method
> -
>
> Key: HBASE-16668
> URL: https://issues.apache.org/jira/browse/HBASE-16668
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Stephen Yuan Jiang
>Priority: Major
> Fix For: 3.0.0
>
>
> In trunk from HBASE-14552, we have deprecated {{void Admin#mergeRegions}} (in
> 1.x this was an asynchronous call) and replaced it with {{Future
> Admin#mergeRegionsAsync}} which is clearly async.
> This leaves us only with the async version.
> We should have an easy way to make {{mergeRegions}} or an equivalant behave
> synchronously.
> For normal java Futures, we could just call the future's {{get()}} method.
> Unforutnately, the future this method returns doesn't follow java Future
> convention and throws Unimplemented operation when a plain {{get()}} is
> called and makes the api harder to use and read. We could make this future
> act more normally, and have the timeout throw an InterruptedException.
> Alternately, we could expose a new method in {{Admin}} that behaves
> synchronously such as {{HBaseAdmin#mergeRegionsSync}}. The caveat here is
> that we shouldn't use the name {{#mergeRegions}} since it exists in 1.x with
> async semantics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18495) [AMv2] Make methods isServerOnline() and isServerDead() in ServerManager mutually exclusive
[ https://issues.apache.org/jira/browse/HBASE-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18495: -- Fix Version/s: (was: 2.0.0) 3.0.0 > [AMv2] Make methods isServerOnline() and isServerDead() in ServerManager > mutually exclusive > --- > > Key: HBASE-18495 > URL: https://issues.apache.org/jira/browse/HBASE-18495 > Project: HBase > Issue Type: Bug > Components: amv2 >Affects Versions: 2.0.0 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 3.0.0 > > > During debugging HBASE-18366, it was found that > ServerManager.isServerOnline() and ServerManager.isServerDead() can return > true at the same time. This causes problems with the code depending on which > method is used. Making them mutually exclusive seems line right thing to do. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16071) The VisibilityLabelFilter and AccessControlFilter should not count the "delete cell"
[
https://issues.apache.org/jira/browse/HBASE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16071:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> The VisibilityLabelFilter and AccessControlFilter should not count the
> "delete cell"
>
>
> Key: HBASE-16071
> URL: https://issues.apache.org/jira/browse/HBASE-16071
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 1.3.3, 1.4.5
>
> Attachments: HBASE-16071-v1.patch, HBASE-16071-v2.patch,
> HBASE-16071-v3.patch
>
>
> The VisibilityLabelFilter will see and count the "delete cell" if the
> scan.isRaw() returns true, so the (put) cell will be skipped if it has lower
> version than "delete cell"
> The critical code is shown below:
> {code:title=VisibilityLabelFilter.java|borderStyle=solid}
> public ReturnCode filterKeyValue(Cell cell) throws IOException {
> if (curFamily.getBytes() == null
> || !(CellUtil.matchingFamily(cell, curFamily.getBytes(),
> curFamily.getOffset(),
> curFamily.getLength( {
> curFamily.set(cell.getFamilyArray(), cell.getFamilyOffset(),
> cell.getFamilyLength());
> // For this family, all the columns can have max of
> curFamilyMaxVersions versions. No need to
> // consider the older versions for visibility label check.
> // Ideally this should have been done at a lower layer by HBase (?)
> curFamilyMaxVersions = cfVsMaxVersions.get(curFamily);
> // Family is changed. Just unset curQualifier.
> curQualifier.unset();
> }
> if (curQualifier.getBytes() == null
> || !(CellUtil.matchingQualifier(cell, curQualifier.getBytes(),
> curQualifier.getOffset(),
> curQualifier.getLength( {
> curQualifier.set(cell.getQualifierArray(), cell.getQualifierOffset(),
> cell.getQualifierLength());
> curQualMetVersions = 0;
> }
> curQualMetVersions++;
> if (curQualMetVersions > curFamilyMaxVersions) {
> return ReturnCode.SKIP;
> }
> return this.expEvaluator.evaluate(cell) ? ReturnCode.INCLUDE :
> ReturnCode.SKIP;
> }
> {code}
> [VisibilityLabelFilter.java|https://github.com/apache/hbase/blob/d7a4499dfc8b3936a0eca867589fc2b23b597866/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelFilter.java]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18359) CoprocessorHConnection#getConnectionForEnvironment should read config from CoprocessorEnvironment
[
https://issues.apache.org/jira/browse/HBASE-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18359:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> CoprocessorHConnection#getConnectionForEnvironment should read config from
> CoprocessorEnvironment
> -
>
> Key: HBASE-18359
> URL: https://issues.apache.org/jira/browse/HBASE-18359
> Project: HBase
> Issue Type: Bug
>Reporter: Samarth Jain
>Priority: Major
> Fix For: 3.0.0
>
>
> It seems like the method getConnectionForEnvironment isn't doing the right
> thing when it is creating a CoprocessorHConnection by reading the config from
> HRegionServer and not from the env passed in.
> If coprocessors want to use a CoprocessorHConnection with some custom config
> settings, then they have no option but to configure it in the hbase-site.xml
> of the region servers. This isn't ideal as a lot of times these "global"
> level configs can have side effects. See PHOENIX-3974 as an example where
> configuring ServerRpcControllerFactory (a Phoenix implementation of
> RpcControllerFactory) could result in deadlocks. Or PHOENIX-3983 where
> presence of this global config causes our index rebuild code to incorrectly
> use handlers it shouldn't.
> If the CoprocessorHConnection created through getConnectionForEnvironment API
> used the CoprocessorEnvironment config, then it would allow co-processors to
> pass in their own config without needing to configure them in hbase-site.xml.
> The change would be simple. Basically change the below
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection((HRegionServer) services);
> }
> {code}
> to
> {code}
> if (services instanceof HRegionServer) {
> return new CoprocessorHConnection(env.getConfiguration(),
> (HRegionServer) services);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-17919) HBase 2.x over hadoop 3.x umbrella
[ https://issues.apache.org/jira/browse/HBASE-17919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17919: -- Fix Version/s: (was: 2.0.0) 3.0.0 > HBase 2.x over hadoop 3.x umbrella > --- > > Key: HBASE-17919 > URL: https://issues.apache.org/jira/browse/HBASE-17919 > Project: HBase > Issue Type: Umbrella > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Priority: Critical > Fix For: 3.0.0 > > > We should try to get hbase 2.x branch working against the recently release > hadoop 3.0.0 alphas. These days 3.0.0-alpha2 is the latest. > HBASE-16733 and HBASE-17593 got the compile level checks in but we should > progress to getting unit tests to pass and a build against hadoop3 up. > This umbrella issue will capture issues around this project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19307) Ignore the case of string when lookup the enum property from configuration
[ https://issues.apache.org/jira/browse/HBASE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19307: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Ignore the case of string when lookup the enum property from configuration > -- > > Key: HBASE-19307 > URL: https://issues.apache.org/jira/browse/HBASE-19307 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Sergey Soldatov >Priority: Critical > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18811) Make filter IA.LimitedPrivate and IS.Stable
[ https://issues.apache.org/jira/browse/HBASE-18811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18811: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Make filter IA.LimitedPrivate and IS.Stable > > > Key: HBASE-18811 > URL: https://issues.apache.org/jira/browse/HBASE-18811 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-18811.v0.patch > > > We have many powerful callback functions to help user to build amazing > application/services. The most of functions are declared as IA.LimitedPrivate > excluding the filters. As i see it, the IA.LimitedPrivate will make the > improvement of filter more flexible. Also, we can introduce more server-side > components to filters. In conclusion, we should consider adding the limited > private level for filter. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19529) Handle null states in AM
[ https://issues.apache.org/jira/browse/HBASE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19529: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Handle null states in AM > > > Key: HBASE-19529 > URL: https://issues.apache.org/jira/browse/HBASE-19529 > Project: HBase > Issue Type: Bug >Reporter: Appy >Priority: Critical > Fix For: 3.0.0 > > > From debugging in HBASE-19457, found some questions that need concrete > answers: > 1) What does a region state of null in meta means? Currently AM treats it as > OFFLINE > 2) What does a table state of null in meta means? Currently TSM treats it as > ENABLED. > More importantly, we need to fix holes in AM so that our state machine is > well defined and doesn't end up in these uncertainties. > Figuring out answers to above questions will help in that direction. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18494) [AMv2] Modify LoadBalancer to consider highest versioned Region Servers as favorites for system table regions
[ https://issues.apache.org/jira/browse/HBASE-18494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18494: -- Fix Version/s: (was: 2.0.0) 3.0.0 > [AMv2] Modify LoadBalancer to consider highest versioned Region Servers as > favorites for system table regions > - > > Key: HBASE-18494 > URL: https://issues.apache.org/jira/browse/HBASE-18494 > Project: HBase > Issue Type: Bug > Components: amv2 >Affects Versions: 2.0.0 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Labels: rolling_upgrade > Fix For: 3.0.0 > > > Modify LoadBalancer to consider highest versioned Region Servers as favorites > for system table regions. Will help with rolling upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15041) Clean up javadoc errors and reenable jdk8 linter
[ https://issues.apache.org/jira/browse/HBASE-15041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15041: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Clean up javadoc errors and reenable jdk8 linter > > > Key: HBASE-15041 > URL: https://issues.apache.org/jira/browse/HBASE-15041 > Project: HBase > Issue Type: Umbrella > Components: build, documentation >Affects Versions: 1.2.0, 1.3.0, 2.0.0 >Reporter: Sean Busbey >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 3.0.0 > > > umbrella to clean up our various errors according to the jdk8 javadoc linter. > plan is a sub-task per module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17762) Add logging to HBaseAdmin for user initiated tasks
[ https://issues.apache.org/jira/browse/HBASE-17762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17762: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Add logging to HBaseAdmin for user initiated tasks > -- > > Key: HBASE-17762 > URL: https://issues.apache.org/jira/browse/HBASE-17762 > Project: HBase > Issue Type: Task >Reporter: churro morales >Assignee: churro morales >Priority: Major > Fix For: 3.0.0, 1.5.0 > > Attachments: HBASE-17762.patch, HBASE-17762.v1.patch > > > Things like auditing a forced major compaction are really useful and right > now there is no logging when this is triggered. Other actions may require > logging as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19595) explicitly discourage public user from creating their Public pojo
[ https://issues.apache.org/jira/browse/HBASE-19595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19595: -- Fix Version/s: (was: 2.0.0) 3.0.0 > explicitly discourage public user from creating their Public pojo > - > > Key: HBASE-19595 > URL: https://issues.apache.org/jira/browse/HBASE-19595 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Priority: Major > Fix For: 3.0.0 > > > HBASE-19496 make both of ReplicationLoadSink and ReplicationLoadSource be > IA.Public since they are exposed by other Public class (ServerMetrics). > Currently, their constructor are annotated with IA.Private to discourage user > from creating their ReplicationLoadSink and ReplicationLoadSource. However, > that is too implicit as users have to trace the source code to "see" the > warnings. > I believe that many pojo in hbase have same issue. We have some kinds of > coding style to warn user not to create their Public pojo. > # IA.Private constructor. for example, ReplicationLoadSink and > ReplicationLoadSource > # pure interface. for example, TableDescriptor, ColumnFamilyDescriptor, > ClusterMetrics > # final class and package private constructor. for example, CacheEvictionStats > All of them have pros and cons. #1 is a simple solution but it is too > implicit for user. #2 will cause the BC issue (see HBASE-19535). #3 is good > solution to prevent user from using the constructor or extending the pojo. > However, it may limit us also. (mockito or extend the pojo internally) > Any suggestions? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16025) Cache table state to reduce load on META
[ https://issues.apache.org/jira/browse/HBASE-16025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16025: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Cache table state to reduce load on META > > > Key: HBASE-16025 > URL: https://issues.apache.org/jira/browse/HBASE-16025 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 3.0.0 > > > HBASE-12035 moved keeping table enabled/disabled state from ZooKeeper into > hbase:meta. When we retry operations on the client, we check table state in > order to return a specific message if the table is disabled. This means that > in master we will be going back to meta for every retry, even if a region's > location has not changed. This is going to cause performance issues when a > cluster is already loaded, ie. in cases where regionservers may be returning > CallQueueTooBigException. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15317) document release announcement template
[ https://issues.apache.org/jira/browse/HBASE-15317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15317: -- Fix Version/s: (was: 2.0.0) 3.0.0 > document release announcement template > -- > > Key: HBASE-15317 > URL: https://issues.apache.org/jira/browse/HBASE-15317 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0 > > > Our release docs should include a release announcement template for folks to > use (and suggested email lists to send to) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-14223) Meta WALs are not cleared if meta region was closed and RS aborts
[
https://issues.apache.org/jira/browse/HBASE-14223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14223:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> Meta WALs are not cleared if meta region was closed and RS aborts
> -
>
> Key: HBASE-14223
> URL: https://issues.apache.org/jira/browse/HBASE-14223
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.5
>
> Attachments: HBASE-14223logs, hbase-14223_v0.patch,
> hbase-14223_v1-branch-1.patch, hbase-14223_v2-branch-1.patch,
> hbase-14223_v3-branch-1.patch, hbase-14223_v3-branch-1.patch,
> hbase-14223_v3-master.patch
>
>
> When an RS opens meta, and later closes it, the WAL(FSHlog) is not closed.
> The last WAL file just sits there in the RS WAL directory. If RS stops
> gracefully, the WAL file for meta is deleted. Otherwise if RS aborts, WAL for
> meta is not cleaned. It is also not split (which is correct) since master
> determines that the RS no longer hosts meta at the time of RS abort.
> From a cluster after running ITBLL with CM, I see a lot of {{-splitting}}
> directories left uncleaned:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs
> Found 31 items
> drwxr-xr-x - hbase hadoop 0 2015-06-05 01:14
> /apps/hbase/data/WALs/hregion-58203265
> drwxr-xr-x - hbase hadoop 0 2015-06-05 07:54
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433489308745-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 09:28
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433494382959-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 10:01
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433498252205-splitting
> ...
> {code}
> The directories contain WALs from meta:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting
> Found 2 items
> -rw-r--r-- 3 hbase hadoop 201608 2015-06-05 03:15
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta
> -rw-r--r-- 3 hbase hadoop 44420 2015-06-05 04:36
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta
> {code}
> The RS hosted the meta region for some time:
> {code}
> 2015-06-05 03:14:28,692 INFO [PostOpenDeployTasks:1588230740]
> zookeeper.MetaTableLocator: Setting hbase:meta region location in ZooKeeper
> as os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285
> ...
> 2015-06-05 03:15:17,302 INFO
> [RS_CLOSE_META-os-enis-dal-test-jun-4-5:16020-0] regionserver.HRegion: Closed
> hbase:meta,,1.1588230740
> {code}
> In between, a WAL is created:
> {code}
> 2015-06-05 03:15:11,707 INFO
> [RS_OPEN_META-os-enis-dal-test-jun-4-5:16020-0-MetaLogRoller] wal.FSHLog:
> Rolled WAL
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta
> with entries=385, filesize=196.88 KB; new WAL
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta
> {code}
> When CM killed the region server later master did not see these WAL files:
> {code}
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:46,075
> INFO [MASTER_SERVER_OPERATIONS-os-enis-dal-test-jun-4-3:16000-0]
> master.SplitLogManager: started splitting 2 logs in
> [hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting]
> for [os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285]
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:47,300
> INFO [main-EventThread] wal.WALSplitter: Archived processed log
> hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.1433475074436
> to
> hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/oldWALs/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.1433475074436
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:50,497
> INFO
[jira] [Updated] (HBASE-19535) Introduce a new annotation to say there is no BC guarantee in the inheritance
[ https://issues.apache.org/jira/browse/HBASE-19535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19535: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Introduce a new annotation to say there is no BC guarantee in the inheritance > - > > Key: HBASE-19535 > URL: https://issues.apache.org/jira/browse/HBASE-19535 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0 > > > We have added many Public "read-only" interface in 2.0 to save user from the > dangerous operations. For example, TableDescriptor, ColumnFamilyDescriptor, > Region, Store, etc. However, this change make us be hard to enhance these > interface (ie. add the new APIs) because the BC will be broken for the user > having the subclass of these interfaces. In contrast with Cell, Filter, and > coprocessor, these new interfaces are NOT designed to be a extendable class > for user but we have no explicit caveat. As I see it, it is necessary to > introduce an new annotation to explicitly say "you CAN extend this Public > class with the BC guarantee". On the other hand, user should not extend the > Public classes which don't have the such annotation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19586) Figure how to enable compression by default (fallbacks if native is missing, etc.)
[ https://issues.apache.org/jira/browse/HBASE-19586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19586: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Figure how to enable compression by default (fallbacks if native is missing, > etc.) > -- > > Key: HBASE-19586 > URL: https://issues.apache.org/jira/browse/HBASE-19586 > Project: HBase > Issue Type: Sub-task > Components: defaults >Reporter: stack >Priority: Major > Fix For: 3.0.0 > > > See parent issue where the benefits of enabling compression are brought up > (again!). Figure how we can make it work out of the box rather than expect > user set it up. Parking this issue to look at it before we release 2.0.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15154) Master puts up a blockcache instance
[ https://issues.apache.org/jira/browse/HBASE-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15154: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Master puts up a blockcache instance > > > Key: HBASE-15154 > URL: https://issues.apache.org/jira/browse/HBASE-15154 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Fix For: 3.0.0 > > > Master is putting up a blockcache instance. It shouldn't. This issue comes of > the unification of Master and RegionServer where Master is a subclass of > RegionServer. Our [~jmspaggi] found out the hard way today when he tried to > bring up a cluster with a large offheap cache only the master member had been > given less memory. > Marking critical rather than blocker because there is a means of configuring > your way out of this little entanglement; set hbase.bucketcache.size to zero > in the master config only. > If you want to set direct memory for regionservers only and not on the > master, you should do the following: > Set HBASE_OFFHEAPSIZE=0G > ... and then turn it on for RegionServers only: > HBASE_REGIONSERVER_OPTS="HBASE_REGIONSERVER_OPTS > -XX:MaxDirectMemorySize=XXXG" where XXX is size of the direct memory you want > to allocate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18785) Blocker doc issues for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-18785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18785: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Blocker doc issues for 2.0.0 > > > Key: HBASE-18785 > URL: https://issues.apache.org/jira/browse/HBASE-18785 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 3.0.0 > > > Hang blocker documentation issues off this one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-14935) ReverseStoreScanner can check for scanner reset before seekToPreviousRow and backwardSeek.
[ https://issues.apache.org/jira/browse/HBASE-14935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14935: -- Fix Version/s: (was: 2.0.0) 3.0.0 > ReverseStoreScanner can check for scanner reset before seekToPreviousRow and > backwardSeek. > -- > > Key: HBASE-14935 > URL: https://issues.apache.org/jira/browse/HBASE-14935 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-14935.patch > > > ReversedStoreScanner#seekToPreviousRow and backwardSeek can check if the heap > has to be reset before doing reseek. There is no bug before this but just an > added check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19663) site build fails complaining "javadoc: error - class file for javax.annotation.meta.TypeQualifierNickname not found"
[
https://issues.apache.org/jira/browse/HBASE-19663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19663:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> site build fails complaining "javadoc: error - class file for
> javax.annotation.meta.TypeQualifierNickname not found"
>
>
> Key: HBASE-19663
> URL: https://issues.apache.org/jira/browse/HBASE-19663
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: script.sh
>
>
> Cryptic failure trying to build beta-1 RC. Fails like this:
> {code}
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 03:54 min
> [INFO] Finished at: 2017-12-29T01:13:15-08:00
> [INFO] Final Memory: 381M/9165M
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project
> hbase: Error generating maven-javadoc-plugin:2.10.3:aggregate:
> [ERROR] Exit code: 1 - warning: unknown enum constant When.ALWAYS
> [ERROR] reason: class file for javax.annotation.meta.When not found
> [ERROR] warning: unknown enum constant When.UNKNOWN
> [ERROR] warning: unknown enum constant When.MAYBE
> [ERROR]
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
> warning - Tag @link: malformed: "#matchingRows(Cell, byte[]))"
> [ERROR]
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
> warning - Tag @link: reference not found: #matchingRows(Cell, byte[]))
> [ERROR]
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
> warning - Tag @link: reference not found: #matchingRows(Cell, byte[]))
> [ERROR] javadoc: warning - Class javax.annotation.Nonnull not found.
> [ERROR] javadoc: error - class file for
> javax.annotation.meta.TypeQualifierNickname not found
> [ERROR]
> [ERROR] Command line was: /home/stack/bin/jdk1.8.0_151/jre/../bin/javadoc
> -J-Xmx2G @options @packages
> [ERROR]
> [ERROR] Refer to the generated Javadoc files in
> '/home/stack/hbase.git/target/site/apidocs' dir.
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> {code}
> javax.annotation.meta.TypeQualifierNickname is out of jsr305 but we don't
> include this anywhere according to mvn dependency.
> Happens building the User API both test and main.
> Excluding these lines gets us passing again:
> {code}
> 3511
> 3512
> org.apache.yetus.audience.tools.IncludePublicAnnotationsStandardDoclet
> 3513
> 3514
> 3515 org.apache.yetus
> 3516 audience-annotations
> 3517 ${audience-annotations.version}
> 3518
> + 3519 true
> {code}
> Tried upgrading to newer mvn site (ours is three years old) but that a
> different set of problems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19283) InfoServer should allow configuring of cipher suite when TLS is enabled
[ https://issues.apache.org/jira/browse/HBASE-19283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19283: -- Fix Version/s: (was: 2.0.0) 3.0.0 > InfoServer should allow configuring of cipher suite when TLS is enabled > --- > > Key: HBASE-19283 > URL: https://issues.apache.org/jira/browse/HBASE-19283 > Project: HBase > Issue Type: Improvement > Components: master, regionserver, security >Affects Versions: 2.0.0-alpha-3 >Reporter: Sean Busbey >Priority: Major > Fix For: 3.0.0 > > > When updating to jetty 9, we gained configuration knobs for restricting TLS > protocols and cipher suites for the REST service and the Thrift service when > running over HTTP. > We should provide an equivalent configuration knob for the HttpServer class > that's used by the informational web page on other roles. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15539) HBase Client region location is expensive
[ https://issues.apache.org/jira/browse/HBASE-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15539: -- Fix Version/s: (was: 2.0.0) 3.0.0 > HBase Client region location is expensive > -- > > Key: HBASE-15539 > URL: https://issues.apache.org/jira/browse/HBASE-15539 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Vladimir Rodionov >Assignee: Mikhail Antonov >Priority: Major > Fix For: 3.0.0 > > > ConnectionImplementation.locateRegion and MetaCache.getTableLocations are hot > spots in a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16513) Documentation for new RowIndex DataBlockEncoding
[ https://issues.apache.org/jira/browse/HBASE-16513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16513: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Documentation for new RowIndex DataBlockEncoding > > > Key: HBASE-16513 > URL: https://issues.apache.org/jira/browse/HBASE-16513 > Project: HBase > Issue Type: Improvement > Components: documentation, Performance >Reporter: binlijin >Priority: Major > Fix For: 3.0.0, 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19444) RSGroups test units cannot be concurrently executed
[ https://issues.apache.org/jira/browse/HBASE-19444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19444: -- Fix Version/s: (was: 2.0.0) 3.0.0 > RSGroups test units cannot be concurrently executed > --- > > Key: HBASE-19444 > URL: https://issues.apache.org/jira/browse/HBASE-19444 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 1.5.0, 1.4.5 > > > TestRSGroups and friends cannot be concurrently executed or they are very > likely to flake, failing with constraint exceptions. If executed serially all > units pass. Fix for concurrent execution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15654) Optimize client's MetaCache handling
[ https://issues.apache.org/jira/browse/HBASE-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15654: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Optimize client's MetaCache handling > > > Key: HBASE-15654 > URL: https://issues.apache.org/jira/browse/HBASE-15654 > Project: HBase > Issue Type: Umbrella > Components: Client >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Major > Fix For: 3.0.0, 1.5.0 > > > This is an umbrella jira to track all individual issues, bugfixes and small > optimizations around MetaCache (region locations cache) in the client. > Motivation is that under the load one could see a spikes in the number of > requests going to meta - reaching tens of thousands requests per second. > That covers issues when we clear entries from location cache unnecessary, as > well as when we do more lookups than necessary when entries are legitimately > evicted. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16537) Add tests to verify create/modify table region reopen with missing coprocessor
[ https://issues.apache.org/jira/browse/HBASE-16537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16537: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Add tests to verify create/modify table region reopen with missing > coprocessor > --- > > Key: HBASE-16537 > URL: https://issues.apache.org/jira/browse/HBASE-16537 > Project: HBase > Issue Type: Bug > Components: master, proc-v2, Region Assignment >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Priority: Major > Fix For: 3.0.0 > > > Add tests to cover the case where a table is created or modified and a non > existent coprocessor is set. > This should result in a nice exception to the user providing the information > about the coprocessor not found. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18152) [AMv2] Corrupt Procedure WAL file; procedure data stored out of order
[ https://issues.apache.org/jira/browse/HBASE-18152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18152: -- Fix Version/s: (was: 2.0.0) 3.0.0 > [AMv2] Corrupt Procedure WAL file; procedure data stored out of order > - > > Key: HBASE-18152 > URL: https://issues.apache.org/jira/browse/HBASE-18152 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 3.0.0 > > Attachments: HBASE-18152.master.001.patch, > hbase-hbase-master-ctr-e138-1518143905142-221855-01-02.hwx.site.log.gz, > pv2-0036.log, pv2-0047.log, > reading_bad_wal.patch > > > I've seen corruption from time-to-time testing. Its rare enough. Often we > can get over it but sometimes we can't. It took me a while to capture an > instance of corruption. Turns out we are write to the WAL out-of-order which > undoes a basic tenet; that WAL content is ordered in line w/ execution. > Below I'll post a corrupt WAL. > Looking at the write-side, there is a lot going on. I'm not clear on how we > could write out of order. Will try and get more insight. Meantime parking > this issue here to fill data into. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17536) User documentation for coprocessor metrics
[ https://issues.apache.org/jira/browse/HBASE-17536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17536: -- Fix Version/s: (was: 2.0.0) 3.0.0 > User documentation for coprocessor metrics > -- > > Key: HBASE-17536 > URL: https://issues.apache.org/jira/browse/HBASE-17536 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Major > Fix For: 3.0.0 > > > After HBASE-9774, we need to document the API in the book. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17822) Set maxBadRows and outputDirectory option for VerifyReplication
[ https://issues.apache.org/jira/browse/HBASE-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17822: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Set maxBadRows and outputDirectory option for VerifyReplication > > > Key: HBASE-17822 > URL: https://issues.apache.org/jira/browse/HBASE-17822 > Project: HBase > Issue Type: Improvement > Components: mapreduce, Operability >Reporter: Shibin Zhang >Priority: Minor > Fix For: 3.0.0 > > Attachments: 2017-06-30_110905.png, 2017-06-30_110915.png, > HBASE-17822-V1.patch, HBASE-17822-master.patch > > > Currently,this tool will print too many rowkey as badrows if source and peer > table have many inconsistent row.So,it is necessay to set maxBadRows to print. > Also,look for badrows rowkey is inconvenient in MR job log .It might be > useful to set a reduce to aggregate badrowkeys which will be print in MR job > output file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19533) How to do controlled shutdown in branch-2?
[ https://issues.apache.org/jira/browse/HBASE-19533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19533: -- Fix Version/s: (was: 2.0.0) 3.0.0 > How to do controlled shutdown in branch-2? > -- > > Key: HBASE-19533 > URL: https://issues.apache.org/jira/browse/HBASE-19533 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 3.0.0 > > > Before HBASE-18946, setting shutdown of a cluster, the Master would exit > immediately. RegionServers would run region closes and then try and notify > the Master of the close and would spew exceptions that the Master was > unreachable. > This is different to how branch-1 used to do it. It used to keep Master up > and it would be like the captain of the ship, the last to go down. As of > HBASE-18946, this is again the case but there are still open issues. > # Usually Master does all open and close of regions. On cluster shutdown, it > is the one time where the Regions run the region close. Currently, the > regions report the close to the Master which disregards the message since it > did not start the region closes. Should we do different? Try and update state > in hbase:meta setting it to CLOSE? We might not be able to write CLOSE for > all regions since hbase:meta will be closing too (the RS that is hosting > hbase:meta will close it last but that may not be enough). > # Should the Master run the cluster shutdown sending out close for all > regions? What if cluster of 1M regions? Untenable? Send a message per server? > That might be better. > Anyways, this needs attention. Filing issue in meantime. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19515) Region server left in online servers list forever if it went down after registering to master and before creating ephemeral node
[ https://issues.apache.org/jira/browse/HBASE-19515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19515: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Region server left in online servers list forever if it went down after > registering to master and before creating ephemeral node > > > Key: HBASE-19515 > URL: https://issues.apache.org/jira/browse/HBASE-19515 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: stack >Priority: Critical > Fix For: 3.0.0 > > > This one is interesting. It was supposedly fixed long time ago back in > HBASE-9593 (The issue has same subject as this one) but there was a problem > w/ the fix reported later, post-commit, long after the issue was closed. The > 'fix' was registering ephemeral node in ZK BEFORE reporting in to the Master > for the first time. The problem w/ this approach is that the Master tells the > RS what name it should use reporting in. If we register in ZK before we talk > to the Master, the name in ZK and the one the RS ends up using could deviate. > In hbase2, we do the right thing registering the ephemeral node after we > report to the Master. So, the issue reported in HBASE-9593, that a RS that > dies between reporting to master and registering up in ZK, stays registered > at the Master for ever is back; we'll keep trying to assign it regions. Its a > real problem. > That hbase2 has this issue has been suppressed up until now. The test that > was written for HBASE-9593, TestRSKilledWhenInitializing, is a good test but > a little sloppy. It puts up two RSs aborting one only after registering at > the Master before posting to ZK. That leaves one healthy server up. It is > hosting hbase:meta. This is enough for the test to bluster through. The only > assign it does is namespace table. It goes to the hbase:meta server. If the > test created a new table and did roundrobin, it'd fail. > After HBASE-18946, where we do round robin on table create -- a desirable > attribute -- via the balancer so all is kosher, the test > TestRSKilledWhenInitializing now starts to fail because we chose the hobbled > server most of the time. > So, this issue is about fixing the original issue properly for hbase2. We > don't have a timeout on assign in AMv2, not yet, that might be the fix, or > perhaps a double report before we online a server with the second report > coming in after ZK goes up (or we stop doing ephemeral nodes for RS up in ZK > and just rely on heartbeats). > Making this a critical issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15728) Add remaining per-table region / store / flush / compaction related metrics
[ https://issues.apache.org/jira/browse/HBASE-15728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15728: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Add remaining per-table region / store / flush / compaction related metrics > > > Key: HBASE-15728 > URL: https://issues.apache.org/jira/browse/HBASE-15728 > Project: HBase > Issue Type: Sub-task > Components: metrics >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Major > Fix For: 3.0.0, 1.5.0 > > Attachments: hbase-15728_v1.patch > > > Continuing on the work for per-table metrics, HBASE-15518 and HBASE-15671. > We need to add some remaining metrics at the per-table level, so that we will > have the same metrics reported at the per-regionserver, per-region and > per-table levels. > After this patch, most of the metrics at the RS and all of the per-region > level are also reported at the per-table level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19308) batch processing does not consider partial success
[ https://issues.apache.org/jira/browse/HBASE-19308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19308: -- Fix Version/s: (was: 2.0.0) 3.0.0 > batch processing does not consider partial success > -- > > Key: HBASE-19308 > URL: https://issues.apache.org/jira/browse/HBASE-19308 > Project: HBase > Issue Type: Bug >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Major > Fix For: 3.0.0 > > > While working on HBASE-19163, some unittest failures exposes an interesting > issue. > When client sends a batch, at the region server, it can be divided into > multiple minibatchs. If one of minibatches runs into some exception(resource > for an example), the exception gets back to the client and it thinks that the > whole batch fails. The client will send the whole batch again will cause the > already-succeeded minibatches to be processed again. We need to have > mechanism to allow partial success of the batch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19701) Close without justification following succesful open
[
https://issues.apache.org/jira/browse/HBASE-19701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19701:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> Close without justification following succesful open
>
>
> Key: HBASE-19701
> URL: https://issues.apache.org/jira/browse/HBASE-19701
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Critical
> Fix For: 3.0.0
>
>
> [~jmspaggi] conjured an interesting condition where we close a region soon
> after open WITHOUT seemingly saying why (It looks like Master is asking for
> region CLOSE but that is not clear looking at RegionServer log).
> Here is log snippet from https://pastebin.com/0r76Y6ap (in case the pastebin
> evaporates)
> {code}
>
> 2017-12-31 09:54:20,864 INFO
> [PostOpenDeployTasks:f49f3cbb7f3db4cf96c7eb3b0cf83869]
> regionserver.HRegionServer: Post open deploy tasks for
> TestTable,0408944640,1505391191559.f49f3cbb7f3db4cf96c7eb3b0cf83869.
> 2017-12-31 09:54:20,870 INFO
> [StoreOpener-330f09f4a0eaf26811c320fbf1b14e70-1]
> regionserver.CompactingMemStore: Setting in-memory flush size threshold to
> 13421772 and immutable segments index to be of type CHUNK_MAP
> 2017-12-31 09:54:20,870 INFO
> [StoreOpener-330f09f4a0eaf26811c320fbf1b14e70-1] regionserver.HStore:
> Memstore class name is org.apache.hadoop.hbase.regionserver.CompactingMemStore
> 2017-12-31 09:54:20,870 INFO
> [StoreOpener-330f09f4a0eaf26811c320fbf1b14e70-1] hfile.CacheConfig: Created
> cacheConfig for info: blockCache=LruBlockCache{blockCount=0,
> currentSize=2454760, freeSize=3347745560, maxSize=3350200320,
> heapSize=2454760, minSize=3182690304, minFactor=0.95, multiSize=1591345152,
> multiFactor=0.5, singleSize=795672576, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2017-12-31 09:54:20,872 INFO
> [StoreOpener-330f09f4a0eaf26811c320fbf1b14e70-1]
> compactions.CompactionConfiguration: size [134217728, 9223372036854775807,
> 9223372036854775807); files [3, 10); ratio 1,20; off-peak ratio 5,00;
> throttle point 2684354560; major period 60480, major jitter 0,50, min
> locality to compact 0,00; tiered compaction: max_age 9223372036854775807,
> incoming window min 6, compaction policy for tiered window
> org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy,
> single output for minor true, compaction window factory
> org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory
> 2017-12-31 09:54:20,903 INFO
> [StoreOpener-166b9c45d7724f72fd126adb4445d6ec-1]
> regionserver.CompactingMemStore: Setting in-memory flush size threshold to
> 13421772 and immutable segments index to be of type CHUNK_MAP
> 2017-12-31 09:54:20,904 INFO
> [StoreOpener-166b9c45d7724f72fd126adb4445d6ec-1] regionserver.HStore:
> Memstore class name is org.apache.hadoop.hbase.regionserver.CompactingMemStore
> 2017-12-31 09:54:20,904 INFO
> [StoreOpener-166b9c45d7724f72fd126adb4445d6ec-1] hfile.CacheConfig: Created
> cacheConfig for info: blockCache=LruBlockCache{blockCount=0,
> currentSize=2454760, freeSize=3347745560, maxSize=3350200320,
> heapSize=2454760, minSize=3182690304, minFactor=0.95, multiSize=1591345152,
> multiFactor=0.5, singleSize=795672576, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
> cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> 2017-12-31 09:54:20,905 INFO
> [StoreOpener-166b9c45d7724f72fd126adb4445d6ec-1]
> compactions.CompactionConfiguration: size [134217728, 9223372036854775807,
> 9223372036854775807); files [3, 10); ratio 1,20; off-peak ratio 5,00;
> throttle point 2684354560; major period 60480, major jitter 0,50, min
> locality to compact 0,00; tiered compaction: max_age 9223372036854775807,
> incoming window min 6, compaction policy for tiered window
> org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy,
> single output for minor true, compaction window factory
> org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory
> 2017-12-31 09:54:20,929 INFO [RS_OPEN_REGION-node1:16020-1]
> regionserver.HRegion: Setting FlushNonSloppyStoresFirstPolicy for the
> region=TestTable,0262144000,1505391191559.166b9c45d7724f72fd126adb4445d6ec.
> 2017-12-31 09:54:20,956 INFO [RS_OPEN_REGION-node1:16020-0]
> regionserver.HRegion: Setting FlushNonSloppyStoresFirstPolicy for the
> region=TestTable,0188743680,1505391191559.330f09f4a0eaf26811c320fbf1b14e70.
> 2017-12-31
[jira] [Updated] (HBASE-15648) Reduce number of concurrent region location lookups when MetaCache entry is cleared
[ https://issues.apache.org/jira/browse/HBASE-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15648: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Reduce number of concurrent region location lookups when MetaCache entry is > cleared > --- > > Key: HBASE-15648 > URL: https://issues.apache.org/jira/browse/HBASE-15648 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 1.2.0, 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Major > Fix For: 3.0.0, 1.5.0 > > Attachments: HBASE-15648-branch-1.3.v1.patch > > > It seems in HConnectionImplementation#locateRegionInMeta if region location > is removed from the cache, with large number of client threads we could have > many of them getting cache miss and doing meta scan, which looks unnecessary > - we could empty mechanism similar to what we have in IdLock in HFileReader > to fetch the block to cache, do ensure that if one thread is already looking > up location for region R1, other threads who need it's location wait until > first thread finishes his work. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15677) FailedServerException shouldn't clear MetaCache
[ https://issues.apache.org/jira/browse/HBASE-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15677: -- Fix Version/s: (was: 2.0.0) 3.0.0 > FailedServerException shouldn't clear MetaCache > --- > > Key: HBASE-15677 > URL: https://issues.apache.org/jira/browse/HBASE-15677 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Major > Fix For: 3.0.0, 1.5.0 > > > Right now FailedServerException clears meta cache. Seems like it's > unnecessary (if we hit that, someone has already gotten some network/remote > error in the first place and invalidated located cache for us), and seems it > could lead to unnecessary drops, as FailedServers cache has default TTL of 2 > seconds, so we can encounter situation like this: > - thread T1 hit network error and cleared the cache, put server in failed > server list > - thread T2 tries to get it's request in and gets FailedServerException > - thread T1 does meta scan to populate the cache > - thread T2 clears the cache after it's got FSE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17553) Make a 2.0.0 Release
[ https://issues.apache.org/jira/browse/HBASE-17553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17553: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Make a 2.0.0 Release > > > Key: HBASE-17553 > URL: https://issues.apache.org/jira/browse/HBASE-17553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 3.0.0 > > > Umbrella issue to keep account of tasks to make a 2.0.0 release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15121) ConnectionImplementation#locateRegionInMeta() issue when master is restarted
[
https://issues.apache.org/jira/browse/HBASE-15121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-15121:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> ConnectionImplementation#locateRegionInMeta() issue when master is restarted
>
>
> Key: HBASE-15121
> URL: https://issues.apache.org/jira/browse/HBASE-15121
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Samir Ahmic
>Assignee: Samir Ahmic
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-15121-v0.patch, HBASE-15121-v0.patch
>
>
> I notice this issue while i was running
> IntegrationTestMTTR#testRestartMaster() test was failing on put operation.
> Here is sequence of events from logs leading to failed put operation:
> Master restart
> {code}
> INFO [pool-5-thread-1] util.Shell: Executing full command [/usr/bin/ssh
> hnode2 "sudo -u hbase ps aux | grep proc_master | grep -v grep | tr -s ' ' |
> cut -d ' ' -f2 | xargs kill -s SIGKILL"]
> {code}
> Client trying to locate region for row=70efdf2ec9b086079795c442636b55fb-17
> (this is additional logging inspecting metaKey which is used to search
> hbase:meta )
> {code}
> 2016-01-15 10:26:05,169 INFO [HBaseWriterThread_9]
> client.ConnectionImplementation: metaKey inspection:
> table=IntegrationTestMTTRLoadTestTool row=
> 70efdf2ec9b086079795c442636b55fb-17 metaKey=
> IntegrationTestMTTRLoadTestTool,70efdf2ec9b086079795c442636b55fb-17,99
> {code}
> Client throwing TableNotFoundException (hbase:meta scan returned null)
> {code}
> 2016-01-15 10:32:58,154 INFO [HBaseWriterThread_5]
> client.ConnectionImplementation: regionInfo result is null:
> HBaseWriterThread_5 throwing TableNotFoundException logging details
> table=IntegrationTestMTTRLoadTestTool row=70efdf2ec9b086079795c442636b55fb-17
> metaKey=IntegrationTestMTTRLoadTestTool,70efdf2ec9b086079795c442636b55fb-17,99
> 2016-01-15 10:32:58,154 ERROR [HBaseWriterThread_5] client.AsyncProcess:
> Failed to get region location
> org.apache.hadoop.hbase.TableNotFoundException:
> IntegrationTestMTTRLoadTestTool
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegionInMeta(ConnectionImplementation.java:890)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:781)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:396)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:344)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:239)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:191)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:949)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:569)
> at
> org.apache.hadoop.hbase.util.MultiThreadedWriter$HBaseWriterThread.insert(MultiThreadedWriter.java:146)
> at
> org.apache.hadoop.hbase.util.MultiThreadedWriter$HBaseWriterThread.run(MultiThreadedWriter.java:111)
> {code}
> And as result we have failed insert operation:
> {code}
> 2016-01-15 10:32:58,179 ERROR [HBaseWriterThread_5] util.MultiThreadedWriter:
> Failed to insert: 17 after 60046ms; region information: cached:
> region=IntegrationTestMTTRLoadTestTool,6660,1452849956427.05b437185a9437f178726a55a29a79b7.,
> hostname=hnode4,16020,1452776418437, seqNum=5; cache is up to date; errors:
> exception from null for 70efdf2ec9b086079795c442636b55fb-17
> org.apache.hadoop.hbase.TableNotFoundException:
> IntegrationTestMTTRLoadTestTool
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegionInMeta(ConnectionImplementation.java:890)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:781)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:396)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:344)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:239)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:191)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:949)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:569)
> at
> org.apache.hadoop.hbase.util.MultiThreadedWriter$HBaseWriterThread.insert(MultiThreadedWriter.java:146)
> at
> org.apache.hadoop.hbase.util.MultiThreadedWriter$HBaseWriterThread.run(MultiThreadedWriter.java:111)
>
[jira] [Updated] (HBASE-18622) Mitigate API compatibility concerns between branch-1 and branch-2
[ https://issues.apache.org/jira/browse/HBASE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18622: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Mitigate API compatibility concerns between branch-1 and branch-2 > - > > Key: HBASE-18622 > URL: https://issues.apache.org/jira/browse/HBASE-18622 > Project: HBase > Issue Type: Bug > Components: API >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 3.0.0 > > Attachments: report.1.2_2.0.html.gz > > > This project is to do what [~apurtell] did in the issue "HBASE-18431 Mitigate > compatibility concerns between branch-1.3 and branch-1.4" only do it between > branch-1 and branch-2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19805:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 3.0.0
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
> newRegionSize = admin.getRegions(tn).size();
> }
> }
> {code}
> A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
> {code}
> static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv
> env,
> final ServerName regionLocation, final RegionInfo hri, boolean
> includeBestSplitRow)
> throws IOException {
> // TODO: There is no timeout on this controller. Set one!
> HBaseRpcController controller =
> env.getMasterServices().getClusterConnection().
> getRpcControllerFactory().newController();
> final AdminService.BlockingInterface admin =
>
> env.getMasterServices().getClusterConnection().getAdmin(regionLocation);
> {code}
> We don't validate that we have a non-null {{ServerName regionLocation}}.
[jira] [Updated] (HBASE-18799) Revisit the AccessControl
[ https://issues.apache.org/jira/browse/HBASE-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18799: -- Fix Version/s: (was: 2.0.0) 3.0.0 > Revisit the AccessControl > - > > Key: HBASE-18799 > URL: https://issues.apache.org/jira/browse/HBASE-18799 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0 > > > We are doing the gardening task for coprocessor. The hooks come and go, but > we should assure our access control is fine. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20333) break up shaded client into one with no Hadoop and one that's standalone
[ https://issues.apache.org/jira/browse/HBASE-20333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20333: -- Fix Version/s: (was: 2.0.0) 3.0.0 > break up shaded client into one with no Hadoop and one that's standalone > > > Key: HBASE-20333 > URL: https://issues.apache.org/jira/browse/HBASE-20333 > Project: HBase > Issue Type: Sub-task > Components: shading >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0 > > Attachments: HBASE-20333.WIP.0.patch > > > there are contexts where we want to stay out of our downstream users way wrt > dependencies, but they need more Hadoop classes than we provide. i.e. any > downstream client that wants to use both HBase and HDFS in their application, > or any non-MR YARN application. > Now that Hadoop also has shaded client artifacts for Hadoop 3, we're also > providing less incremental benefit by including our own rewritten Hadoop > classes to avoid downstream needing to pull in all of Hadoop's transitive > dependencies. > right now those users need to ensure that any jars from the Hadoop project > are loaded in the classpath prior to our shaded client jar. This is brittle > and prone to weird debugging trouble. > instead, we should have two artifacts: one that just lists Hadoop as a > prerequisite and one that still includes the rewritten-but-not-relocated > Hadoop classes. > We can then use docs to emphasize when each of these is appropriate to use. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20383) [AMv2] AssignmentManager: Failed transition XYZ is not OPEN
[
https://issues.apache.org/jira/browse/HBASE-20383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20383:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> [AMv2] AssignmentManager: Failed transition XYZ is not OPEN
> ---
>
> Key: HBASE-20383
> URL: https://issues.apache.org/jira/browse/HBASE-20383
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20383.master.001.patch
>
>
> Seeing a bunch of this testing 2.0.0:
> {code}
> 2018-04-10 13:57:09,430 WARN
> [RpcServer.default.FPBQ.Fifo.handler=46,queue=1,port=16000]
> assignment.AssignmentManager: Failed transition
>
>
> org.apache.hadoop.hbase.client.DoNotRetryRegionException:
> 19a2cd6f88abae0036415ee1ea041c2e is not OPEN
> at
> org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure.checkOnline(AbstractStateMachineTableProcedure.java:193)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:112)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:769)
>
>
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.updateRegionSplitTransition(AssignmentManager.java:911)
>
>
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportRegionStateTransition(AssignmentManager.java:819)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.reportRegionStateTransition(MasterRpcServices.java:1538)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:11093)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>
>
>at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {code}
> Looks like report back from Master OK'ing a split to go ahead but the split
> is already running. Figure how to shut these down. They are noisy at least.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20331) clean up shaded packaging for 2.0
[ https://issues.apache.org/jira/browse/HBASE-20331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20331: -- Fix Version/s: (was: 2.0.0) 3.0.0 > clean up shaded packaging for 2.0 > - > > Key: HBASE-20331 > URL: https://issues.apache.org/jira/browse/HBASE-20331 > Project: HBase > Issue Type: Umbrella > Components: Client, mapreduce, shading >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0 > > > polishing pass on shaded modules for 2.0 based on trying to use them in more > contexts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20281) [DOC] upgrade section needs an explanation of changes to Bucket Cache
[
https://issues.apache.org/jira/browse/HBASE-20281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20281:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> [DOC] upgrade section needs an explanation of changes to Bucket Cache
> -
>
> Key: HBASE-20281
> URL: https://issues.apache.org/jira/browse/HBASE-20281
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache, documentation
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0
>
>
> the first pass version of the upgrade docs has a brief note about the bucket
> cache changing:
> {quote}
> * hbase.bucketcache.combinedcache.enabled
> * hbase.bucketcache.ioengine no longer supports the 'heap' value.
> {quote}
> But the changes are substantial and warrant a section of their own under the
> "Changes of Note!" banner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20309) [hbck] bin/hbase hbck -boundaries throws exception on HBase 2
[
https://issues.apache.org/jira/browse/HBASE-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20309:
--
Fix Version/s: (was: 2.0.0)
3.0.0
> [hbck] bin/hbase hbck -boundaries throws exception on HBase 2
> -
>
> Key: HBASE-20309
> URL: https://issues.apache.org/jira/browse/HBASE-20309
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Fix For: 3.0.0
>
>
> hbck2 on HBase 2 fails with exception:
> {code:java}
> $ bin/hbase hbck -boundaries
> Exception in thread "main" java.util.NoSuchElementException: No value present
> at java.util.Optional.get(Optional.java:135)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.checkRegionBoundaries(HBaseFsck.java:865)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:777)
> at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:5059)
> at
> org.apache.hadoop.hbase.util.HBaseFsck$HBaseFsckTool.run(HBaseFsck.java:4860)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:4848){code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
